Best practices for high-stakes licensure examinations include periodic review of exam content and design. Consistent with that standard, the Testing Task Force undertook a three-year, comprehensive, empirical study to ensure that the bar examination continues to assess the minimum competencies required of newly licensed lawyers in an evolving legal profession, and to determine how those competencies should be assessed. This overview sets out the Task Force’s recommendations for the next generation of the bar examination, which were approved by NCBE’s Board of Trustees in January 2021. A tremendous amount of work will be required to implement the recommendations and transition to administration of the new examination. At the end of this overview, we list some of the steps involved in implementation, a process that is anticipated to take up to four to five years.
This study has been approached systematically, transparently, and collaboratively—unconstrained by the current bar exam’s content and design—with qualitative and quantitative research conducted by external expert consultants in three phases. During Phase 1, we held a series of listening sessions across the country where more than 400 stakeholders from bar admission agencies, the legal academy, and the legal profession provided their views about the current bar exam and ideas for how it could be changed. Phase 2 consisted of a nationwide practice analysis survey completed by nearly 15,000 lawyers that provided a rich set of data on the work performed by newly licensed lawyers and the knowledge and skills they need to perform that work. In Phase 3, we convened two committees composed of bar admission representatives, legal educators, and practitioners who applied their professional experience and judgment to the data produced by Phases 1 and 2 to provide input on what content should be tested on the bar exam and when and how that content should be assessed. The results from Phases 1, 2, and 3 of our study are detailed in individual reports.
Based on this extensive research, the Task Force arrived at high-level decisions about the content and the design for the next generation of the bar examination. Those decisions are founded on the principle that the purpose of the bar exam is
to protect the public by helping to ensure that those who are newly licensed possess the minimum knowledge and skills to perform activities typically required of an entry-level lawyer.
Our decisions were guided by the prevailing views expressed by stakeholders during Phases 1 and 3: that the bar exam should test fewer subjects and should test less broadly and deeply within the subjects covered, that greater emphasis should be placed on assessment of lawyering skills to better reflect real-world practice and the types of activities newly licensed lawyers perform, that the exam should remain affordable, that fairness and accessibility for all candidates must continue to be ensured, and that the portability of Uniform Bar Exam (UBE) scores should be maintained. In those instances where there weren’t prevailing stakeholder views, our decisions were based on what will best ensure that the exam’s content and design achieve the purpose described above and meet the standards required of high-stakes licensure exams by the Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014). Finally, our decisions reflect the fact that newly licensed lawyers receive a general license to practice law, suggesting that the licensure exam should assess knowledge and skills that are of foundational importance and are common to numerous practice areas.
As explained in more detail below, these recommendations specify the use of an integrated examination that measures both knowledge and skills through a mix of item formats. The exam will be offered two times per year as a summative event and delivered by computer. Compensatory scoring will be used to produce a single combined score for making admission decisions.
Snapshot of Next Generation of the Bar Examination
Integrated Examination
The Task Force recommends the creation of an integrated examination that assesses both knowledge and skills holistically, using both stand-alone questions and item sets, as well as a combination of item formats (e.g., selected-response, short-answer, and extended constructed-response items). An item set is a collection of test questions based on a single scenario or stimulus such that the questions pertaining to that scenario are developed and presented as a unit. Item sets can be assembled so that all items within a set are either of the same format or of different formats.
An integrated exam reflects a fundamental shift from the current Multistate Bar Examination (MBE), Multistate Essay Examination (MEE), and Multistate Performance Test (MPT), which are discrete components covering specific knowledge and skills and using single items of the same format within each component.
An integrated exam permits use of scenarios that are representative of real-world types of legal problems that newly licensed lawyers encounter in practice and provides an authentic assessment of lawyering skills. The use of item sets also provides efficiencies in exam development and administration, in that a single scenario applies to multiple items.
Scoring
A single combined score for making admission decisions, based upon a compensatory scoring model, is consistent with the use of an integrated exam and with the interconnected nature of the competencies being measured. Compensatory scoring reflects the candidate’s overall proficiency and allows areas of strength to compensate for areas of weakness and generally is considered fairer to candidates than conjunctive scoring models.
Content to Be Assessed
The following Foundational Concepts & Principles (FC&P) and Foundational Skills are recommended for inclusion on the new bar exam. Note that the FC&P are legal subjects that are common to numerous practice areas, which is consistent with the regulatory framework of a general license.
Foundational Concepts and Principles
- Civil Procedure (including constitutional protections and proceedings before administrative agencies)
- Contract Law (including Art. 2 of the UCC)
- Evidence
- Torts
- Business Associations (including Agency)
- Constitutional Law (excluding principles covered under Civil Procedure and Criminal Law)
- Criminal Law and Constitutional Protections Impacting Criminal Proceedings (excluding coverage of criminal procedure beyond constitutional protections)
- Real Property
- Family Law*
*In October 2023, NCBE announced that Family Law would be added to the list of foundational concepts and principles starting with the July 2028 NextGen bar exam.
Foundational Skills
- Legal Research
- Legal Writing
- Issue Spotting and Analysis
- Investigation and Evaluation
- Client Counseling and Advising
- Negotiation and Dispute Resolution
- Client Relationship and Management
Implementation of the final recommendations will include a process for developing content specifications to ensure that the depth and breadth of coverage of the FC&P is carefully aligned with minimum competence for entry-level practice. Content specifications guide development of test questions and articulate the scope of coverage to provide notice to candidates of what may be tested.
Foundational Skills may be assessed in the context of the FC&P listed above as well as in other legal contexts. Whenever Foundational Skills are assessed in a legal context other than the FC&P, appropriate legal resources (e.g., statutes, cases, rules) will be provided to candidates. As an example, Professional Responsibility or Family Law may serve as the context for the assessment of Foundational Skills with appropriate legal resources being provided.
The list of Foundational Skills includes some skills that might be thought of as performance skills, such as client interviewing and negotiation. To ensure fairness, those skills that can be objectively measured will be assessed using uniform text- or video-based scenarios that require candidates to construct a written response or select the correct response. Of course, it is necessary to also consider accessibility issues in determining appropriate methods for assessing skills.
Timing, Mode, and Frequency of Test Administration
The Task Force recommends that the bar exam be given as a single event at or near the point of licensure. This timing is most consistent with the purpose of the bar exam in that it places measurement of minimum competence as close in time to the award of a license as possible. Jurisdictions could still permit applicants to test in their final semester of law school, as is currently the case. Single-event testing allows more options for equating and scaling and is also more consistent with the use of an integrated exam.
A single-event approach will avoid concerns expressed by some stakeholders about a multi-event approach, where components of the exam would be administered at separate times. Those potential concerns included interfering with internship opportunities, impacting law school curricula, adding the stress of taking a high-stakes exam during law school, creating multiple “hurdles” for admission, and potentially increasing costs for candidates to prepare for and travel to multiple administrations of the exam. One of the primary reasons some stakeholders favored multi-event testing was to permit testing of legal doctrine closer in time to when students learned the content in law school. The Task Force concluded that the use of an integrated exam with an increased emphasis on assessing skills and more limited depth and breadth of coverage of doctrine addresses the underlying reasons some stakeholders favored multi-event testing.
The next generation of the bar exam will be a computer-based test, administered either on candidates’ laptops in jurisdiction-managed facilities and/or at computer testing centers managed by a suitable vendor. If possible, the length of the exam will be reduced, but this will be done only if the necessary validity and reliability of scores can be maintained. The exam will continue to be offered two times each year.
Next Steps
We anticipate that the implementation process to develop and deliver the new exam will take up to four to five years, which will allow time for notice to candidates of what to expect and for law schools to help students prepare. We will continue to collaborate with stakeholders as we work to build the new exam from this road map. Some of the major steps of implementation will include
✔ developing content specifications identifying scope of coverage;
✔ drafting new types of questions for integrated testing of knowledge and skills;
✔ ensuring accessibility for candidates with disabilities;
✔ field-testing new item formats and new exam content;
✔ conducting analyses and review to ensure fairness for diverse populations of candidates;
✔ evaluating options for computer delivery of the exam;
✔ establishing scoring processes and psychometric methods for equating/scaling scores;
✔ developing test administration policies and procedures;
✔ assisting jurisdictions to prepare and supporting them in activities such as setting passing score requirements and amending rules to align with changes to the exam; and
✔ providing study materials and sample test questions to help candidates prepare.
