TDC Feedback on Design Model A
| What are the strengths of MODEL A? In other words, what aspects will best help NCBE meet the stated claim? | What are the potential weaknesses of MODEL A? In other words, what aspects of this model detract from the claim? | What is your overall opinion of MODEL A? |
| This model best demonstrates that a new lawyer has all of the knowledge and skills at one specific time that are typically required of a new lawyer. That statement is weakened, however, because the MPRE is administered at a different time.
I confess that I have a difficult time giving up on looking at the bar exam through the lens of history. I like this approach, but perhaps only because it is familiar. I like the all-or-nothing aspect of this exam. Lawyers deal with stress every day and that is one of the unstated, but clearly tested, portions of the bar exam. Some applicants fail because of the pressure and it is better that they learn how to manage that before they fail when it harms a client. |
The skills we expect new lawyers to have do not need to be tested and scored at the same time. A lawyer who knows the law and has the skills will not forget one or the other if the parts of the tests are administered and scored at different times.
This approach adds significant pressure to the test that may not be necessary. The exam missing important substantive material like wills and family law. Testing relationships (with clients and negotiation) seem too difficult and subjective to test in this format, so I would omit them from the skills that could be tested. If those skills are tested, perhaps trial advocacy should also be tested. |
I like it and would support it |
| I don’t see any strengths. This appears to be a vaguely updated version of what we have now. There may be underlying assumptions about changes in testing that we discussed before—such as changing the scope of doctrinal testing to focus on basic rules rather than technicalities. But as presented here, I don’t see why we would bother changing what we have now. | Students who fail one component but pass another would still have to take the entire test again. All testing is completed after graduation. This doesn’t look very different than the current bar. Sure, the methodology for the skills testing is interesting, but it comes across as a vaguely enhanced version of the MPT. | I do not like it and would not support it |
| It preserves the format and timing the legal community is accustomed to. Professors could continue teaching in the way they are accustomed to. Law firms could continue with the traditional rhythm of their internship programs. 2. It introduces incremental change (computer testing, short answers, new topics, etc.), which is easier for many stakeholders. 3. Incremental change may make it easier to study the effect of the changes. | As under our current format, candidates will be most successful if they can carve out a chunk of time outside of law school and employment to prepare for a comprehensive exam, and perhaps pay for bar prep. If they are unsuccessful on their first attempt, they will have a better chance of succeeding on a second attempt if they can carve out another chunk of time to prepare for the whole comprehensive test again, and perhaps pay for bar prep. Carving out this chunk of time, and paying for bar prep, is easier for those with more personal resources and harder for those with fewer personal resources. So, for some, the exam will test personal resources as much as it tests minimum competence. 2. Because of #1, the results of the exam may look racist. 3. I was persuaded by discussion in our group that none of the * lawyering skills was a skill that was needed to perform activities typically required of an entry level lawyer—even if these skills could be tested well, they shouldn’t be. If no new topics are introduced, the only change this Model offers appears to be that all components will be computer delivered, and some components will be short answers. | I can live with it and would support it |
| This is a recognized model; I like the split. It’s easier to administer. | There’s no compensatory aspect. | I can live with it and would support it |
| N/A | N/A | I do not like it and would not support it |
| I favor a compensatory model. Each applicant has strengths and weaknesses and generally I have not seen marked differences in scores between the MBE and the MEE/MPTs. Generally, an applicant who fares poorly on one, fares poorly on the other and vice versa. There’s little risk an applicant would be incompetent in one area. I’ve only had one student that scored extremely high on the MBE and performed extremely low on the MEE/MPT, but just enough to reach the cut score. This student was never interested in learning to write better and did not value the skill. He ultimately practiced with his father who took a similar attitude.
Taking the exam after graduation ensures that the students have attained the writing skills necessary to succeed on the exam. The MBE could be taken sooner, but I am not dead-set on a step-approach. A design that mimics the current set-up of the bar exam is what law schools and employers are accustomed to dealing with, as well as students. |
Because the design mimics the current set-up with a two-day high stakes exam, it is fraught with the same problems. Having both days back-to-back increases the stress exponentially versus having the two days separated by a year or more. It requires more intensive study, usually requiring not working or a significant reduction in working. Financial hardship ensues or the student must take out a higher interest private loan to pay for bar costs and living expenses. Many employers wait to hire until after bar results come out as many jurisdictions have limited temporary licensure rules. These employers don’t want to risk hiring someone who may have failed. | I like it and would support it |
| This is true of all my responses, but I like the MPRE portion specifically testing on professional responsibility issues and a possible expansion into a short answer component. I like the concept of narrowing the doctrinal law that would be tested, because I actually view that portion of the exam to be less important. The compensatory model is what I tend to lean toward, because people have different strengths. I would qualify that by saying that I am supportive of it to the extent the knowledge and skill areas are sufficiently related to support a compensatory design–I will leave that up to the experts to decide. Case studies and MPT-like components across the lawyering skills mentioned—Including those denoted with an asterisk—would be the strength of this model. I think expanding the types of questions and the way applicants receive the question material to more closely mimic the practice of law is a strength. For example, having simulated video clients and potentially a simulated time-lapse to hit different points of working on a client matter. To the extent that this model would mesh applying the core doctrinal law with the application of lawyering skills, I strongly support it. However, I do not like the idea of these basically being stand-alone components within a single exam—that is, I don’t want a doctrinal testing day and a skills testing day. | One of my concerns is that an exam offered exclusively after law school—at least on the same timeline as now—leads to delays in getting newly-licensed lawyers up and running. I think there needs to be less emphasis on strict knowledge of the law, because having a working memory of the law without relying on research is not how most attorneys practice and I think is one of the better criticisms of the current format. Having to study for an appropriate period of time to put all of that information into their heads and to be able to access it is one of the reasons I think we have to allow so much time between graduation and the exam. I think the better method may ultimately be to be testing the core doctrinal law in the context of the application of lawyering skills, so that a working knowledge of those doctrines helps in efficiently navigating the exam, but just spewing out lists of law is not what is being tested. I would actually advocate flipping the percentage of importance assigned to each section or even giving the lawyering skills portion more than 60% emphasis. For all of the models, I don’t know that a UBE testing client relationships and management is appropriate based on differences in people’s personalities as well as regional differences in how people interact. Negotiation is tricky, because that is necessarily a back and forth process, so I would worry about how that could be reliably tested. I would be interested to see how the computer-scored answer evaluation works, because at least a few years ago when I was looking into it, it seemed like there are ways to “game” the system and get a good score without really providing the correct content. | I like it and would support it |
| It’s the bar exam in the more traditional fashion that graders are familiar with in evaluating applicants. It’s testing for minimal competence in an environment which requires applicants to act and respond as attorneys often do in a higher stress environment than Models B or C. | Testing for minimal competence in this fashion has resulted in lower passing rates nationwide, however, I can live with this. | I can live with it and would support it |
| It better ensures that applicants have a full range of competencies necessary to practice law. | An overall score does not necessarily prove that someone maintains competency in all the areas tested. I.e. they could do well on the multiple choice answers but bomb the written writing and reasoning. | I can live with it and would support it |
| Like the current UBE it measures competency at the appropriate time, when the test taker is ready to commence practice. There is a lot to be said for requiring a candidate to review all learned in law school at this time to further an appreciation of how various subjects may interrelate in actual practice. | It defers all assessments until after law school, leading some students to continue when perhaps they should be redirected. | I like it and would support it |
| Tests appropriate areas of both doctrinal law and lawyering skills for NLL. An increase in the % now devoted to skills is positive. | Continues to test memorization, which is not how attorneys practice. No flexibility on when a student/graduate can take the examination. Must re-take all blue portions, even if they truly showed competency on the other. | I can live with it and would support it |
| Better measure of analytical skills (at least three years of legal education); relationship between theory and practice (doctrine and skills) done at single time; equating and reliability mechanisms much better available in this model | Too much information; heightened testing anxiety; no measurement of relationship of skill-building from core doctrinal knowledge to skills | I can live with it and would support it |
| Testing for legal research and writing. (I apologize for typos throughout, I’m on my phone.) | Selected Response for the Application of Lawyering Skills does not seem like a meaningful way to assess skills like this. Also, I think that testing for skills is more meaningful in terms of testing for minimum competency. Skills transcend practice groups whereas doctrinal law does not. If we have the MPRE and most states require Ethics CLEs throughout practice, testing for ethics is duplicative, and just adds another subject matter that is not necessary. | I can live with it and would support it |
| Tests a good balance of core doctrinal law and lawyering skills and provides a portable score for admission in several jurisdictions. | Does not allow individual to test in core doctrinal law during law school when knowledge is fresh. Also, allows candidates with strength in one area to compensate for weakness in another area—that’s good for candidates but not necessarily for the public / bar. | I can live with it and would support it |
| The primary strength of this model is that it is the closest approximation to the current model for the UBE, it is proven and involves no unknown risk. The apparent expansion of written/skill to include short answer seems relevant to testing discrete analytical skills using black letter law. | Its potential weaknesses are that it too closely resembles the current exam in terms of mode of testing, scoring, and timing. If we adopt a model so close to the status quo we will have not met the objective of this committee in rethinking ways to truly measure competency for practice. By presenting additional options for the timing of the exam, we give bar applicants more flexibility and more opportunities to retest, without impairing their ability to be practice ready soon after graduation. It is possible to achieve a more accurate and equitable outcome if the exam can be bifurcated if will allow students with a broader range of circumstances to maximize their resources. The percentage values assigned to the professional skills do not align with the feedback data on prioritizing writing as the core competency—40% is too low. | I do not like it and would not support it |
| Feels familiar. Content changes, but the test design seems most similar to status quo. | Seems strong. | I like it and would support it |
| One decision means no need to grapple with piecemeal approach to test talking and potential unfairness which might come from a conjunctive approach. This model demonstrates the ability to master large amounts of material for one high-stakes examination. This is a traditional approach which has worked well in my estimation. | It does negatively impact those examines who do not have the ability to process large amounts of information at one time. | I like it and would support it |
| MODEL A’s strength is that in allowing candidates to take the exam that is scored and evaluated collectively, those with jobs and families need only focus on studying at once, instead of in piecemeal. MODEL A would minimize interruption to the lives of applicants who are working parents or full-time workers.by offering predictability to their work schedule and reducing necessity to take multiple time-off to prepare and take the exam. | MODEL A closely tracks the status quo. This is not to say that the current exam approach is flawed; but other models need to be evaluated to fully appreciate the strength or weakness of MODEL A. | I can live with it and would support it |
| Model A seems to offer a robust assurance that anyone who passes is both minimally competent in areas of importance to new lawyers. | The weakness is that it may work too well—in other words, given the likely volume, it also necessarily tests skills that are helpful, but not strictly necessary for or related to being a competent lawyer, such as ability to memorize quickly, ability to test well and for long periods of time (if a multi-day event). Also single decision makes the test higher stakes than necessary. | I do not like it and would not support it |
| This model deviates the least from the current model and would create the least surprise for law students when announced. As a one-decision test, it helps to ensure that all new lawyers possess the knowledge of all tested subject areas at the time of the test and allows examiners to assess the overall preparedness of the candidate. | The compensatory model is a one-decision high-stakes test that will continue to require candidates who must re-take the exam to study for every component—even one on which they have already demonstrated competence—again. | I can live with it and would support it |
| There is less chance for human error in grading. | I’m not sure I agree with computer scoring. Short answer format may not adequately test examinees knowledge of the subject. | I can live with it and would support it |
TDC Feedback on Design Model B
| What are the strengths of MODEL B? In other words, what aspects will best help NCBE meet the stated claim? | What are the potential weaknesses of MODEL B? In other words, what aspects of this model detract from the claim? | What is your overall opinion of MODEL B? |
|---|---|---|
| This form allows for separate testing of the skills needed by new lawyers. This may be a fairer assessment of skills because it relieves the pressure of an all-or-nothing exam. | This form moves the pressure from after law school to during law school. It changes the focus of law school from education for three (or four) years to possibly cramming for two years and then taking the bar exam. I would hate for law students to miss out on valuable law school opportunities because they are focused on taking part of the bar exam. This model does not allow the skills and abilities of new lawyers to offset one another. Someone who can memorize the law should be able to use that skill to offset lesser writing abilities. The skills of a lawyer should not be compartmentalized. The exam missing important substantive material like wills and family law. Testing relationships (with clients and negotiation) seem too difficult and subjective to test in this format, so I would omit them from the skills that could be tested. If those skills are tested, perhaps trial advocacy should also be tested. Does this format create a problem for score reliability? Will each section be valid if taken with different groups of applicants or over an extended period of time? I could live with this model, but it would be my third choice among the options. |
I can live with it and would support it |
| Students who pass one component but fail another would only have to re-take the component that they failed. Some testing done during law school, which I think is great. | I don’t understand at all why the skills testing would need to be after graduation. If you are going to have some testing done before graduation, why not do all of it? I see no reason why students couldn’t do the skills testing during their third year, especially because this kind of testing wouldn’t require cramming a bunch of doctrinal subjects. | I can live with it and would support it |
| 1. It reduces the chunk of time that candidates need to carve out in order to prepare for the exam. If success on the exam depends less on a candidate’s personal resources, then it is likely that more candidates will pass the first time. 2. Because candidates could delay the first component until after law school, students, schools, and law firms could continue with their traditional schedules of classes and internships if they wanted to. |
It may lead to a 2-track path to licensure that splits candidates along lines that appear to be racist or classist. | I like it and would support it |
| This meets the needs of many students who are ready to take a portion of the exam. It allows them to preserve a passing score. | Law professors have raised concerns students would focus on the bar and not the classes. | I can live with it and would support it |
| N/A | N/A | I like it and would support it |
| Separating the two parts allows the student to study for only one part at a time. Theoretically by focusing their study on only one part, their performance on that part should be better than if they were studying on both parts. Taking the first part after 60 hours of law school credit is good timing and allows students to take it several times before graduation—I am assuming there would be more than just two test dates as it would be computerized delivery. | I do not like conjunctive scoring. People have different strengths and the ability to answer multiple choice questions is not a skill attorneys have to have. It’s the underlying analysis skills. But, I could live with it because of the ability to take the MBE multiple times and to start taking it after 60 hours. | I can live with it and would support it |
| I do like that this is considering different delivery methods, and the strength of it is the application of lawyering skills component. The PR component is important, as indicated in my response to model A. As far as the areas tested, I like them. | I do not like the concept of having a component of the exam where memorization is required, except for potentially with the professional responsibility component, because that is what all attorneys need to have a good sense of at all times. To the extent the application of core doctrinal law would be based on the memorize and access in your brain only, I do not generally like it. I think it is inefficient to test skills outside of the context of application of core doctrinal law. Comparing this to exams for doctors, I think there is less of a chance that an attorney will ever need to pull a business organization doctrine or real property doctrine from memory in order to help a client. Put another way, there is never an emergency situation where a split-second recitation of doctrinal law is necessary. I much prefer crafting an exam that would layer those two things together, which would require an applicant to come in with a good working knowledge of core doctrinal law, but would allow them to get into the intricacies and exceptions to exceptions by using a provided library of information. For all of the models, I don’t know that a UBE testing client relationships and management is appropriate based on differences in people’s personalities as well as regional differences in how people interact. Negotiation is tricky, because that is necessarily a back and forth process, so I would worry about how that could be reliably tested. | I do not like it and would not support it |
| Testing for minimal competence under this Model should offer applicants a more comfortable method of demonstrating their abilities without the pressure one might feel under Model A. I think also, knowing the core law at an earlier stage will enable applicants to better develop their lawyering skills. | I personally don’t see a downside to this Model. | I like it and would support it |
| Allows applicants the potential option of passing the first component while in law school while the information is fresh in their minds. Following graduation, they would only have to take the second portion if they were successful on the first portion. | By allowing applicants to take the first part of the test while in Law School, it would distract from their other Law School Courses. At one time my jurisdiction had this allowance, and it was a problem so the Court did away with it. | I can live with it and would support it |
| It tests much of the doctrinal law closer to the time it is studied in law school, which should decrease the amount of preparation required for the post-graduation exam. It increases the importance of skills by having a separate pass/fail decision for each component. | It would not be offered until after the second year of law school and candidates taking it at that time would not have as much time available for summer employment or internships which can be important elements in training them for actual practice. It also is one more year before any assessment. The doctrinal assessment would be solely by multiple choice questions, which tend to discriminate slightly against women.
I also have concerns as to whether the second assessment could be adequately equated to make it fair from assessment to assessment (although all 3 models do indicate there would be computer-scored short answers included) and whether one day of testing would be adequate for it so as not to extend the total amount of testing time. I was disappointed that the models presented were more similar than varied. All 3 retain the MPRE as a separate component, which I support. All 3 have the same division between an examination of core doctrinal law and an examination of lawyering skills. Only B has eligibility before graduation, and that requires waiting until after the second year of law school. None of the models included the possibility for the exam or some portion of it being open-book. If there are to be 2 components, I would rather have the first component test only contract law, criminal law, torts, real property and business organizations after the first year of law school and include civil procedure, criminal procedure, evidence and constitutional law in the second component after law school. (I am not totally certain about the proper placement of constitutional law and business organizations.) This would include more subject matter appropriate for MC questions, making for better equating, but might make it even more difficult to do a one day test. |
I can live with it and would support it |
| Tests appropriate areas of both doctrinal law and lawyering skills for NLL. An increase in the % now devoted to skills is positive. Also: Gives flexibility on testing times; allows examinees to take the doctrinal portion closer to the time they’ve taken the related classes. Allows someone to retake only the portion failed, thus reducing costs/debt load. | Continues to test memorization, which is not how attorneys practice. Could make students choose between studying for and taking the exam and foregoing another opportunity (job, clinic, journal, etc.). Could detract from the 3L year, depending on when the test were offered. | I like it and would support it |
| breaks up amount of testing information; operates as a formative tool for law school stakeholders; gives pathway to examinees from one step to the next; measures knowledge of content as it relates to skills better with at least one year of upper level legal education between the parts | timing; equating (second part performance to first part) and instrument reliability; resources in grading the second component | I like it and would support it |
| Permitting eligible students to have the option to take prior to graduation. This is not related to the goal BUT I think its practical and would be beneficial to applicants and employers. The Conjunctive model I think achieves the goal of testing for minimum competency better. Especially with dividing the test between Doctrinal Law and Lawyering skills. If an applicant passes one section they’ve demonstrated min competency in that section and having them retake it is timely, costly, and does not pursue to end goal. | Selected response for application of lawyering skills. If we have the MPRE and most states require Ethics CLEs throughout practice, testing for ethics is duplicative, and just adds another subject matter that is not necessary. | I can live with it and would support it |
| Permits test of core doctrinal law during law school when learning is fresh. Also, tests a good balance of core doctrinal law and lawyering skills. Finally, requires minimum score on both core doctrinal law and lawyering skills. | Does not provide a portable score for admission in several jurisdictions. | I can live with it and would support it |
| Affording candidates the opportunity to take one component of the exam after the completion of a set number of hours beyond the 1L curriculum of law school credits is ideal. It showcases the responsive and collaborative efforts of this committee. Further, it is responsive to the needs for greater flexibility in licensure exams. It is also the most compassionate of the three models: by allowing applicants to test in law school while the subject material is fresher in their memory, it may spare some applicants the need to undertake costly bar preparation courses.
Candidates who are not successful on the first component of the exam, could have multiple opportunities to retest before graduation. This option allows the NCBE to fulfill its mission in identifying an accurate measure of competence and is also the most likely to be well received by law students and law faculty. This model is highly adaptive and shows great innovation on the part of the NCBE. The Proposal B allows students to participate in a competency exam that better reflects ability, and gives less primacy to the high stakes winner takes all approach. The broader range of mechanisms to practice skills is included (with the exception below) and better aligns with the data regarding relevance to entering the profession as a competent new attorney. |
Several members of the review team noted that the inclusion of client interviewing and management is highly subjective and lends itself to bias. Perhaps if you were testing the students ability to demonstrate a discreet skill, such as to identify the information or questions that needed to formulate a viable theory of litigation or other legal objective. | I like it and would support it |
| The best part is students can have clarity about which part they passed or failed and could retake each part. | It makes every part equal in weight. The eligibility after 60 credits is big problem and would be too disruptive. If anything, that’s the part that actually requires study time. In some ways the application of lawyering skills could probably be administered after 60 credits with less disruption. | I do not like it and would not support it |
| It allows for examiners to take the core substantive examination closer in time to the initial studying of that subject-matter. | I believe the ability to “zero in” on components not previously passed creates an unfair advantage for those taking limited components compared to those who are taking multiple components. Further, in jurisdictions which allow law office study as preparation for the bar examination, difficulty would be created in determining when those applicants would be eligible to sit for the initial component. | I do not like it and would not support it |
| MODEL B’s strength is that it optionally allows candidates to spread out their study over the course of the law school—one during law school and one upon graduation. This approach allows the state bar to focus the “earlier” component on assessing whether law students can understand and apply important concepts of the legal tenets (from contract to constitutional law) to the practice of law while offering a path way to adjusting the “later” component to concentrate on key areas (e.g., practical skills and research analysis) that are more reflective of their competency to practice as a lawyer. | Competency is tested at the time when a respective component is scored and evaluated. Where a particular component is tested earlier on in the law school cycle, it is not clear whether evaluation at this early stage would fully demonstrate a candidate’s competency upon graduation.
Separately, MODEL B will require each state bar to prepare for multiple administrations of different components every year. While the impact might be minimal for state bars with a limited number of applicants, MODEL B would undoubtedly impose significant strain on human resources for states such as California and New York where more than 10,000 candidates apply to take the bar exam. |
I like it and would support it |
| It allows for the evaluation and assessment of relevant competencies at times that make sense. Allowing students to test core doctrinal law after 60 credit hours would create an expectation of what law schools should/must deliver while they are still in the building to demand/request changes if they feel that the curriculum is not adequately cognizant of the bar. Also, by separating the components which can be taken independently, the model seems to envision and normalize the potential for multiple attempts, thus reducing the stakes to some extent (though it will of course remain a high stakes exam). | It would be hard to keep track of and support where every student is in the bar passage process. In other words, because paths can begin to diverge as soon as the 3L year, it could be hard to tailor bar support to 2Ls, 3Ls, and graduates—especially if each component is offered multiple times per year (as with the MPRE). However, if the stakes are reduced enough (as with the MPRE), that would be a fair trade-off. | I can live with it and would support it |
| The conjunctive model opens the possibility of preparing for each component separately such that they can demonstrate their competence in each area on a exam day for which they have prepared specifically for that component. For those who must re-take a given component, it encourages candidates to re-focus their attention on just that aspect of the bar exam. | Introducing the possibility to take 2 of the 3 components of the bar exam during law school will affect the curricular choices of law schools and will likely affect the content of individual “bar courses” that will, over time, become pressured to act as bar-prep courses rather than preparation for the long career of a lawyer. For first generation students or others for whom law school is a particularly challenging for reasons other than potential competence. Unfortunately, it may be that these students are more likely than their peers to need to enter the work-force as soon as possible, such that they will have strong incentives to take as many components of the bar exam as possible during law school. | I do not like it and would not support it |
| Same as A. | Same as B and I am further not sure I support taking before law school graduation and testing as separate pass/fail components. | I can live with it and would support it |
TDC Feedback on Design Model C
| What are the strengths of MODEL C? In other words, what aspects will best help NCBE meet the stated claim? | What are the potential weaknesses of MODEL C? In other words, what aspects of this model detract from the claim? | What is your overall opinion of MODEL C? |
|---|---|---|
| The compensatory grading model allows for strengths and weaknesses to be graded together.
This form allows for separate testing of the skills needed by new lawyers. This may be a fairer assessment of skills because it relieves the pressure of an all-or-nothing exam. This form is better than Model B, however, because it also allows varying strengths and weaknesses to be considered together. |
This form could relieve some of the pressure by allowing the parts of the exam to be completed separately. The pressure, however, mimics what lawyers will feel in their professional lives, and there is value in that.
The multiple administrations could lead to it taking longer for applicants to take and pass the exam. The option to take the exam in parts may cause applicants to decide to focus on one part at a time and unnecessarily delay their passing the entire exam. The exam missing important substantive material like wills and family law. Testing relationships (with clients and negotiation) seem too difficult and subjective to test in this format, so I would omit them from the skills that could be tested. If those skills are tested, perhaps trial advocacy should also be tested. Does this format create a problem for score reliability? Will each section be valid if taken with different groups of applicants or over an extended period of time? |
I can live with it and would support it |
| Like Model A, I don’t see many advantages. I don’t feel strongly enough about whether each component should have minimum scores to care much about the difference. | Students who fail one component but pass another would still have to take the entire test again. All testing is completed after graduation. | I do not like it and would not support it |
| A candidate who passed only one component would have a better shot at passing the other component the second time around. | It’s hard to evaluate this Model without knowing more about the content and timing of each component. It could simply exacerbate the problems of Model A by requiring candidates to carve out 2 chunks of time to prepare for 2 post-law school exams. It could lead to a 2-track path to licensure, where candidates with sufficient personal resources will study for and pass both components at the same time, while candidates with fewer personal resources are advised to focus on one component at a time. Clearly a candidate who passed only one component would have a better shot at passing the other component the second time around, but this Model doesn’t necessarily help level the playing field for that first attempt. This Model may just introduce a pretty big change without any real benefit. | I do not like it and would not support it |
| I like the compensatory aspect. I like that students can prepare for and test separately and only test on the portion that they don’t pass. | I’d like them to be able to test on some aspect of it in law school. I do not like the videos of clients in ANY of the tests. Client counseling is highly subjective, subject to bias and can vary widely based on culture of where you live. | I like it and would support it |
| N/A | N/A | I do not like it and would not support it |
| I like the idea of having a minimum performance on each and letting students choose whether to take one or both components—similar to how the CPA test is done now. That will require jurisdictions to modify temporary licensure rules. And, I like that it is still compensatory model. | I’m concerned that if both are after law school, but not at the same time, it will just prolong the process for applicants. Their financial insecurity would be elongated rather than shortened. Or are you picturing something like one a month? That might work.
A minimum performance expectation is not a bad idea, but I would be concerned that a jurisdiction would set it so close to their cut score that it would mean little difference. |
I like it and would support it |
| I do like that this is considering different delivery methods, and the strength of it is the application of lawyering skills component. The PR component is important, as indicated in my response to model A. As far as the areas tested, I like them. The idea of an overall pass/fail decision is intriguing to me for the reasons stated in my response to Model A. | I do not like the concept of having a component of the exam where memorization is required, except for potentially with the professional responsibility component, because that is what all attorneys need to have a good sense of at all times. To the extent the application of core doctrinal law would be based on the memorize and access in your brain only, I do not generally like it. I think it is inefficient to test skills outside of the context of application of core doctrinal law. Comparing this to exams for doctors, I think there is less of a chance that an attorney will ever need to pull a business organization doctrine or real property doctrine from memory in order to help a client. Put another way, there is never an emergency situation where a split-second recitation of doctrinal law is necessary. I much prefer crafting an exam that would layer those two things together, which would require an applicant to come in with a good working knowledge of core doctrinal law, but would allow them to get into the intricacies and exceptions to exceptions by using a provided library of information. I would actually advocate flipping the percentage of importance assigned to each section or even giving the lawyering skills portion more than 60% emphasis. For all of the models, I don’t know that a UBE testing client relationships and management is appropriate based on differences in people’s personalities as well as regional differences in how people interact. Negotiation is tricky, because that is necessarily a back and forth process, so I would worry about how that could be reliably tested. | I do not like it and would not support it |
| Like Model B, I like the idea that core components can be tested separately and retested if needed. | I feel this hybrid system would be better served if the core components were evaluated earlier rather than after law school so as to better prepare applicants for demonstrating their lawyering skills. | I can live with it and would support it |
| This is the best of both models. It requires a minimum performance on BOTH components. Also, they can test each portion separately and just re-take the portion they failed. | Applicants who have to retake a component may find it costly. With a regular bar exam, you pay for the whole exam one time, depending on costs of the components this could likely be a financial hardship. | I like it and would support it |
| I see none. | It has all of the weaknesses of both A & B. In addition, I have concerns about compensatory grading between two exams given at different times to different groups.
Some additional general comments: I am presently opposed to including any of the skills denoted with an asterisk in any of the models. There is too much danger of bias, both based on sex and ethnicity and based on any personal predilections of the grader. To some extent these skills tend to reflect an individual’s personality. And different methods may work better for different people. To the extent these skills may be acquired, they are more likely to be acquired through experience and mentoring after law school than acquired through school learning. Also, they may not be skills required for all lawyers, although I would agree they should be possessed by those lawyers dealing directly with the public. I also have concerns regarding the statement, “All components will be computer delivered.” With all the remote testing now occurring, I have been hearing a lot about the limits that should be placed on time looking at a screen and problems for those requiring non-standard testing accommodations. It also seems that answering an MPT-type question is made more difficult because of the need to move back and forth among the answer, the file and the library. |
I do not like it and would not support it |
| Tests appropriate areas of both doctrinal law and lawyering skills for NLL. An increase in the % now devoted to skills is positive. Allows someone to retake only the portion failed, thus reducing costs/debt load. Does not require students to choose between taking part of the bar and law-school opportunities or 2L summer jobs. | Continues to test memorization, which is not how attorneys practice. Could delay licensure since it’s not clear whether the exams would be offered at the same time. That those weaker in one area may have a more difficult time passing overall. Testing on key areas occurs 3-4 years after the student has studied that topic. | I can live with it and would support it |
| ability to repeat only one part if not successful on one | planning for applicants on when to take it; requiring the first and the second part after law school unnecessarily delays licensure—their content knowledge could be assessed after 60 credits given this list of subjects; it assumes skills don’t build off knowledge (they do) | I do not like it and would not support it |
| The main plus for me is the ability to retake components. | Selected response in application of lawyering skills. If we have the MPRE and most states require Ethics CLEs throughout practice, testing for ethics is duplicative, and just adds another subject matter that is not necessary. | I like it and would support it |
| Tests a good balance of core doctrinal law and lawyering skills and requires minimum score on both core doctrinal law and lawyering skills. May provide a portable score for admission in several jurisdictions (but not if components taken at different times). | Does not allow individual to test in core doctrinal law during law school when knowledge is fresh and may not provide a portable score for admission in several jurisdictions (if components taken at different times). | I can live with it and would support it |
| I think that the hybrid grading allows for students to establish a base line competence, and yet recognizes that a student to balance an under-performance in one skills are could offset the other. | I cannot support this model unless at least one component is available to administered during law school. The portion testing an applicant’s ability to apply core doctrinal law is well suited to be tested while an applicant is still enrolled in law school and has completed at least 60-75 hours of legal education. Although some may elect to test after law school graduation, the option is key. The separately scored exam components are less significant if all components must be taken together. Option B is far superior to this option. | I can live with it and would support it |
| I like the fact that students could retake individual parts but that it’s still compensatory. This isn’t too far from current practice, but certainly is different and would be seen as a move forward. | None | I like it and would support it |
| This model has some of the good components of model A, with collective scoring and evaluation and a minimum performance evaluation for each section. | This model allows for an examinee to take the components separately and thus suffers from the same weakness as model B, giving the examinee who is taking a reduced number of components an unfair advantage over those who are testing on both components at once. | I do not like it and would not support it |
| The strength of MODEL C is that it enforces a minimum performance of the “earlier component” before allowing candidates to take the “later” component with benefits akin to those observed in Model B. | MODEL C’s biggest weakness is that it creates unpredictability and a winding road to passing the bar exam. Not allowing an applicant to take the “later” component until the “earlier” component has been satisfied could pave an arduous path to licensure. This impact disproportionally harms applicants who are working parents or working students.
Also, testing two separate components after graduation is akin to administering two bar exams upon graduation. This requires significant resources on large state bars to accommodate both components in such a short time while forcing candidates to prolong their study before they can formally commence their new employment (or study during employment). |
I do not like it and would not support it |
| I may be misunderstanding, but the best thing I see about this model is the “minimum performance expectation”—as that is precisely what the NCBE is trying to achieve in its stated charge. I would respectfully add here that I think that should be an express part of all Models (A, B, and C). | As with Model B, the decoupling of the components—even if all taken post-graduation—complicates institutional bar support (thus graduates who should be attorneys may fail the exam for reasons unrelated to their core legal knowledge and abilities). | I can live with it and would support it |
| The weighted hybrid model opens the possibility of preparing for each component separately such that they can demonstrate their competence in each area on a exam day for which they have prepared specifically for that component. For those who must re-take a given component, it encourages candidates to re-focus their attention on just that aspect of the bar exam. The weighted hybrid has the added benefit of allowing candidates to tie the components together such that strength on one component can assist weaker performance on another. Ultimately, I am not convinced that these differences have as much significance as it might seem. | My largest concern with this model would lie in the cut scores in particular jurisdictions rather than with the model design. The potential to set cut scores either too high or too low could have negative gate-keeping effects. This is true of all the models, of course, and does not go directly to the question of test design. | I like it and would support it |
| This model seems to be the most comprehensive test model. | As with B, I am not sure I would support re-testing separate components. | I like it and would support it |
