THE BAR EXAM SUB-COMMITTEE
The Bar Exam Sub-Committee reviewed the Conclave’s findings regarding the bar examination:
• The Supreme Court and the Board of Bar Examiners should be commended for proposing changes in the Bar examination. We recommend that the organized bar, acting through groups such as the Board of Bar Examiners and/or the Commission on Professionalism, attempt to examine the idea of reducing the number of subjects required to prepare for the Bar exam, and substitute a performance examination component to the existing Bar exam, such as that being used in California and is being considered by the National Conference of Bar Examiners. The performance component would encourage the development of more skills courses within the law schools.
• We recommend re-examining and reconsidering the structure and utility of the multi-state component of the Bar exam.
• The Ohio Supreme Court and the Board of Bar Examiners should conduct a review of the ongoing validity of the Ohio Bar examination. The resources expended in preparation, taking and administrating the Bar exam might better be used for practical skills instruction and the like.40
A. The Present Ohio Bar Examination41
Each state has established standards for admission to the practice of law. While these standards vary from state to state, as the MacCrate Report noted, virtually every state has two basic requirements: passage of a bar examination and demonstration that the applicant possesses the necessary character, fitness, and moral qualifications to practice the profession of law. As noted earlier, the jurisdiction and authority to regulate admission to the bar in Ohio are vested in the Supreme Court of Ohio. Such power is exercised through Rule I of the Rules for the Government of the Bar, which is promulgated by the Court. Rule I details the general criteria for admission,42 vests the responsibility for administering the examination in the Board of Bar Examiners,43 and prescribes the content of the examination.44
The bar examination is designed to test for the essential skills and knowledge that the Supreme Court of Ohio and the Board of Bar Examiners have determined are important for admission to the practice of law. These include substantive legal knowledge, analytical ability, effective communication, principled advocacy, and management of legal work.
At the present time the examination is a two-and-a-half day test given twice a year -- in February and July. It consists of one day of the Multistate Bar Examination (MBE), prepared by the National Conference of Bar Examiners, and one-and-one-half days of essay questions, prepared by members of the Ohio Board of Bar Examiners.45
The MBE is a multiple-choice exam that is required by almost all states. It has been in use in Ohio since 1972. It is a six-hour test administered in two-parts: 100 questions in a morning session and 100 questions in an afternoon session. The subject areas tested include constitutional law, contracts, criminal law, evidence, real property, and torts. By its very nature as an exam for use by any state, the MBE tests an applicant’s knowledge of general legal principles, not the law of a particular jurisdiction. It is designed to test such aptitudes as reading skills, identification of legal issues, separation of relevant from irrelevant facts, knowledge and application of substantive legal principles, and identification of the best solution to a problem or issue.
There may be a misperception or misunderstanding regarding the MBE. Because it is a multiple choice test, some persons question its usefulness in distinguishing those who are qualified to practice law from those who are not -- perhaps the principal purpose of a bar examination. This misunderstanding is primarily due to an assumption that a multiple choice exam is of limited usefulness in testing skills and aptitudes and is really just a measure of an individual’s ability to rotely memorize material. However, a properly developed and validated multiple choice test can accomplish more than that. The MBE is the result of a sophisticated development process involving national drafting committees composed of distinguished and renowned practitioners and academics. It undergoes continual intense review and revision.
Moreover, such a test has a number of advantages. It can allow testing on a substantial amount of material in one day of testing. Such broad testing increases the reliability of the overall examination. Further, the MBE can be objectively graded using a computer. Grading of the MBE includes an early item analysis to flag test questions that are poorly drafted and questions that are answered incorrectly by an unusually large number of candidates who score well on other questions. Finally, test scores can be equated to adjust for differences in difficulty from one administration of the MBE to the next and the MBE equating process then can be used to equate essay scores.
Numerous studies and statistical analyses have demonstrated the validity of the MBE. It has been found to have both content-related validity, that is related to the practice of law, and criterion-related validity, that is MBE test scores correlate with other examination measures, including essay test scores and performance test scores. Moreover, studies have verified that the MBE is a test of analytical skills.46
Based upon its review of the multistate component of the bar examination, the majority of Commission members believe that it is an appropriate part of the bar examination and recommends its continued use. The practice of law demands a variety of skills. No one test is necessarily capable of testing all of those skills. What is needed is an examination that, through its various component parts, comprehensively tests the essential skills and reliably separates those qualified in the skills from those who are not. It is believed that the MBE is an appropriate part of such an examination.
As noted previously, the other component of the present bar examination is one-and-one-half days of essay questions. These essay questions are prepared by the Board of Bar Examiners and test the following subjects: business associations (such as agency, partnerships, and corporations), civil procedure, commercial transactions (including secured transactions and negotiable instruments), constitutional law, contracts, criminal law, domestic relations, evidence, legal ethics, property, torts, and trusts and wills. These subjects are considered by many to be core subjects and are widely tested in other jurisdictions also.
Essay questions are designed to test such skills and aptitudes as substantive legal knowledge, particularly knowledge of local law, analytical ability, writing skills, and the ability to demonstrate these various skills within time constraints. The subjects tested by the examination are periodically reviewed and revised by the Board of Bar Examiners.47 While testing some of the same skills as the MBE, essay questions also allow the testing of other skills and aptitudes important to the practice of law. The essay portion of the bar examination is a good complement to the MBE. Moreover, as noted, various types of tests increase the overall reliability of the examination.
B. Performance Testing
While the Commission is persuaded that the MBE and essay questions are appropriate components of the bar examination, it further believes the suggestions in the MacCrate Report and from the Ohio Conclave on Education for the Legal Profession that there are skills required in the legal profession that may not be being adequately developed in law school and tested before licensing lawyers. The Subcommittee spent a great deal of time reviewing the concept of performance testing, including a presentation by representatives of the National Conference of Bar Examiners who have developed a performance test that is currently available.
Performance tests are designed to measure analytical skills in a different way than the traditional bar examination questions. They require an applicant to demonstrate a competency in a manner that more closely simulates the manner in which a beginning lawyer is asked to perform in actual practice. One striking difference between performance tests and the MBE or essay questions is that performance testing places less emphasis on memorizing the laws and more emphasis on the application of substantive law involved in a particular assignment. Because the operative statute or case is often supplied in top performance based exam, a candidate must have only a general knowledge of the substantive law.
As noted, one of the purposes of performance testing is to measure lawyering skills not adequately tested by other portions of the bar examination. As stated in the MacCrate Report:
[T]he process of "lawyering" usually involves the assimilation of facts, research of law, development of legal issues and strategies, segregating relevant from irrelevant facts, and helpful from unhelpful materials, and the exercise of judgment in drawing a conclusion. Few of these processes were thought to be well-tested by either the MBE which depends for the most part on recognition of accurate or inaccurate discrete statements of law, or the traditional essay question, which generally requires some analysis, and the recitation in a brief period of time of as much of a particular area of law, designated by the question, as possible.48
The MacCrate Report went on to quote from a report of the Association of the Bar of the City of New York which also emphasized that performance tests measure skills required of a competent professional:
The performance test expands the scope of lawyering skills tested by the bar examination. Instead of limiting the bar examination to measuring the applicant’s ability to recall specific doctrinal knowledge, to spot issues and to apply fundamental legal principles, the performance questions test the applicant’s ability to evaluate undigested facts and integrate law and facts, and then to develop and employ that information using common lawyering tools such as a deposition, interrogatories or an appellate brief. Proponents believe that the performance test offers an effective method of assessing an applicant’s ability to perform competently some of the tasks a newly admitted practicing lawyer would perform. 49
An understanding of performance tests and the type of lawyering skills that they are designed to test can be obtained from a description of the performance test developed by the National Conference of Bar Examiners.50
The Multistate Performance Test (MPT) developed by the National Conference of Bar Examiners presently consists of two ninety-minute exams. Each exam has its own materials consisting of a file and a library. The file is the source documents for the problem presented and contains all of the facts of the case. Generally, this file includes the specific assignment being given the applicant that is set forth in a memorandum from a supervising attorney. It might also have other materials such as transcripts of interviews, depositions, hearings or trials; pleadings; correspondence; client documents; medical records; police reports; contracts; lawyers’ notes; and other sources of factual information. The library is the legal file and contains such materials as cases, statutes, regulations, rules, and other sources of the law applicable to the problem. As noted, in performance testing the applicant is not called upon to know a particular substantive law; rather, the law that is to be utilized in completing the particular task is provided in the library.
The MPT is designed to test six fundamental lawyering skills: problem solving, legal analysis and reasoning, factual analysis, communication, organization and management of a legal task, and identification and resolution of ethical problems. These skills are measured by requiring the applicant to perform an assignment that simulates the kinds of tasks normally done by a practicing lawyer. For example, an applicant may be asked to write a memorandum to a supervising attorney, prepare a letter to a client, draft a persuasive memorandum or brief, draft a contract provision, prepare a will or counseling plan, outline a discovery plan, prepare a witness examination, or write an opening statement or closing argument. Using the file and library, the applicant completes the assignment in the allotted period by sorting the factual materials; determining what is relevant and what is not; analyzing the statutory, case, or administrative law; applying that law to the relevant facts; identifying and resolving any ethical issue posed by the client’s problem; and producing a written product that is clear, precise, and well-reasoned.
Three states -- Alaska, California, and Colorado -- have been using performance tests for a number of years. Two of the states -- Alaska and California -- have included such tests on their bar examinations for more than ten years. Studies and analyses of data from these three states, as well as from four performance tests developed and administered as part of a research project by the National Conference of Bar Examiners in conjunction with American College Testing, confirm that performance testing can measure aptitudes and skills that may not be fully measured by other parts of current bar examinations.51
The considerations involved in deciding whether to add a performance test component to a bar examination have been summarized as follows:
The goal is to find a test that will provide as much information as possible about an applicant in as little time as possible, for the least amount of money.
It goes without saying that a two-day multiple-choice test would be the most efficient and reliable method of making licensure decisions. But, the practice of law requires a command of language, a skill that cannot be examined by an objective test.
The essay test adds to the validity of the bar examination by presenting the applicant with an open-ended inquiry and requiring a written response. However, the mark of a good essay question is the ability to distill essential facts into a brief statement, so the essay form of testing fails to examine an important element of legal practice: the real world never offers facts coherently, and it is the lawyer’s job to sift through seemingly contradictory facts to find those that are essential, to find those on which a legal point will turn or those that bolster a legal argument.
Performance testing adds an important, authentic dimension to licensure testing for lawyers. It tests skills not tested by other examinations in a format which more closely approximates the actual practice of law, and its monetary cost is comparable to the cost of other forms of testing.52
Based upon its review of these considerations, the Commission believes that performance testing would add a valuable additional aspect to the bar examination. It allows the evaluation in a realistic manner of fundamental lawyering skills needed by a beginning lawyer. The cost is not prohibitive -- twenty dollars per applicant if the MPT is utilized -- and the additional skills and aptitudes measured would seem to give even greater validity and reliability to the bar examination.53
The Subcommittee also recommends that the MPT be utilized. Development and validation of a performance test would involve significant expertise, time, money, and other resources that are not currently available to the Supreme Court of Ohio or the Board of Bar Examiners. Development and validation have been undertaken by the National Conference of Bar Examiners and the studies to date would indicate that the MPT measures the lawyering skills sought to be measured by such tests. There simply seems no logical reason for Ohio to duplicate that effort year after year.
Moreover, performance testing should be added to the Ohio bar examination without expanding the time in which the examination is currently given by reducing the number of required essay questions and essay subjects so that a half day of essays is replaced by performance testing. Part of the cost of performance testing presumably would be recouped by the costs saved in not having to print those additional essay questions. The Commission has conferred with the Board of Bar Examiners on this issue and it is the Commission’s understanding that the Board supports the idea of implementing performance testing. In light of the Board’s jurisdiction over the make-up of the bar examination, it is appropriate to leave to the Board decisions regarding what essay questions would be eliminated with the implementation of performance testing, should this Commission’s recommendations be accepted by the Supreme Court of Ohio.
Finally, adding a performance testing component to the Ohio bar examination offers two additional potential benefits. First, while there are indications that Ohio’s law schools already have begun to offer more skills training in their curricula, the addition of performance testing would serve to encourage and increase those efforts. Second, the further increase of skills training of law students has the potential for bridging (that is narrowing) the gap between law school graduates and first year practitioners. Other efforts to bridge the gap are discussed in the following chapter.