Introduction to the Controversy
On Monday, the State Bar of California revealed that it used AI to develop a portion of multiple-choice questions on its February 2025 bar exam, causing outrage among law school faculty and test takers. The admission comes after weeks of complaints about technical problems and irregularities during the exam administration, reports the Los Angeles Times.
Details of the Exam
The State Bar disclosed that its psychometrician, ACS Ventures, created 23 of the 171 scored multiple-choice questions with AI assistance. Another 48 questions came from a first-year law student exam, while Kaplan Exam Services developed the remaining 100 questions.
Defense of the State Bar’s Practices
The State Bar defended its practices, telling the LA Times that all questions underwent review by content validation panels and subject matter experts before the exam. "The ACS questions were developed with the assistance of AI and subsequently reviewed by content validation panels and a subject matter expert in advance of the exam," wrote State Bar Executive Director Leah Wilson in a press release.
Criticism from Legal Education Experts
According to the LA Times, the revelation has drawn strong criticism from several legal education experts. "The debacle that was the February 2025 bar exam is worse than we imagined," said Mary Basick, assistant dean of academic skills at the University of California, Irvine School of Law. "I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable." Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation, called it "a staggering admission." She pointed out that the same company that drafted AI-generated questions also evaluated and approved them for use on the exam.
State Bar Defends AI-Assisted Questions
Alex Chan, chair of the State Bar’s Committee of Bar Examiners, noted that the California Supreme Court had urged the State Bar to explore "new technologies, such as artificial intelligence" to improve testing reliability and cost-effectiveness.
Conclusion
The use of AI to develop questions for the bar exam has sparked controversy and criticism from legal education experts. While the State Bar defends its practices, citing review by content validation panels and subject matter experts, many are concerned about the implications of using AI-generated questions on a high-stakes exam.
FAQs
- Q: Did the State Bar of California use AI to develop all the questions on the February 2025 bar exam?
A: No, the State Bar used AI to develop 23 of the 171 scored multiple-choice questions. - Q: Who reviewed the AI-generated questions before the exam?
A: The questions underwent review by content validation panels and subject matter experts before the exam. - Q: Why did the State Bar use AI to develop questions for the bar exam?
A: The California Supreme Court had urged the State Bar to explore new technologies, such as artificial intelligence, to improve testing reliability and cost-effectiveness. - Q: What has been the reaction to the revelation that AI was used to develop questions for the bar exam?
A: The revelation has drawn strong criticism from several legal education experts, who are concerned about the implications of using AI-generated questions on a high-stakes exam.
