LSAT Computer Test, Cheating, and Theft

LSAC has thought about computerizing the LSAT for several years. In May 1999, LSAC published a study titled, "Item Theft in a Continuous Testing Environment: What is the Extent of the Danger?" (PDF)

In this study, LSAC and ETS consider "the possibility of organized, large-scale item theft" by "professional thieves" as a result of turning the LSAT into a computerized exam.

(Sadly, they weren't talking about an Ocean's Eleven-style heist or even about hacking. I'll high-five anyone with the skills to make an LSAT version of that trailer, though.)

If the LSAT were computerized, it'd probably be offered on most weekdays, like the GMAT and GRE.

Since it wouldn't be practical to write hundreds of unique exams each year, questions would be recycled. This group of questions is called an "item pool" by standardized test nerds (psychometricians). "Items" are test questions.

Your average "thief" is someone of average ability who remembers a few test questions and passes them along to friends. This has some impact on future test-takers' performance.

However, if "professional thieves" took the exam for the purpose of memorizing test questions and passing them on to future test-takers, this would have a more significant impact. (In the early 1990s, Kaplan employees took the GRE for the sole purpose of memorizing test questions - also see LA Times. It seems they were doing it to embarrass ETS rather than to give their own students an edge.)

In order to counteract these kinds of shenanigans, it's likely that if the LSAT were ever computerized, there would be several different pools of questions. Each one would be large enough so any benefit gained from memorizing previously-administered questions would be minimal-to-none. Besides, the topics of Logical Reasoning questions repeat so often that they tend to blur together in your mind unless you've done the question a few times.

For more details on what LSAC scientists do in their spare time when it comes to simulations about question-stealing, read the LSAC study (PDF). If you skip the mathematical parts, it's actually kind of entertaining.

You can also read more about LSAC's research on computerized testing of the LSAT. The executive summaries are probably enough to get the gist of each article. (After clicking that link, go to "Research" and click on "Computerized Testing Reports.")

Here are two more:

A Bayesian Method for the Detection of Item Preknowledge in CAT (Computer-Adaptive Testing)

Computerized Adaptive Testing Simulations Using Real Test Taker Responses

***
According to this 2003 LSAC internship posting, "LSAC is now actively studying computerized testing."

That's right. LSAC has a summer internship program. Here's this year's posting (page 6):

Law School Admission Council (LSAC)
Newtown, Pennsylvania
Number of Positions: One
Type of Student: Doctoral, preferably at dissertation stage
Deadline: March 15, 2010
The summer intern is given the opportunity to carry
out independent research under the mentorship of
LSAC research staff.
Contact: Lynda Reese, Associate Director of
Psychometric Research, Law School Admission
Council, Box 40, 662 Penn St., Newtown, PA
18940; (215) 968-1369; (215) 504-1408 (fax);
LReese@LSAC.org

Sadly, you have to be a doctoral student, so most of us aren't eligible to get the internship for the sole purpose of getting a sneak peak at the June 2010 LSAT (and the deadline's passed, anyway). If you're the lucky intern, though, let us know what it's like in the belly of the beast.

Photo by extraketchup / CC BY-SA 2.0
Photo by grimages / CC BY-NC-ND 2.0