How to Challenge a Flawed or Unfair LSAT Question

Even the Law School Admission Council (LSAC) makes mistakes.

Occasionally, a flawed LSAT question (a question that has no clearly correct answer) slips through the cracks and makes it onto a scored section of an actual administered LSAT exam.

Fortunately, like any bureaucracy worth its salt, LSAC has a set of rules in place just in case a clever (or stubborn) test-taker decides that his/her logic surpasses LSAC's. You can find these rules in "Policies and Procedures Governing Challenges to Law School Admission Test Questions."

Blog reader Jamie recently emailed me about "withdrawn" LSAT questions (questions that students have successfully challenged). Because LSAC doesn't want to confuse students with ambiguous or poorly-written questions in the published versions of its exams (and because it doesn't want its mistakes to live on in published form), these questions are not reprinted. Instead, the space where they appear is simply noted as "withdrawn."

Jamie writes:

I sometimes come across a "withdrawn" question on a released test: you know, "withdrawn from scoring." I imagine a number of different situations: LSAC scores the exams and realizes that a certain question threw off the difficulty of the test, or two angry LSAC philosophers break into arm wrestling match over a question only to realize there are multiple acceptable solutions as presented, or perhaps a heroic test candidate catches a flaw the LSAC glossed over, reports it to Newtown (Ed: the town in Pennsylvania where LSAC's headquarters are located - Steve), gets a 181 and a ticker-tape parade.

Whatever the cause, what are the implications? If LSAC withdraws a question, and your answer was the credited answer, do you lose out on a point? If you're certain that a test question has an error, do you have a recourse? I just wonder if it's detrimental to note "withdrawn" on the released test, since the released version is no longer an accurate specimen of what you would have sat through for that particular test.

LSAC calculates the scale (translation of raw scores to scores out of 180) before each administration, and it pre-tests questions in previous exams' experimental sections. For this reason, it's not due to the exam's difficulty. (LSAC occasionally recalibrates the scale if an exam turns out to be harder or easier than expected. However, it doesn't withdraw questions for this reason.)

It's also not the philosophers' arm-wrestling match, as amusing as that would be. LSAC philosophers do all their arm-wrestling before the exam is ever administered.

Withdrawn questions only result from the efforts of heroic flaw-catching test candidates.

Yes, if they withdraw a question, but you answered it "correctly," you lose out on the point.

If you're certain that a question is wrong, you can email LSAC afterwards. If they decide in your favor, they'll withdraw the question.

However, under test day pressure, it's better to attempt every question than to puzzle over one you believe to be poorly-written. While you'd live on in the test-takers' hall of fame for your efforts, it's to your advantage to prioritize the other questions instead of getting bogged-down.

If you take a practice exam that contains a withdrawn question, your experience won't be perfectly similar to that of the people who actually took that test, but it'll be pretty darn close, just one fewer question out of approximately 100 questions. It's negligible in the larger scheme of things.

LSAC recalibrates the scale when this happens, so you'll still get an accurate result.

***
Read on for Withdrawn LSAT Questions | Item Removed From Scoring.

Photo by andycarvin