This post is Part 5 of the "The LSAT Curve" series. The series starts with The LSAT Curve | Test-Equating at LSAC.Creating the LSAT's Raw Score Conversion Chart (aka, the Curve)
Let's suppose that, on a given exam, the 170-scorers got 12 questions wrong altogether on the 4 scored sections.
That's an average of 3 questions wrong per scored section. Let's assume they got an average of 3 questions wrong on the games section.
However, let's say that a subset of those 170-scorers all took the same experimental Logic Games section. What if these test-takers got an average of 5 questions wrong on that Logic Games section?
Therefore, we can say that this experimental Logic Games section is harder for 170-scorers than the scored LG section. As a result, this section deserves a slightly more generous "curve" than the scored section of LG does - for 170-scorers.
Let's suppose the average 150-scorer got 40 questions wrong altogether on the 4 scored sections of this very same exam.
That's an average of 10 questions wrong per scored section. Let's assume they got an average of 10 questions wrong on the games section.
What if a subset of those 150-scorers took the very same experimental Logic Games section and scored an average 10 questions wrong on it?
Therefore, we can say that this experimental LG section is no harder for 150-scorers than the scored LG section was. As a result, this section doesn't a more generous "curve" than the scored section of LG does - for 150-scorers.
(Of course, all of this is only about one Logic Games experimental section. Perhaps a different group of 170-scorers took an experimental Logical Reasoning section that was easier for them than their scored Logical Reasoning sections. Perhaps a different group of 150-scorers took this experimental LR section and found it more difficult than their scored LR sections. If the experimental Logical Reasoning section were placed on an exam with the experimental Logic Games section mentioned earlier, the differences might cancel each other out.)
The fact remains that a given exam might be of varying levels of difficulty for test-takers at different levels.
If a particular test is very difficult for 170-scorers, then the "curve" for them will be very generous (meaning someone who "deserves" a 170 won't have to answer as many questions correctly to get a 170 as they would have if the exam weren't as difficult).
If a particular test is of normal difficulty for 150-scorers, then the curve can just be normal for them, meaning it'll require the typical amount of questions correct in order to get a 150. People whose "true scores" are at 150 won't need any messing with the "curve" to get the 150 they deserve.
A nice summary on "true scores" and test-equating from LSAC:
Testing organizations typically disclose test forms after they have been administered to large test-taker populations. Therefore, several test forms must be developed annually to be as similar as possible to one another in terms of statistical and content attributes. Although a great deal of effort is placed on assembling comparable tests, forms will tend to vary somewhat in terms of their statistical characteristics. Hence, scores must be transformed in order to enable direct comparisons across forms. The process by which scores are adjusted so as to make them comparable to each other is referred to as equating. The Law School Admission Council (LSAC) employs item response theory (IRT) true-score equating to equate the LSAT.Source: Assessing the Effect of Multidimensionality on LSAT Equating for Subgroups of Test Takers (Executive Summary)
***
If any of this "LSAT curve stuff" seems confusing or unfair, don't waste time worrying about it.
The bottom line is that it doesn't matter which month you take the exam, and it doesn't matter how easy or difficult your particular exam is. The raw score conversion chart, based on LSAC's statistical data, addresses all of those issues and makes sure that everything's equal in the end.
Just focus on answering as many questions correctly as possible, and let LSAC take care of the rest.
***
Want to start at the beginning? Begin with The LSAT Curve | Test-Equating at LSAC.
Also, in case you missed them:
-I published the Raw Score Conversion Charts for every LSAT PrepTest ever released in one big spreadsheet.
-I created graphs and charts demonstrating changes over time in how many questions you can miss on the LSAT and still get a 170 or 160, respectively.
Photo by revolute / CC BY-NC-SA 2.0
Steve,
ReplyDeleteI need remedial practice with conditional reasoning and formal logic. For me, the best way to learn is by doing drills and exercises. Can you recommend any drill books on these topics? I'm looking for drills like the ones I've been through in the PS bibles.
Any logic textbook is likely to go into FAR more detail than you need, but logic textbooks contain tons of drills and exercises of the type you're looking for. I'd recommend getting an older edition of your favorite logic textbook for a few bucks online.
ReplyDeleteI like Hurley's Concise Introduction to Logic.
I get that the LSAT is not "curved", in that the raw score conversion chart has nothing to do with the pool of test takers for that particular administration.
ReplyDeleteBut if I've understood everything in these posts correctly, couldn't you say that any test is "curved" based on the test taking pool from previous administrations?
And if the test taking pool is skewed towards one end on any given administration (possibly due to an influx of "exploratory" test takers as could be the case recently due to the economy), then couldn't the skewed pool lead to questions on those experimental sections being classified as more difficult than they would have been in previous years, which would lead to any future test on which those questions appeared being curved more generously than it would have been?
I think people have already identified an influx of test takers, which has countered the better preparation that some people attribute to the the curve-tightening trend, as a reason for the curve opening back up. But according to these posts, just because the pool may have been skewed differently on one particular administration is irrelevant. The pool has to have been skewed on previous administrations where the question difficulty level is determined.
Too much speculation?
If the curve has nothing to do with the pool on a particular administration, then wouldn't the curve be determined before the test was taken? If so, why don't they release it immediately after the test?
ReplyDelete