Former LSAT Question-Writer Interview

I recently interviewed Stephen Harris, former LSAT question-writer and author of Mastering Logic Games. (Yes, he's written hundreds of the questions that appear in your books of LSAT PrepTests.)

Our discussion follows.

You can also:

1. Read ALL of my interviews with him (more than 5!)

2. Get them all in a free book I put together.

***

1. What inspired you to become a writer of LSAT questions (an "item-writer"), and what's the process by which one gets that job?

Good question. It’s not exactly the kind of thing you lie in bed at night, hoping someday to grow up to be. When one with a philosophy doctorate looks for a job, there is a publication called “Jobs for Philosophers” that contains job listings, mostly for academic posts, but with a few listings for non-teaching positions. Testing companies list job openings there, and I applied for one. I ended up with a teaching position, but when I was later contacted about writing LSAT items as a freelancer, I agreed.

What makes writing LSAT items so rewarding, compared to writing items for other exams, is that the skills tested, especially in logical reasoning, are so relevant - what makes this argument stronger; what flaw does this argument commit, etc. These skills are among the set of important tools necessary for success not only in law school, but also as a responsible citizen.


2. Which specific LSAT questions, passages, and games did you write? Any tough Logic Games or Logical Reasoning questions?

I wrote several hundred logical reasoning items over the years, but the way the LSAC item review process is set up makes it very difficult to identify your items later on. Basically, there are two levels of review between the item writer and the test form, and adjustments can be made all along the way, so it’s hard to look back at tests and say “yeah, I wrote that one.” Not to mention that it may be several years before your item even makes it to a test form. One thing I definitely learned from working with the LSAC folks is that they go out of their way to ensure that the items that appear on the test are fair and consistent.


3. What sort of guidance / parameters were you given by ACT / LSAC when it came to constructing each item-type?

LSAC did a great job of training item writers. I was sent to a three-day workshop that covered the nut-and-bolts of item writing for logical reasoning, including critiques of practice items I had written. Additionally, item writers received a detailed guide that covers all of the item types, as well as a “don’t do this” list. And editors offered lots of feedback if you got something wrong to help you hit the mark the next time. ACT and LSAC only bought the items that they liked, so there was plenty of incentive all around to get it right.


4. How did you and other item-writers come up with the topics for each section of the exam? Is there some kind of pre-approved list? What's up with all the questions about dinosaur extinction?

This is a challenge for any item writer. I don’t remember a list of approved topics, but it was made clear from the beginning that certain topics were undesirable; in particular, any that are likely to elicit a strong emotional response from test takers. Hence the dinosaur items. When I’m writing items, I simply try to pay attention, often to what I’m reading, but sometimes just to the bugs in the garden. Actually, Google and Wikipedia make item writing in general much easier, although I had stopped writing LSAT items by the time they came around.


5. How long did it take you, on average, to write each type of LSAT question? Everyone seems to hate Parallel Reasoning questions, but which Logical Reasoning question-type is the most difficult to create?

That reminds me of a lawyer joke – “it’s tough to tell what counts as billable time.” Seriously, if you have an idea for an item then writing it takes an hour or two, with editing. Otherwise, you can spend a lot of time staring at a blank screen before fingers hit keyboard. I found that it was hard to average one LSAT item a day when I was writing items as fast as I could. And some items are definitely harder to write than others. Parallel reasoning/flaw items are the toughest, because you need six topics per item – one for the stimulus, and one for each answer choice as well. You develop an appreciation for folks who have to be creative on a daily basis – cartoonists, comics, etc.


6. What do you think is the most difficult Logic Game, and the most difficult Logical Reasoning question, out of all the released LSAT PrepTests?

One of my favorite LR items is the rattlesnake item, number 22 in section 2 of PrepTest 30. It really tests one’s ability to distinguish necessary assumptions from sufficient assumptions. The game that I think is hardest is the bus game - PrepTest 36, Game 3. This is a complicated game that can easily overwhelm a test taker who is unprepared.


7. I've noticed that certain Logic Games are virtual copies of each other. (Especially #3 and #7 in this list.) Does this stem from the item-writers being lazy and reusing old content, LSAC rewarding those who studied a lot, a combination, some other explanation...?

Yes - I especially like to compare PrepTest 13, Game 3 to PrepTest 32, Game 1. Not only the setups, but also the formulations of the first rules are strikingly parallel in these two games. (Even the numbers are near anagrams – coincidence?) I can only speculate as to why this happens, other than to pick up on your point that on a standardized test things are, well, standardized. There really is no question as to whether or not the same things are tested over and over – they are. The only question is whether you’ll be able to see how the items on your test are similar to previous items. As your list indicates, sometimes the similarity can be pretty close.


8. Have you ever heard of folks at LSAC reading online message boards and laughing at the stressed-out test-takers who post there?

I have never heard of LSAC folks basking in the misery of prospective law students. My experience with these people is that they are pretty sympathetic with the test taker and her plight. A guy I worked under at LSAC, for instance, had a hard time getting into grad school because he refused to take the GRE since “it didn’t measure anything important.” This is the kind of guy you want building your standardized test. One thing I have learned from studying thousands of LSAT items is that, first appearances notwithstanding, there is a benevolent intelligence at work behind the LSAT. The test taker just needs to tap into it.

Photo by the.sprouts