LSAT Cultural Bias Interview


LSAT Blog LSAT Cultural Bias Interview
I recently interviewed Stephen Harris, former LSAT question-writer and author of Mastering Logic Games, about whether the LSAT is culturally biased.

(He's written hundreds of the questions that appear in your books of LSAT PrepTests.)

Our discussion follows.

You can also:

1. Read ALL of my interviews with him (more than 5!)


Do you think the LSAT is culturally biased? If so, what steps can be taken to correct for that? In writing questions, did you take measures to reduce your own cultural bias or does the question selection process correct for that? Is the LSAT a test of acculturation?

“Biased” is a word that is used in different senses. In some sense, the whole point of a test is to discriminate. A good test discriminates on the basis of reasonable considerations; a bad test not so much. For most tests, the basis for this discrimination will be cultural factors. Is a spelling test biased against bad spellers? In some sense, the answer is clearly yes. And that’s the point, even though spelling is a cultural phenomenon.

Likewise, the whole point of the LSAT is to help determine the extent to which test takers possess certain abilities, and these abilities are clearly cultural in some sense, like reading and drawing verbal inferences. But does this mean that either a spelling test or the LSAT is culturally biased in a troubling way? Not necessarily. They might be, by choosing idiosyncratic words to spell, for instance, or allowing extraneous factors that privilege one group over another to play a role on the LSAT. We can, and will, argue over which skills are the important ones, and whether a tool tests these skills in idiosyncratic ways that disadvantage otherwise qualified students inappropriately.

By any reasonable measure, it seems to me, the LSAT tests generally relevant skills in a manner that rarely prevents students who are likely to perform well in law school from gaining admission to some school or other. But bias in this pejorative sense is more like crime – you can never eliminate it, only combat it, especially since the standards by which inappropriate bias is judged are subject to dispute and undergoing constant change as a result of larger cultural conversations.

As for socioeconomic bias specifically, undoubtedly LSAT performance is correlated with socioeconomic status; life expectancy, quality of health care and education, and virtually everything else that people think is good is correlated with socioeconomic status. It would be really surprising if LSAT scores weren’t. But it seems to me that the problem with LSAT bias in this sense isn’t the LSAT per se as much as these other factors, and I doubt that there is an LSAT-specific remedy for this issue.

The most reliable way to limit one’s own inappropriate biases, I think, is to choose item topics carefully. Topics that one group might be more familiar with than another are generally poorly suited for test items. As an example, questions about sports very often risk the possibility of gender bias, since for some sports whether people play or follow them is pretty strongly correlated with gender.


Photo by 51170735@N02



18 comments:

  1. There was insufficient evidence presented to support many of the claims made.

    I kid. But interesting answers on the topic of bias.

    -tyler

    ReplyDelete
  2. There is no doubt in my mind that the LSAT is culturally biased. Mr. Harris points out that there are many kinds and causes of bias, but he seems to excuse the LSAT for its bias. As a professor of a university which has 94% Hispanic population, and where most students are bi-lingual English and Spanish, it seems to me that vocabulary is the biggest culprit. Having the vocabulary required by the LSAT has nothing to do with the skills of reasoning, intelligence or drawing inferences. Students with a different, but equally sufficient vocabulary, have all those intellectual skills. Most law schools and courts conduct their communication using the mainstream vocabulary. Why is it that we value what I will call the LSAT vocabulary (GRE and GMAT as well) over a mainstream vocabulary? This is a topic worth studying, and is on my "to do" list!!

    ReplyDelete
    Replies
    1. Then again, with enough time and preparation, a student who is not familiar to 'mainstream' vocabulary could come to learn it eventually, if they had the same level of skill, reasoning, intelligence as someone already familiar with it. If standard examinations did not use 'mainstream' vocabulary to test a wide population, what would they use?

      Delete
    2. And don't you believe that same vocabulary might be used in law school or actually practicing law? If a student hasn't put in the work to learn that vocab before taking the LSAT, then why should anyone believe he/she will learn before law school, the bar exam, or before actually representing clients?

      Delete
    3. "Most law schools and courts conduct their communication using the mainstream vocabulary."

      Silliest thing I've heard in months.

      Delete
    4. Sorry but the vocab on the LSAT is of a far lower level than that of your average case book. If you can't understand the LSAT well enough to do well then you really will struggle in law school (not to mention the rest of your life). Now the GRE on the other hand...

      Delete
  3. LSAT is really a monopoly business...

    ReplyDelete
    Replies
    1. Thank you, that was a very insightful comment.

      Delete
  4. The LSAT is ridiculous and totally useless. Call it was it is, an aptitude test, and move on.

    ReplyDelete
    Replies
    1. Haha. I love that the above is said with no sense of irony.

      Delete
  5. He's real smert

    ReplyDelete
  6. LSAT is intended and specifically engineered to keep people of color out of legal profession. Shocker....in light of the huge growth in Latino population during the past Census, the institutional bias is highlighted even more. Try this experiment. Go to any Big Law firm and look for a Spanish-surname Associate....no Italian last names don't count and white women who married a Latino don't count as well.(Hint: it will take quite a while to find one...yes just one). Now do a search for Partners using any Big Law firm site....get the picture? Nothing is more brutally honest then data.

    ReplyDelete
    Replies
    1. You're an idiot.

      Delete
    2. Please stop whining about a requirement that one be proficient in standard English to get a "fair shake" on the LSAT or in life. Read a Fortune 500 contract or an offering memorandum; if your attorney can't handle English (the international language of business, like it or not), then you get screwed.

      Delete
  7. Really, "idiot" and "whining" about data. How about a counter to the data? I am fully bilingual in two languages (perhaps you have trouble reading English) as well as data analysis....So far both responses lack any logic or seem to counter the evidence presented. So how many Latino Associates and/or Partners have you found in law firms ranked in the top 100 in U.S.? I personally know two Partners at big firms and both think the system is rigged. According to Columbia law school study the gpa/lsat score for Latino applicants has gone up while the amount of applicants has stayed the same since 1992. Again, respond with some solid data...actually any data countering this reality would be illuminating....and yes, the LSAT is obviously slanted in favor of a certain demographic.

    ReplyDelete
  8. Correction:
    "According to Columbia law school study the gpa/lsat score for Latino applicants has gone up while the amount of ADMITTED LATINO applicants of has stayed the same since 1992."

    ReplyDelete
  9. Everyone stop whining and start studying. To blacks and Hispanics, study harder. Life isnt fair. Get used to it.

    ReplyDelete
    Replies
    1. Even the ABA acknowledges the profession lacks diversity. Third only to the dental and natural science professions. Try reading some academic material regarding the subject matter before ignorantly stating "study harder" to people of color.

      Delete