LSAT Logic in The Economist Magazine

A recent article in The Economist discusses a study of poverty's effects on children's memories. However, as the LSAT teaches us, studies that imply causality can be flawed in a variety of ways.

Nick, an LSAT student of mine, sent me a linguistics professor's analysis of this article in The Economist.

However, as one might expect, the professor's analysis was overly technical. For this reason, I'd like to share with you my breakdown of what Professor Liberman had to say about the article and my thoughts on his analysis.

Here's the most relevant portion of The Economist article, as excerpted by Liberman:

THAT the children of the poor underachieve in later life, and thus remain poor themselves, is one of the enduring problems of society. Sociologists have studied and described it. Socialists have tried to abolish it by dictatorship and central planning. Liberals have preferred democracy and opportunity. But nobody has truly understood what causes it. Until, perhaps, now.

The crucial breakthrough was made three years ago, when Martha Farah of the University of Pennsylvania showed that the working memories of children who have been raised in poverty have smaller capacities than those of middle-class children. [emphasis added]

Liberman finds 3 major flaws in The Economist article. I have no background in statistics or linguistics, but here’s what I got out of his response to it:

1. Group vs. individual
The characteristics of a group (on average) may not represent those of an individual. The working memory of a studious child raised in poverty may exceed that of a lazy middle class child.

However, Liberman addresses this issue by allowing us to bet on 100 trials with 100 impoverished and 100 middle-class children.


2. Relatively small gap between the groups
The performance gap between raised-in-poverty and raised-middle-class children is not that large. This means that the article makes an over-generalization. The difference is small enough that Liberman’s betting odds aren’t worth taking. His odds imply that the middle-class kids are more likely to beat the poor ones than they actually are.

He suggests that if you’d only read the article, you’d be likely to take his bet when you shouldn’t.


3. What a working memory test actually tests
This is the most interesting and relevant flaw - working memory is not the only thing tested by a working-memory test. I’m probably oversimplifying this, but I believe that Liberman's graph suggests language and memory impact working-memory test results more than working memory does.

The bottom line: the working-memory test may actually test one’s comprehension abilities more than it tests working memory, so the test doesn’t do enough to isolate the variable it supposedly tests.

***
For another great example of real-world LSAT logic, check out "Facebook Use Linked To Lower Grades In College." (found via Freakonomics blog post - "Your Brain on Facebook").

In this article, the study's authors do an excellent job of acknowledging their study's limits. How? By raising the potential of a third variable that may lead to Facebook use and to lower grades.

Update: "Facebook Use Does Not Make You Stupid." It just goes to show correlations don't always mean as much as you might think.