Aug 29, 2013

The Lie Factor

In the last few days, a lot of my Chicago friends have been posting links to a graph showing how Chicago Catholic High School students ACT scores exceed those of Illinois and the nation.  Here's the graph that the Chicago Archdiocese provided (click to go to their page with more info and a much bigger graph):



Assuming the data behind the graphs is true (and I have no reason to doubt that it is), it looks like the Catholic school students have a very impressive lead on everyone else--the Catholic students seem to have done twice as well as everyone else in several of the skill areas!

But if you look closely, you'll notice that the bars in the graph don't actually cover the full range of the possible test scores, but instead start at 18.5.

This is a great example of what Edward Tufte called the Lie Factor in his book “The Visual Display of Quantitative Information.” Tufte says:
The representation of numbers, as physically measured on the surface of the graphic itself, should be directly proportional to the quantities represented.
In the above graph, the representation is anything but proportional to the quantities represented--it's only showing the tips of the bars. Here's the same graph with the Y axis starting at zero instead of 18.5:


There's still an obvious lead for the Catholic schools in question, but hardly the massive lead that their graph seems to show.  Looks to me like the Catholic schools are doing great, so why stretch the truth?


2 comments:

exuberance said...

Private school attendance is correlated with parental income and parental income dominates these stats.

http://secondaryschoolprivatevspublic.weebly.com/uploads/4/1/0/0/4100451/6064553.jpg?388

It's a fun game to play actually. Pick your fun fact: "Having more than one pair of shoes." "Family owns multiple vacuum cleaners." "Taking summer vacation." ... - ben

Bren in SoCal said...

Yet another reason why I encourage non-engineering/ math majors to take AP Stat in high school instead of AP Cal. Once you get stats, even a little bit, it's harder to be lied to.