Sunday, April 26, 2009

Surveys and Astrology

Astrology predicts lots of good things. Best time to plant crops, eclipses, tides, menstruation (good if making babies), migration, shadows on the sundial. The only problem is that this was.... well assumed to be related to many other things. This is easy for ancient man as it is for the modern Oprah Secret crowd. If you can prove one thing is regular as clockwork, why not simply assume it controls many other things.

Lunar tides, our path around the Sun, and the effects are very real. It is just natural to start fantasizing about birthdays, planets, and constellations.

Read Malcom Gladwell's 'Outliers'. It is a fact that Hockey players born in specific month were more likely to be the better players on their teams. You could easily apply this to astrology. The root cause is really the relative maturity when kids can sign up. Just a coincidence of birthday and a yearly ritual.

Same could be said of ancient warriors. If they sign up at a specific time each year, those born such that they are older and more mature than the average kid, they become the stronger warriors and maybe the sergeants and captains.

For our warrior, the day each year they start the warrior tryouts is what defines who will be more successful.  The age of the recruits, not their star sign, is what defines the odds of being a kick ass warrior. Kids born a month or two before the annual date would have a low success. Those born just after would be nearly a year older and bigger.  If tryouts are in April, those born in May will be the most successful, while those born say in February would be less so.

Astrology works for many such coincidences. Given one coincidence, people see and extrapolate even more. Even to self delusion. Astrology is alive and kicking today because it is child's play to extrapolate from tides to planets moving through constellations. Add some tricky tables and calculations, you get something that sounds like science.

Surveys, personality tests, focus groups, ratings, and other tools of the trade can become astrology. All it takes is a little truth to make a bigger lie.

One that struck me funny was the "Bradley Effect". Tom Bradley, and African America, was ahead in the polls for Califormia governor. But on election day, Bradley lost. The thought was that for a pole people would say they would vote and in the booth, their prejudices, hidden by the privacy of the voting curtain, would rise up and they would vote for the white guy. Of course this was a common topic for the presidential election.

Is the Bradley Effect real? Well it seemed to work a few times. My guess that for the next 20 years whenever race is an issue, you will see the "Bradley Effect". It only had to seem right once or twice. To do that, all you need is a poll that is up before the election with a loss of the election by the same candidate - oh and they need to be a minority.

There is absolutely no evidence that the Bradley Effect was caused by switching votes. In fact there is no way to prove it because of the sanctity of the voting booth and an American's god given right to lie to a pollster. The real reason could be apathy or a push for the opposition between polling and voting or even just random chance of the poll samples missing the real voters intentions.

I'd love to say there is a way out of this. Surly their is some way to dispel the crackpot ideas. Maybe, maybe not. Anything that can be extrapolated based on one or two data points is in danger of wild speculation that becomes fact. People see the hand of God or the genius of a pollster, or that only eating a hot dog in the 7th inning will win a ball game

Only lots of data can prove your point about bad numbers. If you go for statistically accurate data and eliminate possible biases(like randomizing questions are asked, when, who asked, and what order), you can have a fighting chance. If you have data that can go either way, your data can be misused or misinterpreted. But be warned, people still don't see statistics, they see meaning.

The real problem comes with other peoples data and conclusions. Really look at their stats and how they collected the data. Don't accept at face value the result of any amateur survey or straw poll.

Sure it may be right, but play the skeptic for a while to be sure. At least that's what my horoscope said today.

Ever been stuck with bad data and a wild guess at what it proved? Were you able to disprove the bad assumptions? What was the clinching argument or fact? Let me know in the comments.