Comments on: Gravis Marketing: A Deeper Analysis

By: dawolf

dawolf — Thu, 01 Nov 2012 19:01:28 +0000

mclever, a bit of variation is fine but from 4% to 0.5%, and back up? With the subsampling as well? Doesn't hold up. Age groups dropping by 5%? Only women answering in a group (try doing the probability on only female asians. 16 people assuming no demographic weighting, all of one sex. You might be surprised how unlikely it is).

We're not talking about tiny sub groups in the ages etc either.

By: mclever

mclever — Thu, 01 Nov 2012 15:34:47 +0000

With regard to how likely it is that they got Muslims almost all going for Romney, it depends on how many Muslims they contacted. We see that their weighted number was 1.73% of the 914 responses, so it's possible that they only managed to reach a half dozen and weighted that up. If they only managed to reach a small number, then it's like rolling five dice and having them all come up as sixes. Possible and actually likely to occur on occasion due to randomness. (Hey, I've played Yahtzee. It can happen.)

Now, if they'd managed to reach 100 Muslims and still got them all going for Romney or Other, then that would more like rolling those dice and having them come up sixes multiple times in a row, which would be verging on impossible unless the dice were weighted.

Picking apart one poll like this--even from a reputable company--will often find anomalies in the small numbers, because that's where the unusual results are most likely to occur. It's also part of why, when weighting the polls, most pollsters use "targets" rather than absolute values to weight the numbers against, because if they only reach one person in a certain demographic that should be, say, 5% of the population, overweighting that single person to a full 5% of the result will almost certainly be the wrong thing to do. Different pollsters use different guidelines, but usually there's some maximum weighting that they'll give to a single voter (like, no more than double or triple) to avoid giving one person too much of a distorted effect on the total poll results. That can also explain why, even weighted polls will have demographic variance across multiple iterations.

By: channelclemente

channelclemente — Thu, 01 Nov 2012 15:08:32 +0000

Never has the phrase "Gravis is pure Bullsh*t" been uttered so many ways.

By: shortchain

shortchain — Thu, 01 Nov 2012 14:00:17 +0000

Monotreme,
Hey! That's my method of obfuscating the pollster's results. Or maybe a Buddhist for Johnson.

dawolf,
I'm not a "professional statistician" -- I only teach the stuff, I don't do it -- but your analysis doesn't have any egregious errors I can see (without feeding it into a program: we don't do this stuff by hand anymore). I pick option 3.

By: dawolf

dawolf — Thu, 01 Nov 2012 13:17:16 +0000

Mono - you might get one or two doing that, but it seems very unlikely to be so many.

BTW, the point of demographic weighting is to adjust for uneven demographics, the effects of robocalling etc. But there is no evidence at all that demographic weighting took place.

By: Monotreme

Monotreme — Thu, 01 Nov 2012 12:03:58 +0000

I would also think that robocalling introduces a huge demographic shift in the population and (much worse) increases the probability of outright lying. If Gravis contacted me, I might just say I'm a Muslim for Romney just to be puckish.