A Gravis Response
On October 14, we ran an article by dawolf, one of our regular readers and commenters, regarding Gravis Marketing, a company that appeared a couple of months ago out of nowhere in the political polling world, and rapidly became one of the most prolific pollsters. Gravis’s results were far to the right of the consensus, to the point where I quickly began correcting their results by moving them about five points to the left.
grantcart, a writer at the left-leaning Democratic Underground (DU) website, became suspicious of Gravis’s results for much the same reasons, and began to write a series of articles expressing growing suspicion of the legitimacy of Gravis and its head, Douglas Kaplan. Daily Kos picked up the ball and ran with it, exposing the story to a much broader audience. Around that time, dawolf decided to investigate the credibility of the polled numbers, and wrote the article that we ran here at Logarchism.
In retrospect, I feel bad that I didn’t take the time to approach Kaplan prior to publishing the article. It’s not that dawolf’s analysis was wrong, but rather we should have provided Kaplan to present his side of the story. After the story ran, dawolf suggested that I contact Kaplan to discuss it, which I did. We talked on October 16, and again yesterday.
The first time we talked, he was extremely agitated about the way his polls have been portrayed on DU in particular, but also here at Logarchism. He called the article “100% false”, and was most upset about accusations that the polls were never conducted. It was hard to get a clear picture of his position at times, since his agitation led him to sputter and rant, which made it difficult at the time to obtain a coherent explanation for the results we’re seeing. Our second conversation yesterday was more relaxed.
The gist of his explanation is as follows. He has for several years run a direct communication company specializing in, among other things, interactive voice recognition (IVR) and predictive dialing technologies. Among the functions that an IVR company can perform is polling. Historically, Gravis has been contracted to conduct polls on behalf of other firms (Kaplan wouldn’t tell me who they are, but it’s common for such contracts to include nondisclosure clauses), but Gravis never did more with the resultant data than to send them to the contracting firm. That is, Gravis had no experience at interpreting the raw results of a poll, and that’s where the real value lies in polling.
Kaplan didn’t think the interpretation would be difficult to do, and since he was able to conduct polls at wholesale prices (his retail price is about a nickel per respondent; he wouldn’t share his cost per call), he figured he could build a reputation as a polling firm by conducting his own polls. So he hired a statistician and started polling. But he didn’t start running them until very late in this election cycle.
This proved to be a major mistake, as polling firms typically use the earlier months in an election cycle to fine-tune their models so as to have as accurate a set of results as possible by the last couple of months prior to the election.
What sorts of tuning did Gravis need to do? Recall that they are predominantly an IVR firm, so their polls are dominated by fully automated techniques (though Kaplan tells me they do conduct some live polling and have experimented with other techniques as well). Fully automated polling excludes firms from calling mobile numbers, or calling anyone in Indiana or North Dakota. Households without landlines are significantly more likely to be Democratic, for a whole host of reasons. But this means that Gravis has to correct for both the unknown of how many such households are being missed and the second-order unknown of which candidate would be supported by that unpolled household.
So, like any new polling firm, Gravis’s statistician tries to correct for demographics. What percent of voters will be African-American? What percent will be between 18 and 25 years of age? What percent will have incomes between $25,000 and $35,000? Comparing the IVR results to the expected voting demographics requires weighting some responses more, and others less. This is not actually an easy task. How well do the African-Americans who respond to the poll represent the ones who don’t? Is the Democratic turnout in Iowa in 2012 going to be close to the Democratic turnout in Iowa in 2008? Are all of those who claim to be likely to vote actually likely to vote? Answering these questions requires the experience of having run dozens of such polls, validating the results, and applying the learning to future iterations.
This is why coming into an election cycle three months before the election is a recipe for disaster. And there is no question that Gravis’s early polls were a disaster in terms of accuracy. Kaplan readily agreed that the statisticians he hired were inexperienced at handling political polling.
If you look at last week’s polls, however, Gravis has been within the normal bounds of mainstream polling firms. Colorado, Florida, and Ohio were all within a point or two of the consensus, and with no discernible lean. One week of three polls isn’t enough to say that Gravis is worthy of a top-tier placement, but it is at least a positive sign.
Are Gravis’s polls legitimate, then? At the very least, it’s hard for me to say definitively that they are faked. It’s certainly possible to do, but unlike Research2000, who didn’t have the infrastructure to conduct the polls, Gravis’s core business is that very infrastructure. In their case, it might actually cost more to fake the results, particularly if they’re using otherwise idle resources to conduct the polls. Is Gravis in the tank for the Republican Party? Possibly, but more recent topline results suggest otherwise.
At this point, my belief is that Gravis is a company that got in over their heads at a really bad time to do so.
Even if I’m wrong, and they are nothing more than shills for the Republican Party, I’m still uncomfortable with the way that Kaplan has been personally attacked by many members of the Democratic Underground and Daily Kos. When addressing one’s professional actions, attacking one’s professional credentials is certainly legitimate. And if that person has a personal conflict of interest that influences professional actions, those too are legitimate.
But is Kaplan’s DUI arrest from decades ago relevant with respect to his ability to conduct a legitimate political poll? I can’t see why it is. But what about the legal troubles he’s had in the past?
This is a little tougher to answer. He has clearly tried to build numerous businesses that involve direct communication marketing, an area where it is easy to run afoul with the law. It is a field with a tremendous number of regulations, and yet also a tremendous number of small, inexperienced firms. We’ve all been on the receiving end of undesired telephone calls, even after registering on the National Do Not Call List. Those of us who still have facsimile machines receive unsolicited facsimiles. And it appears that some such contact may have come from one of Kaplan’s firms.
Does it make sense that a person who has gotten into trouble in the past for contacting people he shouldn’t would also be guilty of not contacting people that he should?
All in all, I don’t feel as if I’m in a position to judge the overall legitimacy of Gravis Marketing as a company. Democratic Underground will most likely continue to conduct their investigation into the company and its officers. I just hope that such an investigation is conducted with class.
- A Gravis Fraud
- Romney leads in 1st post-debate poll in Colorado
- Gravis Marketing Political Poll for September 7 and 8