Doug Kaplan

On Octo­ber 14, we ran an arti­cle by dawolf, one of our reg­u­lar read­ers and com­menters, regard­ing Gravis Mar­ket­ing, a com­pany that appeared a cou­ple of months ago out of nowhere in the polit­i­cal polling world, and rapidly became one of the most pro­lific poll­sters. Gravis’s results were far to the right of the con­sen­sus, to the point where I quickly began cor­rect­ing their results by mov­ing them about five points to the left.

grant­cart, a writer at the left-​​leaning Demo­c­ra­tic Under­ground (DU) web­site, became sus­pi­cious of Gravis’s results for much the same rea­sons, and began to write a series of arti­cles express­ing grow­ing sus­pi­cion of the legit­i­macy of Gravis and its head, Dou­glas Kaplan. Daily Kos picked up the ball and ran with it, expos­ing the story to a much broader audi­ence. Around that time, dawolf decided to inves­ti­gate the cred­i­bil­ity of the polled num­bers, and wrote the arti­cle that we ran here at Log­a­rchism.

In ret­ro­spect, I feel bad that I didn’t take the time to approach Kaplan prior to pub­lish­ing the arti­cle. It’s not that dawolf’s analy­sis was wrong, but rather we should have pro­vided Kaplan to present his side of the story. After the story ran, dawolf sug­gested that I con­tact Kaplan to dis­cuss it, which I did. We talked on Octo­ber 16, and again yesterday.

The first time we talked, he was extremely agi­tated about the way his polls have been por­trayed on DU in par­tic­u­lar, but also here at Log­a­rchism. He called the arti­cle “100% false”, and was most upset about accu­sa­tions that the polls were never con­ducted. It was hard to get a clear pic­ture of his posi­tion at times, since his agi­ta­tion led him to sput­ter and rant, which made it dif­fi­cult at the time to obtain a coher­ent expla­na­tion for the results we’re see­ing. Our sec­ond con­ver­sa­tion yes­ter­day was more relaxed.

The gist of his expla­na­tion is as fol­lows. He has for sev­eral years run a direct com­mu­ni­ca­tion com­pany spe­cial­iz­ing in, among other things, inter­ac­tive voice recog­ni­tion (IVR) and pre­dic­tive dial­ing tech­nolo­gies. Among the func­tions that an IVR com­pany can per­form is polling. His­tor­i­cally, Gravis has been con­tracted to con­duct polls on behalf of other firms (Kaplan wouldn’t tell me who they are, but it’s com­mon for such con­tracts to include nondis­clo­sure clauses), but Gravis never did more with the resul­tant data than to send them to the con­tract­ing firm. That is, Gravis had no expe­ri­ence at inter­pret­ing the raw results of a poll, and that’s where the real value lies in polling.

Kaplan didn’t think the inter­pre­ta­tion would be dif­fi­cult to do, and since he was able to con­duct polls at whole­sale prices (his retail price is about a nickel per respon­dent; he wouldn’t share his cost per call), he fig­ured he could build a rep­u­ta­tion as a polling firm by con­duct­ing his own polls. So he hired a sta­tis­ti­cian and started polling. But he didn’t start run­ning them until very late in this elec­tion cycle.

This proved to be a major mis­take, as polling firms typ­i­cally use the ear­lier months in an elec­tion cycle to fine-​​tune their mod­els so as to have as accu­rate a set of results as pos­si­ble by the last cou­ple of months prior to the election.

What sorts of tun­ing did Gravis need to do? Recall that they are pre­dom­i­nantly an IVR firm, so their polls are dom­i­nated by fully auto­mated tech­niques (though Kaplan tells me they do con­duct some live polling and have exper­i­mented with other tech­niques as well). Fully auto­mated polling excludes firms from call­ing mobile num­bers, or call­ing any­one in Indi­ana or North Dakota. House­holds with­out land­lines are sig­nif­i­cantly more likely to be Demo­c­ra­tic, for a whole host of rea­sons. But this means that Gravis has to cor­rect for both the unknown of how many such house­holds are being missed and the second-​​order unknown of which can­di­date would be sup­ported by that unpolled household.

So, like any new polling firm, Gravis’s sta­tis­ti­cian tries to cor­rect for demo­graph­ics. What per­cent of vot­ers will be African-​​American? What per­cent will be between 18 and 25 years of age? What per­cent will have incomes between $25,000 and $35,000? Com­par­ing the IVR results to the expected vot­ing demo­graph­ics requires weight­ing some responses more, and oth­ers less. This is not actu­ally an easy task. How well do the African-​​Americans who respond to the poll rep­re­sent the ones who don’t? Is the Demo­c­ra­tic turnout in Iowa in 2012 going to be close to the Demo­c­ra­tic turnout in Iowa in 2008? Are all of those who claim to be likely to vote actu­ally likely to vote? Answer­ing these ques­tions requires the expe­ri­ence of hav­ing run dozens of such polls, val­i­dat­ing the results, and apply­ing the learn­ing to future iterations.

This is why com­ing into an elec­tion cycle three months before the elec­tion is a recipe for dis­as­ter. And there is no ques­tion that Gravis’s early polls were a dis­as­ter in terms of accu­racy. Kaplan read­ily agreed that the sta­tis­ti­cians he hired were inex­pe­ri­enced at han­dling polit­i­cal polling.

If you look at last week’s polls, how­ever, Gravis has been within the nor­mal bounds of main­stream polling firms. Col­orado, Florida, and Ohio were all within a point or two of the con­sen­sus, and with no dis­cernible lean. One week of three polls isn’t enough to say that Gravis is wor­thy of a top-​​tier place­ment, but it is at least a pos­i­tive sign.

Are Gravis’s polls legit­i­mate, then? At the very least, it’s hard for me to say defin­i­tively that they are faked. It’s cer­tainly pos­si­ble to do, but unlike Research2000, who didn’t have the infra­struc­ture to con­duct the polls, Gravis’s core busi­ness is that very infra­struc­ture. In their case, it might actu­ally cost more to fake the results, par­tic­u­larly if they’re using oth­er­wise idle resources to con­duct the polls. Is Gravis in the tank for the Repub­li­can Party? Pos­si­bly, but more recent topline results sug­gest otherwise.

At this point, my belief is that Gravis is a com­pany that got in over their heads at a really bad time to do so.

Even if I’m wrong, and they are noth­ing more than shills for the Repub­li­can Party, I’m still uncom­fort­able with the way that Kaplan has been per­son­ally attacked by many mem­bers of the Demo­c­ra­tic Under­ground and Daily Kos. When address­ing one’s pro­fes­sional actions, attack­ing one’s pro­fes­sional cre­den­tials is cer­tainly legit­i­mate. And if that per­son has a per­sonal con­flict of inter­est that influ­ences pro­fes­sional actions, those too are legitimate.

But is Kaplan’s DUI arrest from decades ago rel­e­vant with respect to his abil­ity to con­duct a legit­i­mate polit­i­cal poll?  I can’t see why it is. But what about the legal trou­bles he’s had in the past?

This is a lit­tle tougher to answer. He has clearly tried to build numer­ous busi­nesses that involve direct com­mu­ni­ca­tion mar­ket­ing, an area where it is easy to run afoul with the law. It is a field with a tremen­dous num­ber of reg­u­la­tions, and yet also a tremen­dous num­ber of small, inex­pe­ri­enced firms. We’ve all been on the receiv­ing end of unde­sired tele­phone calls, even after reg­is­ter­ing on the National Do Not Call List. Those of us who still have fac­sim­ile machines receive unso­licited fac­sim­i­les. And it appears that some such con­tact may have come from one of Kaplan’s firms.

Does it make sense that a per­son who has got­ten into trou­ble in the past for con­tact­ing peo­ple he shouldn’t would also be guilty of not con­tact­ing peo­ple that he should?

All in all, I don’t feel as if I’m in a posi­tion to judge the over­all legit­i­macy of Gravis Mar­ket­ing as a com­pany. Demo­c­ra­tic Under­ground will most likely con­tinue to con­duct their inves­ti­ga­tion into the com­pany and its offi­cers. I just hope that such an inves­ti­ga­tion is con­ducted with class.

Related arti­cles