NEP Survives Its Baptism by Fire

Nov 8, 2004  •  Post A Comment

Most members of the National Election Pool agreed the country shouldn’t toss the exit-polling baby out with the bath water, even though the National Election Pool disseminated some misleading data on Election Day.

There were no miscalls by news organizations that received the data, TV news and polling executives said. The TV news organizations’ systems-refined statistical models, revamped decision desks and additional safety nets-worked overall, even when some of the cogs didn’t. For example, the NEP principals-ABC News, CBS News, CNN, Fox News, NBC News and the Associated Press-were unable to receive NEP data for a brief spell late Tuesday night because of an NEP computer glitch, but the principals still were able to maintain coverage.

The consortium created NEP in 2003, after its creaky predecessor, the Voter News Service, caused embarrassing miscalls in 2000 and a computer meltdown in 2002. Under the new setup, the AP does all the raw vote collecting, which proceeded without a hitch Tuesday.

The NEP exit polling process had sailed through test runs in the primary season without a hitch.

NEP principals were concerned but not dismayed that the maiden general election run for NEP did not go so well.

The first delivery, in the early afternoon of Nov. 2, of exit poll information to NEP members and client news organizations suggested the voting wind was at the back of Sen. Kerry.

On closer inspection, TV news veterans of election coverage, who were well aware that early data is incomplete, sensed discrepancies with history and with their own statistical models. By the time the second wave of data arrived, around evening-news time, it came with cautionary flags from NEP on data from some half-dozen states. Some TV news executives noticed what seemed to be inexplicable differences between the first and second batches of exit information and between the pre- and post-vote trends.

The actual vote totals became essential to projections by networks-each of which retooled its decision desks and statistical models after 2000.

“Anytime you see something that is not what you expect, you treat it cautiously,” said CBS News Director of Surveys Kathy Frankovic.Routine postmortems were under way less than 12 hours after the lead anchors left the air, with the 2004 presidential race unresolved though not in doubt until Sen. John Kerry conceded to President Bush Wednesday.

Fox News Channel, at 12:41 a.m., and NBC News, a few minutes later, projected that Ohio would go to President Bush, which raised his electoral count in their books at that point to 269, where it stopped. It was the diciest call of the night, but both networks said internal statistical models encouraged their decision desks to make the call, which would be their last of the night. ABC, CBS and CNN declined to do the same, since provisional ballots still to be counted quite likely outnumbered the votes separating President Bush, in first place, from Sen. Kerry.

Questions about how exit-poll data could have so muddied the trends began early, especially among and about the bloggers, who had, with little or no cautionary notes, posted the early (and, by definition, incomplete) data that had been leaked to them.

But the questions were inevitable. And they were, perhaps, most bluntly answered at Fox News. Fox News media critic Eric Burns told “Studio B” anchor Shepard Smith: “We shouldn’t be trusting them again.”

“Clearly there were issues,” said Marty Ryan, executive producer of election coverage at Fox News, which commissioned its own phone survey of close to 10,000 voters Tuesday, as it had for backup in 2002.

One of those who feels that TV newsmen were able to safely make their projections and refine estimates throughout the night is Dan Merkle, the ABC News decision desk director, who said, “The data comes in waves, and the early waves are more variable than later data.”

By Thursday night NEP principals had received a summary of areas that will be investigated by Joe Lenski and Warren Mitofsky, of Edison Media Research and Mitofsky International, respectively, who had been hired to collect and compile the exit poll data. The issues, some perennial, will range from whether survey takers were able to get essential access to polling places to whether Democrats were more inclined to talk to poll takers or whether additional adjustments for absentee and early ballots must be made.

When a definitive report on problems and necessary fixes might be concluded is unknown, but there was widespread confidence that NEP can improve the process and the mechanisms for the 2006 and 2008 elections.

“We were satisfied that we were able to interpret the data with some reasonable sense of timeliness. I don’t think the public was greatly inconvenienced,” said NBC News VP Bill Wheatley, who said exit polling about who votes for what and why produces snapshots of history and is “a worthy and important thing to do.”

But if Mr. Wheatley has his way, clients and members of NEP will have to wait longer next time to receive their first exit poll data. That would limit the time unauthorized and untutored recipients have to misinterpret.

“In these seemingly polarized times peoplewill be skeptical of any data that doesn’t reflect their points of view,” said Mr. Lenski, who said most people do not understand that projections of outcome is not the primary use for exit polling data. “That’s a fundamental misunderstanding.”