If you watch tonight's vice-presidential campaign on CNN, make sure not to confuse pretty graphics with political insight.
The Perception Analyzer, used by the network to measure the response of a selected audience to the candidates, isn't — in the words of Daily Show host John Stewart — "a patronizing piece of made-up technology." But the network's use of the technology leaves much to be desired.
"It might be revealing, but probably not enough to justify it getting as much screen attention as the debate itself," said University of Virginia psychologist Brian Nosek.
At worst, rather than reflecting viewers' reactions to the debates, the technology could impact viewers' perceptions.
"Rather than being insulated from the spin, it could serve as a form of spin," said University of Nebraska political scientist Kevin Smith.
"I'm sure both presidential candidates would love to have one of their
'persuadables' in that group."
The analyzer itself consists of a dial turned from zero to 100 in accordance with a user's impressions. It's typically used to test marketing campaigns, but last Friday night the messages came from Senators John McCain and Barack Obama. The analyzers will again be used tonight and during the remaining presidential debates.
In its first presidential deployment, CNN gave analyzers to 32 registered voters, somewhat equally divided between Democrats, Independents and Republicans, who claimed to be undecided. They turned the dials as they watched the debate, and their average impressions showed up as three colorful lines scrolling beneath the candidates and across the TV screens of 7.4 million viewers.
The methodology was only thinly described by CNN correspondent Soledad O'Brien, and wasn't apparent to anyone who didn't watch the pre-debate coverage, prompting confusion among viewers. "You see the lines on the bottom of the screen, and if you're the viewer, you don't know what they mean," said David Paull, vice president of MSInteractive, the Perception Analyzer's developer and marketer. He said the company was not responsible for CNN's implementation of the technology.
Many experts were only slightly more charitable than Stewart: The Perception Analyzer is a valid way of measuring real-time reactions, but real-time reactions have limited political significance — and CNN's test group was too small to represent anything more than itself.
"It's not scientifically generalizable, or representative of populations at large, unless they used some type of representative sampling technique," said John Tedesco, a political communications researcher at Virginia Polytechnic Institute. "With a small group, like
32, you're not going to get representation."
Whether real-time impressions actually determine a voter's final perception of a debate, much less their vote, is questionable. Nosek said that judgments tend to be made retroactively, influenced by a few especially powerful moments. Tedesco cited research showing that people's opinions don't solidify until they've heard post-debate media analysis or participated in a focus group.
But if real-time impressions do matter, then CNN's viewers' own real-time impressions would have been influenced by 32 dial-turning people. "That real-time feedback has the potential to itself become a persuasion device," said Nosek — and in such a small group, a single person turning their dial all the way up or down can warp the average.
Paull was not concerned about outliers skewing the results, but said that serious statistical research would require a considerably larger test audience. The use of the Perception Analyzer for political news entertainment, he said, involves "different methodology from how researchers and pollsters will use the technology."
Image: Eyeliam
See Also:
- Obama Campaign Reveals Science Advisors
- Obama Answers Your Science Questions
- McCain Equates Embryos and Fetuses in Stem Cell Statement
- McCain Makes Sharp Right Turn on Stem Cells
- Neuropolitics Researchers Accused of Liberal Bias
- Conservatives Scare More Easily Than Liberals, Say Scientist
WiSci 2.0: Brandon Keim's Twitter stream and Del.icio.us feed; Wired Science on Facebook.
