Quinnipiac University’s Polling Institute in Hamden has gained national recognition for its public opinion polls. It's been highly rated for accuracy in predicting primary and general elections. It's also cited regularly by major news outlets around the country.
The polling process, however, is continuously being analyzed -- and there are more challenges than you might think.
Public opinion polls are everywhere. They're indispensable for researchers, politicians, and news organizations who watch them closely when it comes to finding out what people think about presidential candidates or policy issues.
Campaign managers change their strategies based on them.
But they often differ -- sometimes dramatically.
I recently visited the Quinnipiac Polling Institute and watched as over 100 pollsters -- wearing headsets sitting in front of computers at partitioned desks -- asked questions like:
If the election were held today and the candidates were Hillary Clinton the democrat and Donald Trump the republican for whom would you vote?
Doug Schwartz has been director of the poll, which is a year-round operation, for more than 20 years. He said several things set the Q Poll apart from other polling organizations.
It regularly polls in nine states and New York City as well as nationally.
"There are a lot of polls that just poll in their home state, or they just do national polls," Schwartz said. "So as far as I’m aware, we’re the only poll that regularly polls in so many different places."
The Q Poll also conducts swing state polls. And it adheres to so-called traditional gold standard methodology, meaning pollsters use live callers to call land lines and cell phones, as opposed to computer generated or internet polls.
It’s an expensive undertaking that’s not without challenges, specifically when it comes to mobile-only households, said Schwartz.
"They tend to be disproportionately young people," he said. "Also, we’ve seen in the data that blacks and Hispanics are more likely to be cell phone only. So you’re going to underrepresent them. You’re going to underrepresent young people if all you do is call land lines."
Cell phones pose other challenges, too: people not answering when they don’t recognize the calling number, or moving out of state and keeping an old number.
That doesn’t work if you’re trying to collect data from Florida, but the person you’re reaching now lives in Connecticut.
It's all a sign of the times, said Kathleen Weldon of the Roper Center for Public Opinion Research at Cornell University.
"If the phone simply doesn’t ring in your house, and you don’t answer it when it does, that is very different than it was 40 years ago," said Weldon. "The phone rang, you answered it. But now that isn’t necessarily true. So we have to adapt and make changes every time there’s a technological or social change."
Of course, some of these challenges add to poll discrepancies.
For example, some pollsters use Robopolls, which are prohibited by law from calling cell phones. That’s a lot of potentially unrepresented people.
Methodology is certainly a major factor when it comes to discrepancies, said Schwartz, along with sample size and selection. But other details factor in as well.
"Part of it is margin of error that can explain differences," he said. "Part of it is how a pollster might weight their data, or select likely voters, or the question order, or even the question wording."
And that’s where it becomes an art. Question wording and order can be misleading, said Weldon.
"It may be something that by the end of 20 questions about the problems of smoking you are convinced smoking is the number one health problem facing the country," she said. "But we’re not sure that you would have said that if we asked you, in an open-ended response at the beginning of a questionnaire, What’s the most important health issue facing the country?"
A simple word can have an impact, said Schwartz, as they found with polls on gun control.
"We took out the word control and did, Do you think gun laws should be made stronger?" he said. "And we find that there’s more of an appetite for stronger gun laws. Not as much of an appetite for stronger gun control. It made a difference -- just a word."
Weldon said that kind of experimentation is both necessary and useful, which is why transparency in polls is important.
"Every time there’s an election pollsters have this opportunity to test their methodologies against the real world and improve their accuracy," she said.
Most pollsters agree a good poll is both an art and a science. But the ultimate poll, Weldon said, is election day.
