In the last days before the 2016 presidential election, most forecasts were predicting an easy win for Democrat Hillary Clinton, based on polling results that had consistently shown her ahead of Republican candidate Donald Trump, nationally and in several key states. During the campaign, however, self-administered (e.g. internet) polls such as the experimental USC Dornsife / LA Times 2016 Election “Daybreak” poll, often showed higher levels of support for Trump compared to interviewer-administered (e.g. telephone) polls. These results were suggestive of the possibility of “hidden” Trump voters who might not want to admit they were voting for Trump, or who might be avoiding participating in polls. The idea of hidden voters roused little interest, however, given the outspoken nature of many Trump supporters. Trump’s stunning Electoral College win gave renewed credence to the idea that ‘shy’ voters may have played a critical role in the failure of pre-election polling to anticipate the outcome.
The Bradley Effect
Concern about hidden voters skewing polling results is not new. In the early 1980s, African American candidates in three states outpolled their actual vote by surprising margins, a phenomenon discussed as the Bradley, or Wilder, effect (e.g. Hopkins, 2009). The Bradley effect was named after the African American mayor of Los Angeles who was running for Governor of California in 1982, and lost despite being ahead in the polls. The effect was thought to arise from voters who either prevaricated out of concern that they could be perceived as racist by the survey interviewer (social desirability bias), or avoided participating in polls altogether (differential nonresponse bias).
While the candidacies of Barack Obama and Hillary Clinton in the 2008 presidential elections raised the specter of a potential Bradley effect around either race or gender, none was detected. A 2009 study of elections in which one or more candidates were African American or female concluded that social desirability bias had declined over time; the author declared the Bradley/Wilder effect to be “political history” (Hopkins, 2009). In 2012, Obama under-polled his vote, heaping more fuel on the funeral pyre of the Bradley effect.
A Shy Trump Voter Effect?
During the primaries, Trump had slightly outpolled his vote overall. Separate post-primary analyses by fivethirtyeight.com and New York Times concluded that social desirability had not been a factor (Enten, 2016a; Cohn, 2016). However, as the contentious general election campaign wore on, there were some concerning indications. In August, FiveThirtyEight.com reported seeing higher Clinton vote forecasts when models excluded self-administered polls, particularly in red states (Enten, 2016b). GOP “insiders” in a Politico study in October asserted that mainstream Republicans would not want to admit to backing the controversial candidate (Shepard, 2016).
The experimental internet Daybreak poll, among the same panel of participants each week, often showed higher levels of Trump support than other polls. We determined to investigate if hidden voters might account for some that difference. Other researchers were also curious. The week before the general election, the authors of a Politico/Morning Consult study which conducted simultaneous self-administered and interviewer-administered surveys noted evidence of hidden voters, but in insufficient numbers “to spring a surprise victory for [Trump] on Election Day.” (Easley, 2016)
The findings from the Daybreak poll were more conclusive. During the last week of October, we asked our participants to rate their comfort level with disclosing their presidential candidate choice to family, close friends, acquaintances, to telephone pollsters, and to our own poll. Ratings were from 0, meaning extremely uncomfortable to 100, meaning extremely comfortable.
As illustrated in Figure 1, average comfort levels were lowest when talking to telephone pollsters, followed by acquaintances. Respondents were relatively more comfortable talking to family and friends, and the higher mean of 86 for our poll might serve as an upper bound, given the potential for positive bias.
Our analysis revealed that some groups, particularly Trump voters, those in rural areas and small towns, and voters who did not graduate from college, were least comfortable talking to telephone pollsters. For the vote-based analysis reported here, we merged the participants’ self-reported vote from our post-election survey. We report only statistically significant results, employing regression tests for significance at the .05 level or greater.
Voters who cast a ballot for Trump or voted for a third party candidate (means of 54 and 51, respectively) were less comfortable talking to telephone pollsters, compared those who voted for Clinton (mean of 62). We observed a particularly pronounced difference in urban zip codes, where comfort levels for Trump’s supporters were 26 points lower on average than for Clinton’s (means of 45 and 71 respectively).
Overall, comfort levels among self-reported city dwellers averaged 10 points or higher than in other areas (mean of 64). Voters who reported living in small towns and rural agricultural areas were the least comfortable (means of 51 each). The widest gap was between women in rural zip codes and men in urban codes (means of 46 and 60 respectively).
Overall, college grads were more comfortable than non-graduates (means of 60 and 50 respectively). This split applied mainly to white voters, however. Average comfort level for whites with and without college degrees differed by 15 points (63 to 48), but the difference in comfort levels between non-whites with and without degrees was not statistically significant.
Conclusions
Our analysis indicates that Trump voters were significantly less comfortable talking to telephone pollsters about their candidate choice than Clinton voters, in the last weeks of the election campaign. While this data does not provide conclusive evidence that higher levels of discomfort led voters to mislead or avoid telephone pollsters, it may provide some insight into why polls in states with Trump-supporting groups such as lower educated, rural whites did not detect the levels of Trump support that gave the Republican candidate enough electoral votes to win the election despite losing the popular vote.
We conclude that it is important to rely on multiple sources of data obtained using different methods and modes during a campaign. Each type of poll has strengths and limitations. Supplementing information from telephone polls, which can be subject to social desirability bias, with information from rigorously conducted self-administered polls may help reduce overall error and support more accurate predictions of results on election day.
References
Cohn, N. (2016, May 17). Is Traditional Polling Underselling Donald Trump’s True Strength? The New York Times. Retrieved from https://www.nytimes.com/2016/05/18/upshot/is-traditional-polling-underselling-donald-trumps-true-strength.html
Easley, C. (2016, November 3). Yes, There Are Shy Trump Voters. No, They Won’t Swing the Election. Retrieved December 7, 2016, from https://morningconsult.com/2016/11/03/yes-shy-trump-voters-no-wont-swing-election/
Enten, H. (2016, May 20). Trump Supporters Probably Aren’t Lying To Pollsters. Retrieved from https://fivethirtyeight.com/features/trump-supporters-probably-arent-lying-to-pollsters/
Enten, H. (2016, August 31). Live Polls and Online Polls Tell Different Stories About The Election. Retrieved from https://fivethirtyeight.com/features/live-polls-and-online-polls-tell-different-stories-about-the-election/
Hopkins, D. J. (2009). No More Wilder Effect, Never a Whitman Effect: When and Why Polls Mislead about Black and Female Candidates. The Journal of Politics, 71(3), 769–781. https://doi.org/10.1017/s0022381609090707
Shepard, S. (2016, October 28). GOP insiders: Polls don’t capture secret Trump vote. Retrieved December 7, 2016, from http://politi.co/2eMmUuc
You must be logged in to post a comment.