Preelection poll accuracy and bias in the 2012 general elections.

Panagopoulos, Costas ; Farrer, Benjamin

Introduction

The 2012 presidential election witnessed Democratic incumbent Barack Obama triumph over Republican challenger Mitt Romney by 332-206 in the Electoral College and by 51 to 47% in the popular vote. Turnout among eligible voters was 58.2%, a slight but noticeable decline from the 61.6% participation rate in 2008 (McDonald 2013). Obama captured a second term with fewer electoral votes and fewer popular votes than he had accrued in the historic 2008 election cycle but became only the third Democrat to win a majority of the popular vote more than once. Most polls conducted prior to the November 6 election predicted such a result, with most showing a lead for Obama.

Overall, polls showed support for Obama remained fairly stable (relative to support for Romney) over spring 2012 and into the summer and began, in mid-August, to climb steadily for about a month before dipping downward until the final 10 days or so of the campaign when support strengthened anew (Panagopoulos 2013). Some analyses suggest events, including the conventions, presidential debates, and Hurricane Sandy, likely left lasting imprints on voter preferences, helping to explain the pattern of campaign dynamics observed over the course of the election cycle (Panagopoulos 2013). As Election Day approached, most reputable national polls showed a tight race between the two contenders, although most polls projected an Obama victory. Only a handful of polls suggested Romney would ultimately win, despite postmortem reports that internal polling for the Romney campaign had been much more optimistic even at this late stage (Silver 2012a).

In this report we help to assess accuracy and bias in the preelection polls conducted during the 2012 general election cycle. We summarize the accuracy of the final presidential, gubernatorial, and U.S. Senate preelection polls, conducted on both the state and national levels. We also place these findings in historical context.

National Presidential Preelection Polls in 2012

Polls were conducted nearly daily in the 2012 election cycle. A variety of different organizations conducted polls, ranging from newspapers and other media groups to think tanks, universities, and advocacy groups, as well as dedicated polling houses. Other polls were undoubtedly conducted by the parties and candidates themselves, but these were not made public. The poll aggregation website Pollster.com tracked 589 polls conducted between January 1, 2012, and Election Day (November 6). This made for an average of nearly two polls per day over this period, though most of these were concentrated toward the end of the cycle. In addition, thousands of state-level polls were conducted over the same period, asking not only about presidential preferences in each state, but also about gubernatorial and U.S. Senate contests.

We begin by assessing the accuracy of 21 final, national preelection polls for president conducted in the last week of the election cycle. We judge accuracy by three metrics: Mosteller et al.'s (1949) M3 and M5, and the A measure proposed by Martin, Traugott, and Kennedy (2005). M3 is the average absolute difference between the poll estimate and the final election result for each candidate. So, if a poll gave Romney 45% and Obama 50%, and the actual election result was 47% to 51%, then M3 would be 1.5 for this poll. M5 compares the polled margin between the two leading candidates to the eventual outcome margin between the same candidates and returns the absolute value of the difference between these margins. In the example above, M5 would be 1.

An alternative method for assessing poll accuracy was developed by Martin, Traugott, and Kennedy (2005). The new measure of predictive accuracy (A) is based on the natural logarithm of the odds ratio of the outcome in a poll and the actual election outcome (see Martin, Traugott, and Kennedy 2005 for a complete description). Among several advantages associated with this measure is the ability to compare accuracy across elections and polling firms and to detect the direction of bias because a signed statistic is produced (not an absolute value). A positive sign indicates a pro-Republican bias, while a negative sign indicates a pro-Democratic bias (Traugott 2005). (1) In the example above, A would be--0.12.

Table 1 presents the values for the three measures discussed above for each of the final, national 2012 polls we evaluate. The average value for the Mosteller Measure 3 is 1.72 for the 21 polls included in the analysis, while the average value for Mosteller Measure 5 is 2.72. Table 2 helps to situate poll accuracy in 2012 in historical context by presenting summaries of Mosteller's Measures 3 and 5 for elections since 1956 (see Panagopoulos 2009; Traugott 2005). The evidence reveals 2012 polls overall performed better than average against both the Mosteller Measure 3 (the average error for the 1956-2008 period was 1.9) and the Mosteller Measure 5 (the average error for the 1956-2008 period was 3.2) indicators. Still, both indicators imply 2012 polls were somewhat less accurate than polls conducted in the final days of the previous (2008) campaign.

Although there is no comparable time series of values for the measure of predictive accuracy developed by Martin, Traugott, and Kennedy (2005), the authors computed the average values of the statistic for 1948, 1996, 2000, and 2004. Martin Traugott, and Kennedy (2005) report that the average value of A for final preelection polls conducted in 1996 was--0.0838, suggesting a slight Democratic bias that overestimated Clinton's margin over Dole. In 2000, polls overestimated Bush support and the average value for A was +0.0630. For the 2004 election, the average value of A was--0.024, suggesting Bush's electoral margin of victory was slightly underestimated (see Traugott 2005). Panagopoulos (2009) reported an average value of A of--0.013 for the 2008 cycle, indicating polls reflected a slight but statistically insignificant Democratic bias.

In 2012, the average value of A (given the 21 polls included in our analysis) was +0.054, implying the final preelection polls were less accurate than in the prior three (2000, 2004, and 2008) cycles; moreover, polls in 2012 appeared to overestimate support for Romney. Assuming a tied election, our estimate of bias implies polls favored Romney by 1.35 percentage points on average. (2) However, the standard error (or S.E.) associated with the mean value for A we report for the full sample of national polls is .054, indicating the bias overall was not statistically significant; in fact, in 19 out of 21 cases, individual polls were not significantly biased in 2012. In two cases (YouGov and Gallup), however, A can be distinguished from zero; A is positive in both cases, implying both organizations' final poll projections significantly overestimated Republican support (by 0.9 and 2.5 percentage points respectively, assuming a tied election).

These analyses comport well with the postelection breakdown provided by Nate Silver (2012b) whose website, fivethirtyeight.com, was a popular source of aggregated information about polls in 2012. Although Silver used different criteria for inclusion (all polls conducted within the last 21 days of the election, rather than simply the final poll), he also rated Gallup, American Research Group, and Gravis Marketing among the least accurate polls, and Ipsos/Reuters and Angus-Reid as among the most accurate.

Statewide Preelection Polls in 2012

Beyond national polls, pollsters in 2012 also assessed statewide preferences for presidential as well as U.S. Senate and gubernatorial candidates over the course of the election cycle. Following the elections, the National Council on Public Polls (NCPP) compiled and analyzed a compendium of 404 final, state-level preelection polls conducted after October 17, 2012 (see NCPP 2013 for details and a complete list of polls included). The NCPP reported that most state polls (42%) were conducted by telephone using only live interviewers, while 31 % were conducted using Interactive Voice Response (IVR), 18% were Internet polls, 8% used a combination of IVR and human interviewers/Internet (mixed mode), and two polls were conducted by mail (NCPP 2013).

We use the complete set (3) of polls included in the NCPP report to assess predictive accuracy in statewide polls in 2012. Using each poll as a single (unweighted) observation, we present the frequency distribution of A in Figure 1. We note the polls include estimates of support for presidential as well as other statewide candidates (U.S. Senate and governor). In the absence of overall bias, we would expect the distribution to be centered on zero. The mean value of A in the complete sample of polls is +0.050, suggesting the pattern of pro-Republican bias detected in the national presidential polls also characterizes statewide polls as well, but the bias is not statistically significant at conventional levels (mean standard error = 0.081). Assuming all races were perfectly tied, this would translate into a percentage point difference (or Republican overstatement) of 1.25 percentage points (see note 2 above).

[FIGURE 1 OMITTED]

The measure of predictive accuracy (A) developed by Martin, Traugott, and Kennedy (2005) permits us to compare accuracy across a range of poll characteristics. Table 3 presents mean levels of (A) and the corresponding standard errors for statewide polls grouped by a variety of characteristics, including election type, survey mode, sample type, interviewing period, and sponsor. We begin by comparing the poll performance of 10 individual polling organizations that conducted at least 10 statewide polls and that polled in multiple (more than three) states. (4) Evidence presented in Table 3 reveals slight biases in a Republican direction can be detected for statewide polls conducted by all polling organizations but one (Pharos Research Group), (5) but the standard errors associated with these estimates indicate the biases are statistically insignificant for all polling organizations. This is consistent with the pattern established in our discussion of the final, national presidential polls and implies the pro-Republican bias in polling was a recurring feature in 2012. Overall, however, we conclude there were no significant biases in polling conducted by any individual organization in 2012.

Our sample includes polls conducted by partisan as well as nonpartisan organizations. Mean values of A for statewide preelection polls conducted by Democratic and Republican organizations and for all nonpartisan polling firms combined are presented in Table 3. We detect hints of pro-Republican bias across all three types of organizations in 2012, but none of these are statistically significant.

Using A, we can investigate bias in subsamples of polls by the level of electoral contests. In statewide presidential polls (N = 220), the mean value of A is +0.035, suggesting a slight but statistically insignificant (S.E. = 0.079) bias favoring Romney. Similarly, statewide polls in U.S. Senate races (N = 154) reveal a modest but statistically insignificant pro-Republican bias; the mean value of A is +0.067 (S.E. = 0.085). Statewide gubernatorial polls (N = 21) also appear to reflect a bias in favor of Republican candidates--the mean value for A is +0.083--but the bias is also statistically insignificant (S.E. = 0.081). We conclude from these results that accuracy overall was somewhat lower across all types of races in 2012 compared to 2008 (see Panagopoulos 2009), but bias was minimal in the final statewide polls conducted in 2012, whether examining statewide presidential, U.S. Senate or gubernatorial preelection polls.

We turn next to examining overall bias by poll mode. The key results are presented in Table 3- Mean values of A across modes suggest all polls reflected a pro-Republican bias on average regardless of how they were conducted, but none of the biases were statistically significant for any mode. The results suggest the two mail polls were the most accurate in 2012, while mixed-mode, state-level polls were about as accurate (A = +0.031), on average, as polls conducted by telephone (A = +0.037). Internet-based surveys were somewhat less accurate than these (A = +0.047), while IVR polls were least accurate (A = +0.075) on average in 2012. This contrasts with the finding that statewide IVR polls were most accurate in the 2008 cycle (Panagopoulos 2009).

There has been considerable debate in recent election cycles about the range of procedures employed by polling organizations in their respective estimations of likely voters in preelection polls (Erikson, Panagopoulos, and Wlezien 2004, Martin, Traugott, and Kennedy 2005). In 2012, all but three statewide polls we include in our analysis were based on samples of likely voters, but the evidence presented in Table 3 suggests polls based on registered-voter samples were about as accurate as likely voter samples; the biases, however, run in different directions: likely voter samples reflect a pro-GOP bias on average (A =+0.051) while the registered-voter samples reflect a pro-Democratic bias (A = --0.057) of roughly equal magnitude. This contrasts with previous studies (Panagopoulos 2009) that revealed the opposite pattern.

Longitudinal analysis can also be used to analyze the dynamics of poll accuracy, relative to electoral outcomes, and to gauge whether or not the predictive capacity of preelection polls improves as Election Day approaches. Scholarly evidence about the relationship between poll timing and accuracy is mixed; while some studies find accuracy improves over the course of a campaign (Crespi 1988), others find no significant impact of poll timing on accuracy (Lau 1994, Martin, Traugott, and Kennedy 2005; Panagopoulos 2009). Figure 2 presents lowess-smoothed levels of overall bias and accuracy in the 2012 statewide polls by the number of days until Election Day. The dashed line presents the smoothed pattern of the absolute value of A over this period and suggests preelection poll accuracy improved steadily within the final weeks of the election cycle (the absolute value of A trended toward zero). As in 2008 (Panagopoulos 2009), accuracy improved most dramatically during the final few days prior to the election. The solid line in Figure 2 plots lowess-smoothed levels of mean predictive accuracy (A) over the same duration. The pattern suggests statewide preelection polls reflected a stable, pro-Republican bias on average for most of the period we study, but this bias eroded in the final few days of the election cycle. In fact, the bias seems to have switched slightly in favor of Democrats in the last day or two of the 2012 cycle. These patterns reinforce the notion that overall poll accuracy and bias can change over the course of a campaign, particularly during the final campaign period we examine; substantively, however, these changes may not account for very much. We investigate this more rigorously below using multivariate techniques.

[FIGURE 2 OMITTED]

To explain overall levels of poll accuracy and bias more rigorously, we conduct a series of multivariate regression analyses. One advantage of A as a measure of predictive accuracy is that it (or variants of A as discussed below) is amenable to explanation using multivariate techniques that utilize various poll attributes as explanatory variables. The results of two such estimations are presented in Table (4). In both regressions, we include controls (fixed effects) for "house" (polling organization) and state effects. In model 1, the dependent variable is the absolute value of A; as such, higher values represent less accurate poll estimates, relative to election outcomes. The results of the regression analysis reveal statewide preelection polls for U.S. Senate candidates were significantly less accurate, and gubernatorial polls as accurate as statewide presidential polls (the excluded category) in 2012; U.S. Senate polls were also less accurate, compared to state-level presidential polls in 2008 (Panagopoulos 2009). We find no other evidence, however, that other poll attributes were related to poll accuracy in 2012. Neither mode, sample type, timing nor partisanship of the sponsoring organization significantly affected accuracy in statewide polls in 2012.

Model 2 presents the results of a probit regression analysis in which the dependent variable is coded 1 if the preelection poll reflected a pro-Republican bias (A > 0) and 0 if the poll reflected a pro-Democratic bias (A < 0). Using the same poll attributes as in model 1 to explain whether or not polls reflected a pro-Republican bias overall, we find that both U.S. Senate and gubernatorial polls were, all else equal, significantly more likely to be biased in a Republican direction in 2012, compared to presidential polls (the excluded category). A similar pattern was revealed in 2008 (Panagopoulos 2009). With controls in place for all factors, however, only internet polls were significantly (negatively) related to pro-Republican bias in 2012. This finding corroborates 2008, when both Internet and IVR polls reflected a significant pro-Democratic bias (relative to phone surveys) (Panagopoulos 2009).

The multivariate analyses described above are useful in that they reveal the impact of a range of poll attributes on overall accuracy and direction of bias while simultaneously controlling for other poll characteristics. In several instances, the results may cause analysts to reconsider, update, or confirm initial conclusions about the impact of various factors on accuracy and bias in preelection polls. For example, the interview period does not appear to influence overall levels of accuracy or bias. Moreover, the multivariate results, in both 2008 and 2012, reveal U.S. Senate polls are less accurate, at least compared to presidential polls, but that both U.S. Senate and statewide gubernatorial polls consistently reflect significant biases favoring Republican candidates. By contrast, polls conducted via Internet appear to be consistently biased in a pro-Democratic direction. The accumulation of such findings can foster improvements in polling methodology as well as poll interpretation.

Conclusion

Preelection polls throughout the 2012 election cycle painted a portrait of a tight presidential election cycle in which voters preferences were roughly evenly split between the two major-party contenders. This scenario, along with close statewide contests at all levels, fueled considerable interest in poll projections and sustained a high level of poll volume once again in 2012. In addition, advances in polling methodologies and lessons learned from previous cycles enabled survey researchers to refine and hone their methodologies to obtain preference estimates that were as accurate as possible. On the whole, preelection polls performed quite well in 2012, as they had in previous cycles, with a healthy record of correctly projecting election outcomes. Polls overall, however, were somewhat less accurate in 2012, at least compared to recent presidential election cycles-- and especially compared to 2008--suggesting sources of inaccuracy merit continued scrutiny. Beyond that, polls more or less across the board in 2012 consistently reflected pro-Republican bias; even if there is scant evidence that these biases were significant, this pattern suggests systemic contextual factors may account, at least in part, for potential bias in poll results. This possibility warrants further attention and consideration. On a more optimistic note, significant differences in accuracy or bias across survey mode or other poll characteristics scarcely arise in our analyses, a finding that may reflect improvement in preelection polling methodology. Analysts are wise to continue to monitor these patterns rigorously in the pursuit of still greater improvements in accuracy and bias.

References

Crespi, Irving. 1988. Pre-Election Polling: Sources of Accuracy and Error. New York: Russell Sage Foundation.

Erikson, Robert, Costas Panagopoulos, and Christipher Wlezien. 2004. "Likely (and Unlikely) Voters and the Assessment of Poll Dynamics." Public Opinion Quarterly 68 (4): 588-601.

Lau, Richard. 1994. "An Analysis of'Trial-Heat' Polls during the 1992 Presidential Election." Public Opinion Quarterly 62: 2-20.

Martin, Elizabeth, Michael Traugott, and Courtney Kennedy. 2005. "A Review and Proposal for a New Measure of Poll Accuracy." Public Opinion Quarterly 69 (3): 342-69.

McDonald, Michael P. 2013. "Presidential Voter Turnout Rate (Eligible Voters) 2012." United States Elections Project. http://elections.gmu.edu/Turnout_2012G.html (accessed September 3, 2013).

Mosteller, Frederick, Herbert Hyman, Philip McCarthy, Eli Marks, and David Truman. 1949- The Preelection Polls of 1948: Report to the Committee on Analysis of Pre-Election Polls and Forecasts. New York: Social Science Research Council.

National Council on Public Polls [NCPP]. 2013. "Analysis of Final 2012 Pre-Election Polls." http:// www.ncpp.org/files/NCPP%20Election%20Poll%20Analysis%202012%20-%20FINAL%20 012413.pdf (accessed September 3, 2013).

Panagopoulos, Costas. 2009. "Preelection Poll Accuracy in the 2008 General Election" Presidential Studies Quarterly 39 (4): 896-907.

--. 2013. "Campaign Effects and Dynamics in the 2012 Election." Forum 10 (4): 36-39. Silver, Nate. 2012a. "When Internal Polls Mislead, A Whole Campaign May Be to Blame." New York Times, December 1. http://fivethirtyeight.blogs.nytimes.com/2012/12/01/when-internal-polls -mislead-a-whole-campaign-may-be-to-blame/ (accessed September 3, 2013).

--. 2012b. "Which Polls Fared Best (And Worst) in the 2012 Presidential Election Race."

New York Times, November 10. http://fivethirtyeight.blogs.nytimes.com/2012/ll/10/which-polls -fared-best-and-worst-in-the-2012-presidential-race/?_r=0 (accessed September 3, 2013).

Traugott, Michael. 2005. "The Accuracy of the National Preelection Polls in the 2004 Presidential Election." Public Opinion Quarterly 69 (5): 642-54.

Winter, Deena. 2012. "NE: Questions Swirl Around Pharos Pollster." Nebraska Watchdog. October 31. http://watchdog.org/60775/pharos-pollster/ (accessed September 3, 2013).

COSTAS PANAGOPOULOS

Fordhatn University

BENJAMIN FARRER

Binghamton University

(1.) Polls reflect no bias when A equals zero.

(2.) See Martin et al. (2005, 11, note 11) for a discussion and formula used to convert the parameter A to a percentage point difference.

(3.) We exclude seven of the presidential polls because there was another conducted by the same polling organization in the same state that was completed later in the campaign and one U.S. Senate poll (Survey USA poll in California) because it was fielded starting October 16. Following NCPP (2013), we treat PPP and PPP(D) as distinct entities.

(4.) This is consistent with the approach adopted in Martin et al. (2005, 362) and Panagopoulos (2009).

(5.) Pharos Research Group was a relatively new polling firm that attracted some skepticism about its practices in 2012 (Winter 2012).

Costas Panagopoulos is an associate professor of Political Science and Director of the Center for Electoral Politics and Democracy at Fordham University.

Benjamin Farrer is a predoctoral research associate at the Center for Electoral Politics and Democracy at Fordham University and a Ph.D. candidate in the Department of Political Science at Binghamton University.

TABLE 1
2012 Final National Preelection Poll Results and Poll Accuracy
Estimates

                                   Field                        Sample
Firm                               Dates      Romney   Obama   Size (N)

Election Result                               47.2     51.0
Democracy Corps (D)             11/1-11/4     45       49       1080
Pew Research                    10/31-11/3    45       48       2709
ABC/Washington Post             11/1-11/4     47       50       2345
Angus-Reid                      11/1-11/3     48       51       1019
Ipsos/Reuters                   11/1-11/5     46       48       4725
YouGov                          10/31-11/3    47       49      36472
PPP (D)                         11/2-11/4     48       50       1200
Daily Kos/SEIU/PPP              11/1-11/4     48       50       1300
Purple Strategies               10/31-11/1    46       47       1000
NBC News/Wall Street Journal    11/1-11/3     47       48       1475
UPI/C VOTER                     11/3-11/5     48       49       3000
IBD/TIPP                        11/3-11/5     49       50        712
CNN/ORC                         11/2-11/4     49       49        693
Monmouth/SurveyUSA              11/1-11/4     48       48       1417
Po/iftro/GWU/Battleground       11/4-11/5     47       47       1000
Washington Times/JZ Analytics   10/29-10/31   49       49        800
Newsmax/JZ Analytics            11/3-11/5     47       47       1041
American Research Group         11/2-11/4     49       49       1200
Gravis Marketing                11/3-11/5     48       48        872
Rasmussen                       11/3-11/5     49       48       1500
Gallup                          11/1-11/4     49       48       2700
Average

                                Mosteller   Mosteller    Predictive
Firm                            Measure 3   Measure 5   Accuracy (A)

Election Result
Democracy Corps (D)             2.1         0.2         -0.008
Pew Research                    2.6         0.8          0.013
ABC/Washington Post             0.6         0.8          0.016
Angus-Reid                      0.4         0.8          0.017
Ipsos/Reuters                   2.1         1.8          0.035
YouGov                          1.1         1.8          0.036
PPP (D)                         0.9         1.8          0.037
Daily Kos/SEIU/PPP              0.9         1.8          0.037
Purple Strategies               2.6         2.8          0.056
NBC News/Wall Street Journal    1.6         2.8          0.056
UPI/C VOTER                     1.4         2.8          0.057
IBD/TIPP                        1.4         2.8          0.057
CNN/ORC                         1.9         3.8          0.077
Monmouth/SurveyUSA              1.9         3.8          0.077
Po/iftro/GWU/Battleground       2.1         3.8          0.077
Washington Times/JZ Analytics   1.9         3.8          0.077
Newsmax/JZ Analytics            2.1         3.8          0.077
American Research Group         1.9         3.8          0.077
Gravis Marketing                1.9         3.8          0.077
Rasmussen                       2.4         4.8          0.098
Gallup                          2.4         4.8          0.098
Average                         1.72        2.72         0.054

Note: To be consistent with previous years' analyses of poll
accuracy, we include poll estimates produced within the final
week of the election. The analysis above excludes polls that
completed interviewing prior to October 31,2012. We acknowledge
this approach excludes final preelection polls from several,
prominent national outlets including: CBS News /New York Times
(final poll fielded 10/25-10/28; A = 0.056); National Journal
(final poll fielded 10/25-10/28; A = /0.028); Fox New? (final
poll fielded 10/28-10/30; A = 0.077); NPR (final poll fielded 10/
23-10/25; A = 0.098) and AP/GfK (final poll fielded 10/19-10/23;
A = 0.121).

TABLE 2
Average Errors in Presidential Polls, 1948-2012

                  Number    Number of
Year             of Polls   Candidates   M3    M3

1956                i           2        1.8   3.5
1960                i           2        1.0   1.9
1964                2           2        2.7   5.3
1968                2           3        1.3   2.5
1972                3           2        2.0   2.6
1976                3           3        1.5   2.0
1980                4           3        3.0   6.1
1984                6           2        2.4   4.4
1988                5           2        1.5   2.8
1992                6           3        2.2   2.7
1996                9           3        1.7   3.6
2000               19           3        1.7   3.5
2004               19           2        1.7   2.1
2008               20           2        1.5   1.5
2012               21           2        1.8   2.8
Yearly Average                           1.9   3.2
(1956-2012)

Note: Data for 1956-2004 period obtained from Traugott (2005,
649); 2008 from Panagopoulos (2009), and 2012 update compiled
by authors.

TABLE 3
Mean Predictive Accuracy (A) by Poll
Characteristics, 2012 Statewide Polls

                                             Mean
Poll Characteristics          Number      Predictive    Standard
(Type/Sponsor)               of Polls    Accuracy (A)    Error

Presidential                   220          0.035        0.079
U.S. Senate                    154          0.067        0.085
Governor                        21          0.083        0.081

Democratic                     104          0.062        0.079
Nonpartisan                    281          0.045        0.082
Republican                       5          0.038        0.093

Likely Voters                  392          0.051        0.081
Registered Voters                3         -0.057        0.098

Interactive Voice Response     122          0.075        0.075
Internet                        71          0.047        0.075
Mail                             2          0.010        0.054
Mixed                           32          0.031        0.092
Phone                          168          0.037        0.087

Sponsors
Angus-Reid                      10          0.053        0.093
Marist                          14          0.039        0.067
Mason-Dixon                     12          0.045        0.089
PPP                             15          0.078        0.076
PPP (D)                         25          0.034        0.066
Pharos Research                 15         -0.015        0.079
Rasmussen                       31          0.085        0.086
SurveyUSA                       24          0.031        0.093
We Ask America                  13          0.035        0.061
YouGov                          46          0.062        0.071

Note: Following Martin, Traugott, and Kennedy (2005; Panagopoulos
2009), only polling organizations that conducted at least 10
statewide polls and covered at least three separate states in
2012 are included in the analysis.

TABLE 4
The Impact of Poll Attributes on Bias and
Accuracy in Statewide Preelection Polls, 2012

Independent variables:             Model 1:      Model 2: Pro-
(Poll characteristics)             Accuracy     Republican Bias

U.S. Senate                          0.04 ***       0.71 ***
                                    (0.01)         (0.27)
Governor                             0.02           1 79 ***
                                    (0.02)         (0.69)
Effective sample size                0.00           0.00
                                    (0.00)         (0.00)
Registered voters                    0.07            --
                                    (0.10)
Nonpartisan Polling Organization     0.03          -1.13
                                    (0.02)         (0.91)
Internet                             0.00          -6.04 ***
                                    (0.09)         (1.57)
IVR                                  0.15           2.75
                                    (0.11)         (1-93)
Mail                                 0.07            --
Mixed                                0.09           0.83
                                    (0.09)         (1.40)
Days to Election                    -0.00           0.04
                                    (0.00)         (0.05)
Constant                            -0.23          -0.93
                                    (0.18)         (1.94)
N                                  394            215
[R.sup.2]/Pseudo [R.sup.2]           0.58           0.34
Log Likelihood                       --           -91.76

Notes: Model 1: OLS. Dependent variable is the absolute value of
A; Model 2: Probit. Dependent vari- able = 1 if A > 0, and 0 if A
< 0. Observations/fixed effects that predict outcome perfectly
are excluded from the model, resulting in the smaller number of
cases.

*** signifies statistical significance at the p < .01 level,
two-tailed. Standard errors in parentheses.