Electoral Insight – International Electoral Co-operation
Improving Vote Count Verification in Transitional Elections
Eric C. Bjornlund
Principal, Democracy International
Despite the increasing popularity of exit polls in international election assistance programs, this article argues that "parallel vote tabulations" (PVTs) or "quick counts" are a more appropriate tool for verifying vote counts in transitional or post-conflict elections. PVTs – often conducted by local monitoring organizations – are generally more reliable than exit polls in political environments characterized by intimidation or uncertainty, as is typical in such elections. Although exit polls can provide insights about voter motivation, the use of exit polls to verify or project results often fails to serve the purpose of true democratization by undercutting the role of domestic organizations. To contribute more effectively to democratic elections, international donors and advisors must better understand these techniques.
The sophistication of vote count verification techniques developed since the 1980s has made it increasingly difficult for autocrats to manipulate the aggregation of election results without being exposed. International and domestic election monitors have developed effective techniques to detect this kind of fraud. Yet, despite years of experience and a large body of evidence, international organizations and experts have disagreed sharply about the appropriateness, effectiveness and reliability of particular vote count verification techniques. Different verification methods compete for resources and attention, sowing confusion and uncertainty in tense political situations. This poor coordination has threatened the international community's effectiveness in encouraging and monitoring democratic elections.
Since the 1980s, international and domestic election monitoring organizations have conducted parallel vote tabulations (PVTs) to assess the accuracy or verify the integrity of election results as reported by electoral authorities in transitional or post-conflict elections. In recent years, monitoring organizations have made increasing use of exit polls in such environments as well. In a PVT (also known as a quick count), local monitors observe the actual balloting and counting at polling stations and independently report the local results. Footnote 1 PVTs enable monitoring organizations to verify the aggregation (or "tabulation") of election results after the ballots are counted at polling stations. In an exit poll, researchers ask selected voters from a sample of polling places about how they have just voted.
But PVTs and exit polls have sometimes worked at cross-purposes. Exit polls sponsored by international groups may distract from PVTs conducted by domestic groups or may not be reliable in less-than-free political environments. Indeed, if reliable exit polls are possible in a given country, PVTs – which tend to be more expensive and difficult to organize – are probably not necessary. Where both PVTs and exit polls exist, the results of a reliable PVT should take precedence for vote count verification, and interested parties should look to exit polls primarily for insights about voter motivation as opposed to vote count verification.
Exit poll: A survey of voters exiting politically representative polling places, asking them about their ballot choices and motivations.
Comprehensive parallel vote tabulation or comprehensive quick count: An attempted forecast or verification of electoral results based on actual observation of the vote count in all polling places in an election.
Parallel vote tabulation or quick count: A forecast or verification of electoral results based on actual observation of the vote count in statistically significant, randomly selected polling places. Also called "sample-based parallel vote tabulation" or "sample-based quick count," to distinguish it from "comprehensive parallel vote tabulation" or "comprehensive quick count."
Domestic election monitoring organization: A non-partisan civil organization (or coalition of organizations) formed to observe and report on election processes in its own country.
Experiences from recent elections in Macedonia and Ukraine offer some important lessons about the need for better coordination among the sponsors of different election monitoring techniques. In Macedonia in 2002, a foreign-sponsored exit poll used to quickly project results overshadowed a well-executed PVT by a national group. This did little to advance the larger democratic development goals shared by all the organizations involved. In Ukraine in 2004, exit polls suggested fraud, but a PVT did not support this conclusion. Such discrepancies can hurt the credibility of election monitoring.
Comprehensive and sample-based PVTs (quick counts)
Official observers appointed by candidates join an international observer to keep a watchful eye on transparent ballot boxes in the southern city of Mykolayiv during the repeat of Ukraine's runoff presidential election on December 26, 2004.
The National Citizens' Movement for Free Elections (NAMFREL) in the Philippines pioneered the idea of an unofficial "quick count" to check the accuracy of the official ballot count. By aggregating results collected from individual polling stations throughout the country for the critical 1986 "snap" presidential election, NAMFREL called the officially announced results into question and provided the basis for international criticism of the process. Footnote 2 For the 1988 plebiscite in Chile on whether President Augusto Pinochet could continue in office, domestic monitoring groups working with outside advice added an important innovation to the quick count methodology: the use of statistical sampling. Footnote 3 Since then, multilateral organizations and domestic monitoring groups advised by international organizations have successfully used statistically based quick counts to verify election results in dozens of countries.
Statistical sampling is necessary for reliable verification because a comprehensive PVT, which attempts to collect all of the local election results in a country, generally cannot provide a basis for an assessment of the accuracy of the official vote count. There are two important reasons for this.
Eric C. Bjornlund is also the author of Beyond Free and Fair: Monitoring Elections and Building Democracy (Woodrow Wilson Center Press and Johns Hopkins University Press, 2004), which provides a comprehensive account of election observation in new democracies.
First, monitors can never collect results from all of the polling stations in a country, even under the best of circumstances, even with plenty of time and extensive resources. It is generally more difficult to obtain results from rural or harder-to-reach areas, which might have different voting patterns than urban areas. Because the missing data are not random, it is not possible, if the election is close, for a comprehensive tabulation to assess whether the reported vote count is accurate. Even collection of a large percentage of the results will be statistically skewed and potentially misleading.
Second, civic groups using a comprehensive methodology generally cannot process and interpret the enormous amount of data in a reasonable time after the elections. This is enough of a challenge for the government and the election authorities, with all the resources and authority they command. Accordingly, the results of a comprehensive PVT are not available in time to check the officially announced results.
A comprehensive independent tabulation nevertheless can serve constructive purposes, such as providing an organizational focus for volunteers, deterring vote count fraud and providing a basis for later investigation of claims of vote count fraud in particular localities. But an independent tabulation drawn from a statistically significant sample is both faster and more accurate than an independent count that seeks to obtain the results from all of the polling places in the country, which are inevitably incomplete and unrepresentative. Like an exit poll, a PVT can use statistical sampling to project results within statistically significant margins of error and compare them to results as reported. But a PVT differs from an exit poll in that it is based on actual polling results, as counted by election officials and witnessed by monitors, rather than on what individual voters report in interviews about how they voted.
Election workers in Fort Lauderdale, Florida, watch the November 7, 2000, election results after they have finished counting the ballots. On that election night, American television networks initially (and mistakenly) named Democratic challenger Al Gore the winner in the close presidential race, based on information from Florida exit polls.
Nevertheless, in recent years, domestic and international organizations have increasingly turned to exit polls to verify the officially reported results in transitional elections. They compare the findings of exit polls (and public opinion surveys) to results as reported by election authorities. Yet exit polls may not always be reliable, especially in post-conflict or transitional environments. And although they can offer insights into voter motivations, exit polls cannot generally be used reliably for verification. Exit polls are inherently too uncertain to justify questions about the credibility of official results.
Exit polls have been problematic and controversial even in the United States, where they have a long history. For one thing, there continues to be concern that the early release of exit polls will influence those yet to vote. More important, the reliability of exit polls is in question. In Florida in 2000, for example, television networks relying on exit polls first called the race for Al Gore, then later for George W. Bush, only to finally conclude that they did not actually know the results. In 2004, exit polls erroneously showed John Kerry leading in several key states, which would have made him president.
The validity of exit polls is particularly suspect in transitional or semi-authoritarian societies, where an historic climate of intimidation may make many voters unwilling to participate. The validity of any exit poll relies on the willingness of voters to report how they voted to a stranger. Indeed, international and domestic election monitoring groups invented PVTs in part because exit polls seemed inappropriate in the climate of intimidation that has often prevailed in transitional or post-conflict elections.
Foreign-sponsored exit poll overshadows PVT in Macedonia
In Macedonia in September 2002, a foreign-funded exit poll went ahead, even though the elections took place in an uncertain political climate in the wake of violent ethnic clashes the year before. Footnote 4 Violence, intimidation and extreme nationalist rhetoric had plagued the pre-election environment. These conditions necessarily raised questions about whether voters felt safe to express their political preferences and, thus, about the appropriateness and accuracy of an exit poll. Intimidation was so pervasive that, despite the country's population of only two million, the international community mobilized one of the largest international monitoring efforts ever, including some 800 observers from the Organization for Security and Co-operation in Europe.
At the same time, a non-partisan Macedonian election monitoring group, Citizen's Association MOST, conducted a PVT based on random samples of actual results and reported these findings for each of the six electoral districts. Their data provided a stronger basis for assessing the credibility of the official count. Footnote 5 Nevertheless, the media and international community initially ignored these valid data because an exit poll conducted and announced by an international organization had already provided the first public numbers. Greater co-operation among monitoring organizations involved in vote count verification, both national and international, could have reduced the chances that the exit poll would undercut the position of the local organization.
Because there was no significant controversy about the election results in Macedonia, the merits of the exit poll as a means of verifying the reported results were not called into question. At the very least, though, the use of an exit poll by one international organization while another was mobilizing a virtually unprecedented monitoring effort to counter a climate of intimidation suggests at least the existence of sharply different perspectives within the international community about what monitoring approaches were appropriate.
Questions about exit poll in Ukraine
Officials of Ukraine's election commission empty a ballot box after voting is over at a polling station in Kiev, during the first runoff presidential election on November 21, 2004.
In Ukraine, it is conventional wisdom that then-opposition leader Victor Yushchenko won the initial runoff presidential election on November 21, 2004. A Washington Post editorial on December 2, for example, declared, "Despite the government's brazenly unfair campaign, a majority of Ukrainians voted for… Yushchenko [and] authorities then tried to steal the election…" Footnote 6 But while international observers condemned the election process, their statements did not go so far as to assert a winner. Although these reports confirmed that the broader election process in Ukraine was seriously flawed, they offered little or no evidence that a majority actually voted for the opposition candidate.
It was an exit poll that gave the impression that the opposition had actually gained more votes. The Ukrainian election commission reported that the government's candidate, Viktor Yanukovych, won 15 million votes (49.5%) to 14.2 million (46.6%) for Yushchenko, but the exit poll by a consortium of local organizations found a 54-to-43% majority for the opposition candidate. Footnote 7 Supported by international donors, the poll surveyed 20,000 voters through ostensibly anonymous questionnaires. If this was accurate, the election commission's count overstated the result for Yanukovych by about 2 million votes, as he "really" received about 13.1 million of the 30.5 million votes cast, and correspondingly understated the result for Yushchenko by 2.2 million votes.
Yet, it would be extremely difficult to carry out such a truly massive amount of election day fraud by cheating at individual polling places. Contemporaneous reports by observers of problems, albeit extremely troubling, provided little support for the theory that several million votes were stolen a few dozen or a few hundred at a time. Rather, to carry out the extent of fraud implied by the exit polls would seem to require manipulation of the process of aggregating vote counts.
Supporters of opposition presidential candidate Viktor Yushchenko react favourably to the first exit poll results during a rally at the Independence Square in Kiev, on December 26, 2004.
To detect such manipulation, as it had in the past, the non-partisan Committee of Voters of Ukraine (CVU) conducted a PVT based on a random sample of actual polling station results from 1,500 polling places. But it declined to report PVT results after the election because, it said, the difference between the candidates was within the statistical margin of error. In other words, the PVT showed a close race and thus appeared to rule out extensive tabulation fraud. After Ukraine's supreme court ordered a new election, the CVU did release a detailed report on the fraud its observers had witnessed around the country. Footnote 8 Although these accounts leave little doubt that there were indeed widespread problems, they seemed inadequate in scale and scope to explain the difference between the results of the exit polls and the official count.
The international community never really knew which candidate actually received more votes in Ukraine's presidential election on November 21. Although Ukrainian and international outrage paved the way for a fairer election to take place on December 26, the failure of Western governments and the observers they funded to acknowledge the limitations of their analytical tools exacerbated the tension between Russia and the West and may complicate efforts to hold other countries to international norms.
Exit polls in Macedonia and Ukraine were accorded significant weight. Whether those exit polls were appropriate and their methodology sound remains open to question. For different reasons, these exit polls overshadowed well-executed PVTs in both countries.
Choosing appropriate tools and improving coordination
The experiences in Macedonia, Ukraine and elsewhere raise important questions about the appropriateness and effectiveness of different kinds of vote count verification techniques. To continue to deter or detect manipulation of the vote counting process, election monitoring organizations must continue to maintain the discipline of rigorous, robust verification of election results, and they must adapt to new technological and political challenges. Statistically based PVTs – which draw on much larger sample sizes than exit polls and are based on actual results, like comprehensive tabulations – will continue to be important in transitional societies lacking a history of successful polling or a fully stable, secure political environment.
Greater international co-operation is needed to consider the appropriate circumstances for PVTs, exit polls and other tactics to assess the legitimacy of vote counts in transitional or post-conflict elections. Variables in such a calculus will include the available budget, the salience of the election, the size and complexity of the country, the nature of the electoral system, the state of political development, and the capability of domestic election monitoring organizations.
Although international organizations, donors and advisors share the same goals for elections in new and emerging democracies, they sometimes work against one another. It is critically important for the relevant international organizations and experts to consider carefully the issues involved in designing, implementing, interpreting and assessing vote count verification exercises, and to attempt to agree on which vote count verification techniques are appropriate in which circumstances.
To make a continuing contribution to combating election fraud, PVTs and similar verification efforts must be publicly explained and well understood by authorities and international advisors. Legitimate concerns must be better addressed, and international actors in the democracy field have to try to learn from and co-operate with each other. Better coordination among donors and implementing organizations is essential to ensure the continued effectiveness of vote count verification in controversial elections.
Return to source of Footnote 1 Practitioners, donors and commentators have not maintained a rigorous distinction between the terms "parallel vote tabulation" and "quick count." In some regions and countries, the term "parallel vote tabulation" refers to a sample-based verification exercise, and the term "quick count" connotes a comprehensive tabulation. In other places, the term "quick count" has been used when the purpose of the exercise was to rapidly project results after the close of polls, rather than to later verify official results. For the purposes of this article, I use the two terms as synonyms.
Return to source of Footnote 2 NAMFREL, The NAMFREL Report on the February 7, 1986 Philippine Presidential Elections (Manila: National Citizens Movement for Free Elections, 1986); Lewis M. Simons, Worth Dying For (New York: W. Morrow, 1987), p. 247; David G. Timberman, A Changeless Land: Continuity and Change in Philippine Politics (Armonk, New York: M. E. Sharpe, and Singapore: Institute of Southeast Asian Studies, 1991), p. 147; National Democratic Institute for International Affairs (NDI), Reforming the Philippine Electoral Process: Developments 1986–88 (Washington: NDI, 1991), p. 48.
Return to source of Footnote 3 Larry Garber and Glenn Cowan, "The Virtues of Parallel Vote Tabulations," Journal of Democracy Vol. 4, No. 2 (April 1993), p. 100.
Return to source of Footnote 4 Williams and Associates, "Exit Poll Analysis: Republic of Macedonia 2002 Parliamentary Elections" (November 2002), available at www.iri.org/pdfs/MK_Exit_Poll.pdf.
Return to source of Footnote 5 Citizen's Association MOST, Monitoring Report: Parliamentary Elections 2002 (election report), pp. 26–28, available at www.accessdemocracy.org; "Citizen's Association MOST Announces Parallel Vote Tabulation Results in Four Districts" (press release, Skopje, September 16, 2002), available at www.accessdemocracy.org; Eric C. Bjornlund, Beyond Free and Fair: Monitoring Elections and Building Democracy (Washington: Woodrow Wilson Center Press, and Baltimore: Johns Hopkins University Press, 2004), p. 287.
Return to source of Footnote 6 "Democracy for Ukraine," Washington Post (editorial, December 2, 2004), p. A34.
Return to source of Footnote 7 Peter Finn, "Partial Vote Results Show a Tight Race in Ukraine Runoff," Washington Post (November 22, 2004), p. A15.
Return to source of Footnote 8 Committee of Voters of Ukraine, "Report of the Committee of Voters of Ukraine on Observation of Voting and Vote Tabulation on November 21, 2004" (report, December 2, 2004), available at www.cvu.org.ua, under the section on "Presidential" elections.
The opinions expressed are those of the authors; they do not necessarily reflect those of the Chief Electoral Officer of Canada.