Secondary menu

Consultation and Evaluation Practices in the Implementation of Internet Voting in Canada and Europe

Recommendations for Implementation of Best Practices for Consultation and Evaluation

The cases examined in this report and other research gathered to prepare this project lead to a number of recommendations about the consultation and evaluation procedures that could best be employed if future Internet voting trials are to be considered or implemented. The recommendations are designed to relate to the Elections Canada strategic goals of expanding trust, accessibility and engagement. Some of these recommendations are also applicable for jurisdictions seeking consultation to refine existing approaches to Internet voting or ameliorate present evaluation procedures associated with I-voting.

Many of the cases explored here have commonalities between the approaches used to conduct consultations and evaluation. However, they also possess distinct elements associated with discussion and assessment that merit individual strategies for Internet voting development in the coming years. Given that each jurisdiction, in Canada and internationally, faces unique contextual factors, recommendations should be considered with these in mind. While robust public consultation may be important in one area, it may be redundant or not needed in another. If public involvement strategies are incorporated, the type, design, and scope of the consultation method is best achieved without a one-size-fits-all approach. Although we cannot make specific recommendations or detail a recipe for Internet voting deployment or policy development with respect to consultation and evaluation practices, we are able to suggest some general recommendations that governments, election agencies and policy-makers may wish to consider adopting when considering Internet voting for the first time or refining existing models.


First, we recommend that a policy of openness and transparency be adopted when plans for Internet voting are being developed. Trust and transparency are closely linked. The more open and forthcoming a process, the more likely it is the public and other stakeholders will have faith in decision making and outcomes. Trust is essential to ensure confidence in the electoral process, which must be maintained to preserve the legitimacy and integrity of the system. Types of trust (social and political) are also needed to foster social capital, which helps promote citizen engagement – at the core of participation in elections.

Trust in government and changes to electoral institutions and processes are well supported by regular communication with the public and by ensuring electors are informed as plans proceed. In Canada, for example, in the case of the Town of Truro, election officials were able to foster trust in an electronic voting system through extensive communication and information efforts. This included disseminating pamphlets to every residential address, writing a regular column in the local newspaper for the six months prior to the election, hosting a booth in a local farmers' market, engaging the local media, and educating local librarians and nursing home workers about the process so that they could assist electors and act as ambassadors. The success of this approach is evident by the 140 percent increase in voter turnout the community experienced while doing away with in-person polling station voting. In Norway, too, transparency of the communication plan helped promote confidence in electronic ballots. Communication involved an open call for municipalities to participate, informational visits to the communities by the Ministry responsible for conducting the voting, local publicity, and an open trial operation of the system before voting day. These examples were unique to the locations mentioned, but had in common an openness to public input.

Our second recommendation is to adopt strategies that foster public engagement, although the scope and type of engagement should depend on the particular circumstances. Generally, there is a need for additional and improved forums for public engagement. Given that Internet voting is considered to be a citizen-centred service initiative and would have its success in large part judged on how well accepted and used it is by citizens, it is important that residents be consulted prior to its adoption. With a few exceptions, existing consultation mechanisms exist mostly with elites (i.e. government officials, policy-makers, technical experts and specific interest groups) but lack a general public focus. To expand the scope of consultation, jurisdictions might also consider using digital and mobile technologies to engage with electors, particularly through social media applications or another kind of online portal that enables virtual and interactive communication. If we are moving toward a more digital world and Internet-based service model, it seems only logical to facilitate public consultations through those same forums, in addition to more traditional approaches to public consultation, such as meetings or focus groups.

The Truro example cited above is a good illustration of the use of information and education-focused strategies to promote public engagement, but it should be noted that the town has a relatively small population to reach (with an electorate of about 10,000). For larger communities, additional tactics should be considered to involve the public. The Edmonton case, using public opinion surveys, citizen roundtable meetings, a mock election and a Citizens' Jury on Internet Voting, offers a good example of the use of additional options.

A third recommendation is to consider developing outreach programs that combine information about Internet voting with a focus on citizenship education more generally, emphasizing the importance of electoral participation for the renewal of democratic health. Such programs would fit well with school curriculums that teach civic education, and further the strategic goal of enhancing youth engagement with the electoral process. Unfortunately, we have not encountered examples of such educational programs for schools in our cases of study, but Markham and Truro did include some elements of an appeal to citizen participation in their informational campaigns. Whereas Markham focused on using digital technology to involve and engage electors, Truro employed traditional strategies by going out and speaking to people, and contemporary tools through its engagement of social media. In Europe, the tradition of electoral authorities encouraging electoral participation is less well established, and this is usually left to political parties.

Aside from a focus on engagement of the general public, a fourth recommendation is to involve and/or consult with other election stakeholders, notably candidates, the media, and if applicable, advocacy organizations. Techniques may include training sessions for political candidates (and political parties, if applicable), meetings with the media, or discussion with other groups. An Internet voting option particularly affects candidates. In communities where candidates have been consulted, they have been more likely to embrace Internet voting as an alternative voting method and even promote it in their campaign literature. Since the introduction of Internet voting option in 2005, Estonia has offered training sessions for political parties contracted to the e-Governance Academy in Tallinn. In Halifax, by comparison, candidates were more receptive to the technology after attending information sessions about Internet voting. Since Internet voting changes the nature of campaigns when it is made available in advance polls, by placing the emphasis on the beginning of the campaign instead of the end, candidates can be incorporated into the process or at least educated about it (Goodman in press). Candidates and media are particularly salient sources for elector information, making it important that this information is accurate.

Fifth, the timing is important. We recommend that discussions proceed on multiple levels at an early stage. Government administrators should provide discussion forums for elected officials and political parties (if applicable) as well as other groups. Once political support is established and additional research is conducted, time should be allocated to survey all stakeholders and consult with them openly. Conducting consultations before a steadfast plan is in place is important to ensure that feedback from stakeholders can be incorporated into program development and could be useful for building trust and support.

Sixth, the collection of systematic data using quantitative and qualitative methods is needed to gain information for future development. Specifically, information of this nature will be helpful for jurisdictions deciding whether to explore methods of Internet and electronic voting. If they decide against it, this information could help provide justification for not proceeding. If an area decides to move forward, data obtained for consultation purposes would help administrators and elected officials determine which digital options may be appropriate for that particular area. Data obtained from consultation practices can also be compared with information gathered through evaluation procedures, offering multiple data points for which administrators can assess the impact on the electoral process.

In all three European countries cited in this report – Estonia, Norway, and Switzerland – survey research has been conducted, but it has been at the evaluation stage, once the trials have been held. Canadian federal election studies, and other polls, ask at times about the hypothetical use of an Internet voting option, but little survey data have been collected from cases where Internet voting has been used. Edmonton is an exception, having obtained survey data through its public consultation initiative. In addition, as noted above, the City of Guelph commissioned a survey to consult with the public about voter experiences, the potential impact of Internet and telephone voting, and whether electronic voting methods would be desirable, as officials there determine whether to introduce an online voting option. The collection of additional data from Canadian municipal jurisdictions of varying characteristics (i.e. geographic location, size, eligible electorate) would provide a good start to conduct more thorough assessments and analysis. This information will be an important tool as governments increasingly grapple with the decision of whether to offer online voting.

In terms of communications methods, we recommend that electoral management officials utilize direct communication methods that are appropriate to the circumstances. For example, we would reference visits to remote communities by boat in Norway and the staffing of a booth at the farmers' market in Truro. Appropriate methods will depend on the contextual factors present and particular features of the jurisdiction. Places where large segments of the electorate congregate would be a good start. As well, a combination of traditional communications methods and some new strategies, particularly via digital and mobile technologies, is probably a wise approach to ensure as many potential electors are targeted as possible.

Finally, forming partnerships and building relationships with other jurisdictions, universities, research institutes, advocacy organizations, or practitioners can be extremely beneficial for collecting knowledge, conducting research, and generating strategies to deal with potential problems. Learning from the consultation strategies of others or working with partners to help develop unique and engaging consultation methods can make the process more cost-effective, efficient, and thorough. In Canada, Edmonton's partnering with the University of Alberta's Centre for Public Involvement is a prime example of this. Meeting with other jurisdictions has been beneficial, such as Edmonton hearing from Markham and Halifax, but greater collaboration and information sharing among jurisdictions and levels of government could be encouraged to save on information gathering, improve cost-effectiveness, and deliver greater insight for system development. This is true in a Canadian context and internationally. Areas such as New South Wales in Australia, for example, are currently working toward an online voting model, but find locating current and thorough research challenging.


The first and most important recommendation for jurisdictions is to develop rigorous evaluation frameworks. This means preparing a detailed plan in advance for how evaluation will be conducted and the criteria upon which the assessment will take place. Many of these criteria should be based on the core values of election operations in the jurisdiction and the goals of election administration. For many Canadian municipalities, evaluation procedures consist of varied ad hoc activities and/or compilation of written documents. There are some similarities across municipalities, but each evaluation system currently employed seems to be unique to that community. In many jurisdictions, evaluation processes should be made more systematic and could also benefit from the sharing of best practices and the adoption of similar evaluation frameworks among cities and towns. For the most part, existing protocols seem to work well for the officials who run them. Many officials have been around for a long period of time, know the area and culture well, and are extremely perceptive at analyzing the operations and making improvements where necessary. However, more rigorous processes are needed to assess Internet voting given that concerns about security, voter privacy, authentication, and fraud make it subject to greater public scrutiny.

In Europe, jurisdictions that conduct Internet voting options have benefited from formal evaluation reports conducted by the OSCE-ODIHR. These have been augmented by formal evaluation reports commissioned from independent institutes in the countries concerned. The report from the Institute for Social Research in Norway (ISF 2010) and Professor Pascal Sciarini of the University of Geneva in Switzerland (Sciarini 2011) are examples. Criteria used in these evaluations could usefully be scrutinized by other jurisdictions planning Internet voting trials.

The contextual areas discussed at the beginning of this report (in the Consultation and Evaluation in Context section) can provide a core set of criteria for evaluation. These are:

In addition, we recommend that internationally recognized standards of legality and fairness be applied as criteria for evaluation. These criteria will help ensure an accessible and fair process and are important for maintaining the integrity of the electoral process, particularly in light of the introduction of new technology.

Standards by which Internet voting is assessed are considered an addition to the evaluation criteria usually applied by an international organization like the OSCE-ODIHR for the entire election, which involve consistency with the legal framework in the country, respect for the right to vote, the rights for candidates to stand for election, the application of party and campaign financing laws, and the transparent and honest counting of the votes. Some basic criteria, such as the operation of the voter registration process, have particular applicability to the Internet voting option, when PIN numbers or codes are involved in authenticating voters. Finally, the security considerations are much more prominent criteria for the Internet voting operations than for polling place voting.

Coincident with more systematic evaluation frameworks is the second recommendation, for more thorough record keeping. In Canada, many communities share computer files, collect notes that are kept in a file folder or simply rely on officials attempting to remember evaluation details themselves. In Europe, the situation varies. Geneva keeps elaborate statistical details on the use of Internet voting in all the referendums and elections, but Neuchâtel provides a paucity of information. Estonia provides overall statistics on the use of Internet voting, but not further breakdowns. It is not clear, for example, how many people with disabilities were able to vote by Internet in any European or Canadian jurisdiction, despite the fact that ease of access of persons with disabilities to the polls is a frequent goal. To learn from evaluation, be able to share specific details, compare with other jurisdictions or compare with other elections over time, rigorous record keeping is essential.

Third, to be able to facilitate systematic evaluation we recommend that goals of Internet voting operations be specified in advance, in order to provide criteria for evaluations. In some cases it was noted that evaluation was difficult since desired outcomes had not been established at the outset. In other cases goals may have been set, but no criteria for their systematic observation were determined. At times, specific goals were avoided because doubts existed about the ability of the system to accomplish them, such as the case of increasing turnout. For election officials, thinking through goals is important not only to ensure systematic evaluation, but also to make sure that the type of Internet voting chosen (i.e. remote Internet, kiosk, Internet and telephone) and the way it is made available (i.e. advance polls or the entire election) are designed to complement those goals. In the first recommendation above, we have provided a set of criteria for evaluation, but not all of these may be specific goals established by any one jurisdiction. That is to be expected, but the key is to consider and specify the goals in advance.

Fourth, we recommend the systematic collection of post-election feedback. While most jurisdictions collect some feedback, key groups are often left out. For example, in some instances only paper ballot election staff are interviewed or surveyed, while broader information from citizens or other stakeholders is not gathered. Furthermore, some standardization of feedback collection will be helpful for comparisons over time and across jurisdictions. The additional information and insight will also improve the calibre of evaluation and model development. As with pre-election consultations, the use of both quantitative and qualitative methods is recommended. Ideally findings would be shared with other jurisdictions, levels of government, perhaps the public, and other relevant organizations or groups.

Fifth, we recommend that independent organizations be commissioned to provide evaluations, and independent audits of security, of Internet voting operations. Independent evaluations are useful for ensuring impartial evaluation, especially of security aspects. They are also helpful in fostering trust in the alternative voting approach. The evaluation report by the Institute for Social Research in Norway is an example of this. All Canadian cases explored here had an independent auditor assess the system and provide comments. In some cases this was an internal person to keep costs down. Ideally an independent person or group would be chosen to carry out this function.

Finally, as with consultations, partnerships at the evaluation stage are a key recommendation. Specifically, jurisdictions should consider establishing a co-operative network with local universities or research institutes to participate in evaluation of Internet voting trials; other actors such as industry or other groups could also be engaged. The case of the City of Edmonton and the Centre for Public Involvement is cited in this report as an example of a relationship where public consultation has been organized in such a way as to provide an objective approach. The evaluation conducted by scholars at the University of Tartu of the Estonian Internet voting system is another example. With respect to collaboration with industry, the post-election surveys and reports Markham has obtained are invaluable for evaluation and research. Partnerships of this nature can help ease research burdens or other costs associated with evaluation and can also help bring together experts for advice and analysis. Working with other jurisdictions and actors to streamline or standardize evaluation procedures could also be a long-term benefit.