Secondary menu

Consultation and Evaluation Practices in the Implementation of Internet Voting in Canada and Europe

Canadian Cases of Internet Voting

City of Markham

The City of Markham first made the decision to introduce Internet voting more than a decade ago. City officials were concerned about turnout, but a more pressing rationale was that they wanted to move toward a service model that was citizen-centered and make the election process more accessible and convenient for electors. The city was also able to negotiate a special rate from Election Systems & Software (ES&S) to become first major municipality in Canada to trial Internet voting. In this way, Markham broke ground as a local leader in Internet voting and e-government. Since 2003, Markham has continued to offer Internet voting as an alternative voting method in the advance poll portion of three consecutive local elections. Furthermore, it has managed to develop the most comprehensive approach to evaluation of all the Canadian jurisdictions. Markham's approach to consultation, however, has been much like that of other communities in the sense that election stakeholders have been mostly informed of Internet voting plans after the fact, once details were approved by elected members and solidified by municipal administration, and no open consultation was carried out (Froman, November 12, 2012; Huycke, December 21, 2012).

Markham conducted considerable research prior to proceeding with Internet voting, which included speaking with other jurisdictions and hiring a Ryerson University professor to write a risk analysis report that explored the increased risk associated with using an online voting system compared with other methods of voting and that urged consultations with Internet voting vendors and information technology companies. Markham has continued to stay abreast of Internet voting developments; its recognition as a municipal leader in I-voting has attracted many requests for discussions with other jurisdictions, which has resulted in frequent consultations. In addition, travelling to speak and conversing with officials from other areas about Internet voting has helped to increase Markham's knowledge base and keep city officials informed of any Internet voting trends or emergent developments elsewhere. Markham has also extensively consulted with Toronto-based digital strategy firm Delvinia regarding education and outreach campaigns related to Internet voting before each election. In 2010, this involved crafting a multi-channel social media strategy using Facebook, Twitter, YouTube, and Flickr to inform and engage the public (Goodman and Copeland 2011). The public was not consulted about aspects of the Internet voting model, but they were able to ask questions and communicate with city officials through these channels. Greater stakeholder engagement in Markham has been incorporated into the evaluation process.

Markham partnered with Delvinia in 2003, 2006, and 2010 to collect survey data from voters to learn about and better understand the motivations to use Internet voting and the impacts of doing so. Survey questions were developed with the help of expert market researchers and, in 2010, with the help of academics. Markham also played an important role in question selection. In particular, city officials ensured that key items probing feedback regarding accessibility were included. In all election years an exit-poll survey was made available to online voters once they had successfully cast their ballots. Participation was optional and questions probed information such as satisfaction with Internet voting, use of the Internet and knowledge of computers, and likelihood of future use locally and at other levels of government. In 2010, a survey of candidates was added and distributed to all candidates. Completion was voluntary. Questions focused on candidate satisfaction with Internet voting as an alternative voting method, the effects of online voting on the campaign process, and other benefits or drawbacks they encountered as a consequence of Internet voting. The surveys developed through this partnership have given Markham the benefit of examining data on the effects of Internet voting over time.

Delvinia compiled and issued reports analyzing the survey data after all three election cycles. Markham officials were given access to these to help with evaluation and in understanding the effects of online ballots on voters and candidates. Other data analysis, including demographic analysis, and any additional surveys or research conducted by Delvinia was also passed on to the city as part of their collaborative relationship. Markham also issued its own survey to candidates in 2006 and 2010 to assess what worked and what did not with respect to the election, including Internet voting (Huycke, December 21, 2012).

In terms of documents and reports, the Markham election team conducts "lessons learned" sessions throughout the election process. This includes one session after the election and some informal meetings throughout to debrief key election officials on any important issues. A final report is prepared about this debrief and presented to council after the election. In addition, a "lessons learned and feedback document" is saved on a shared staff drive so that it is accessible to everyone. Any important information or details about how the election proceeded can be recorded here and is reviewed afterward (Huycke, December 21, 2012).

Finally, two types of audits were carried out for Internet voting in Markham. Prior to the 2010 election, a security company, under a confidentiality agreement, conducted an audit of the Internet voting vendor's program code. Although this review did not identify any weaknesses, a few enhancements were made to the Markham model as a result. One notable change, made as a result of evaluations of Internet voting operations in previous elections, was to add date of birth (DOB) as a security credential for online voting registration. Previously only a personal identification number and the creation of a personal security question had been required. During the Internet voting period, and afterward, an IT staff member who was independent of the clerk's department conducted an overall process audit. The auditor was provided with PIN codes to check that the system was performing as expected. Since Markham used the same vendor as the Cape Breton Regional Municipality did in its most recent election, the process to close down the online polls is similar and described in more detail later on (Huycke, December 21, 2012). Markham has plans to use Internet voting once again in its 2014 election.

Halifax Regional Municipality

With an electorate of about 310,000, the Halifax Regional Municipality (HRM) is the largest municipality in Canada to have offered Internet voting to electors in a binding election. HRM first offered Internet and telephone voting options in its 2008 regular municipal election and in a 2009 by-election. These alternative methods were again used in the regular election of 2012 and have now become an accepted means of participating in the municipality's electoral process. The primary reason for using electronic ballots in HRM was to enhance the accessibility of the electoral process and to establish the viability of electronic voting (Goodman, Pammett and DeBardeleben 2010).

Prior to the introduction of Internet voting, HRM conducted extensive research and consulted with other jurisdictions that had piloted e-voting. Despite requests, however, municipal officials did not provide specific details about the approaches used for consultation and evaluation prior to 2012. HRM continues to monitor Internet voting developments and practice. Much like Markham, HRM is recognized as a model case given its size, the fact that it trialled Internet voting early and its experimentation with different Internet voting techniques (i.e. offering I-voting through the advance period up until and on election day in the 2009 by-election). As a consequence, HRM frequently consults with other jurisdictions. Most recently, city administrators observed the City of Edmonton's Citizens' Jury on Internet Voting and played a key role as a presenter in a gathering of all 54 Nova Scotia municipalities organized by the provincial department Service Nova Scotia and Municipal Relations (SNSMR) along with the Association of Municipal Administrators, Nova Scotia (AMANS) (White, December 17, 2012).

HRM Council ultimately makes the decision as to whether Internet voting is offered as a voting channel. Formal consultations with election stakeholders were not conducted prior to the 2012 election, but forums for feedback existed and stakeholders were provided with information about Internet voting through the pre-election process. Information sessions for candidates were held in the spring as well as a session for city councillors wherein attendees were able to ask questions or offer comments. The majority of questions from candidates centred on the campaign and how Internet voting impacted the traditional process. More generally, HRM received some questions about security and anonymity (McKinnon, December 11, 2012). Discussions were also conducted with HRM's accessibility committee to discuss plans for Internet voting and prospective poll locations for paper balloting.

As the 2012 election neared, social media, particularly Twitter, was used to reach out to electors, seeking to inform and engage them. This method was predominantly used to remind electors of the electronic voting options, to indicate where physical poll locations were, and to clarify the documentation required for registration. However, some electors asked questions back, even those located overseas (McKinnon, December 11, 2012). Overall, there was not a perceived need on the part of HRM staff to reach out to the public or other stakeholders, probing opinions or thoughts, since Internet voting has been offered twice previously and electors and other relevant parties were assumed to be aware of the option (McKinnon, December 11, 2012).

Other activities were undertaken during the pre-election process that could be considered consultations, notably discussions with technical experts. HRM hired a security company to conduct tests and try to expose vulnerabilities in the Internet voting system as they developed the proof of concept. Aside from work with the third party and technical experts from the vendor, development of the system and consultation with IT personnel was conducted internally (McKinnon, December 11, 2012). Council was also consulted prior to the election. A report was presented to Council in September 2011, which detailed the Internet voting options, the costs associated with the plan, and any legislative amendments that would be necessary to proceed. The only major change discussed was the extension of the advance voting period (McKinnon, December 11, 2012).

In terms of evaluation, the primary rationale for offering Internet voting was to improve accessibility, but assessing the effectiveness of this goal proved difficult due to measurement problems. Turnout, by comparison, is much simpler to assess, although a positive effect on voter participation was not necessarily a central goal. In the 2008 HRM election 10 percent of overall turnout came from ballots cast electronically during the advance-voting period (the time during which e-voting was offered). During the longer e-voting period offered in 2012 (sixteen days instead of four), 22 percent of all ballots cast were cast in advance electronically. Another 15 percent chose to vote via paper ballot on election day. While turnout remained 37 percent overall in both elections, the proportion of voters using e-voting increased substantially. Although not a direct goal, the impact on electoral participation is an important one and certainly positive effects are a key consideration in the election review process for HRM officials.

HRM's election post-mortem involved city election staff, internal IT, communications, and marketing groups. Meetings also took place with an accessibility committee to provide feedback on all aspects of the election – those related to Internet voting and traditional, paper ballot proceedings. HRM also received a good deal of public opinion through unsolicited e-mails and phone calls. Municipal officials did not reach out to citizens for feedback this time, but would consider doing so in the future.

As the 2012 evaluation was proceeding, there was a judicial recount. An initial recount was performed by HRM staff at the request of the second-place candidate in district 3, who challenged the results in that riding given that there was a difference of only six votes (a fraction of a percent) between himself and the winner. HRM returning officer, Cathy Mellett, indicated that the recount revealed a mistake had been made at one poll station wherein the results had been called in twice. The recount declared the second-place candidate as the rightful winner with a 68-vote lead. Shortly after, an application was put forward for an official judicial recount and a Supreme Court of Nova Scotia judge supported the request (CBC News 2012).

A procedure for recount of the Internet voting component was established in 2008 with the adoption of the HRM e-voting bylaw (HRM 2008). The recount process (protocol) permits the Returning Officer to re-run the electronic results report and therefore effectively recount all of the electronic votes in this way. The judicial recount confirmed that the second-place finisher was actually the rightful winner of the race. The results of the e-vote tally did not differ in the recount (McKinnon, December 11, 2012). A report was prepared on the 2012 recount in HRM district 3, which explained the circumstances for the challenge and provided details of the recount procedure. A Specified Auditing Procedures report was compiled by Ernst & Young. This report is a pre-defined list of procedures to verify that the system maintains the integrity of the e-voting electoral process. In this case the vendor also provided HRM with a formal report that included information regarding the number of electors who voted, how the votes flowed in, and how many votes were cast online compared with those cast by telephone (Crutchlow, November 29, 2012). Finally, a report was prepared for HRM council and Service Nova Scotia and Municipal Relations by HRM staff. An oral debrief was also delivered at the post-election meeting of Nova Scotia municipalities, organized by SNSMR and AMANS (McKinnon, July 15, 2013).

With regard to the evaluation of security, HRM chose an electoral board to help oversee the security process in addition to an independent auditor. This board consisted of two auditors from Ernst & Young, an election officer, the assistant returning officer, and the returning officer. Each member of the electoral board was given a piece of a master key that been broken into smaller pieces and saved on smart cards. These cards were kept separate to ensure no overhead administrator could tamper with the election and these cards were put together or "reconstituted" after the election to tally the results. In addition to the auditor's role as a key holder, and to compile an official report, the auditor in HRM's 2012 Internet voting model actually began work long before the election, participating in the development of the e-voting proof of concept (Crutchlow, November 29, 2012).

Finally, in terms of cost assessment, keeping within budget is a useful means of evaluating expenditures. HRM election coordinator, Lori McKinnon, was pleased, commenting on the evaluation of the financial component of the election by noting that with the incorporation of Internet voting, HRM "ran a pretty lean election this time around" (McKinnon, December 11, 2012). The cost of the Internet and telephone portion of the election was $450,000 in 2012, the same as in 2008. This amount included the Internet and telephone voting contract, external auditor, and technology consulting and equipment. The total cost of the election remained a constant $1.6 million in 2008 and 2012. Since there were more eligible electors in 2012, the cost of carrying out the election on a per-voter basis decreased in 2012 (McKinnon, May 22, 2013).

Cape Breton Regional Municipality

Cape Breton Regional Municipality (CBRM) is the second-largest municipality in Nova Scotia, with about 83,000 electors. CBRM offered remote Internet and telephone voting for the first time in October 2012 as part of its advance poll period, which took place from October 9 to 16. On election day, electors were able to vote only by paper ballot at a polling location. CBRM made the decision to introduce Internet voting primarily because officials were increasingly concerned with falling levels of voter turnout. While national averages of voter turnout for Canadian cities and towns range between 25 and 30 percent, turnout in the CBRM area was upwards of 70 percent in the 1990s but dropped to 50 percent in the 2008 election (White, October 15, 2012). Since levels of turnout have traditionally been robust in this community, the trend of decline was especially worrisome for the municipality. Having followed HRM's experience closely, and observing the positive impact on turnout in other communities such as Markham, council added the alternative voting method in advance polls (White, October 15, 2012).

CBRM engaged in a couple of different types of consultations prior to implementing online voting. First, it consulted heavily with other Canadian jurisdictions that had successfully implemented Internet voting programs, notably HRM and Markham. Officials from the CBRM election office also sat on the steering committee for HRM to participate in discussions regarding Internet voting (White, October 15, 2012). A great deal of discussion was also conducted with different information technology companies and Internet voting vendors. This was done to learn about the different systems and processes available, to help determine which approach would be most beneficial for the CBRM community. Research with both sectors had gone on for some time.

CBRM had considered implementing electronic voting for the 2008 election, but had refrained because of a proposed electronic plebiscite that would ask residents as a pilot test for e-voting whether they wanted a reduced council. Officials did not want the public associating the advent of online voting with the controversial topic of downsizing the council and so delayed the introduction until 2012. Ironically, the proposed e-vote plebiscite never went ahead (White, October 15, 2012).

CBRM has established no systematic process of election evaluation, but the incorporation of Internet voting and the additional elements and issues that come along with it has led to staff rethinking that they may need to create one. Generally, all election issues and results are well documented and an official meeting is held to discuss these items with key staff members. Poll workers are also surveyed for their input and feedback from their experiences. In addition, discussions take place with senior citizens' clubs, candidates, and more informally with the media. All of this information is placed in a file, where staff members can add information or notes, and this is kept until preparation for the following election begins. Assessing the response of the electronic voting call centre and call centre staff was something new for 2012, specific to the introduction of electronic voting (Campbell, October 15, 2012; White, October 15, 2012).

Also unique to Internet voting is the audit or evaluation process that takes place once the electronic ballot box closes. Different vendors may suggest that jurisdictions follow alternate processes, but the main companies offering service in Canada require the election office or government to hire an independent auditor. CBRM hired Debbie Rudderham, Chief Information Officer for Cape Breton University, to carry out the audit and provide an independent report commenting on the security of the system. Evaluation for the auditor starts before the election begins. Rudderham was required to test the system before the election commenced, during the voting period, and once it closed to ensure electors were able to cast ballots when the electronic polls were open, but unable to vote when the online voting period had ended. The system used in CBRM provided by Intelivote included an "auditor module," which allowed the auditor to view the audit votes and see whether they had worked or not (Smith, October 16, 2012).

Once the electronic polls were officially closed, Rudderham, the auditor, waited for any voters who accessed the system prior to closing time to complete their selections, cast their ballots, and officially exit the online voting portal. She then printed a summary of the votes and placed the document in a sealed envelope, which was signed by all observing parties. A separate copy of the vote count report was put on an encrypted flash drive that was password protected. Municipal Clerk and Returning Officer Bernie White kept the file, which was locked in a safe, while the Assistant Municipal Clerk, Deborah Campbell, kept the password but was unable to access the file. This process ensured that the e-voting results were kept secret until it was time to tally the ballots after the polls closed on election day. In addition to the vote count report, which is used to calculate the results on election night, Rudderham prepared a formal report outlining the safety, security, reliability, and accessibility of the electronic voting system (Smith, October 16, 2012; White, October 15, 2012).

In terms of other documents, a brief Report of the Returning Officer was prepared. In addition, summaries of public feedback – usually the best and worst experiences with the voting process – were kept and summarized. An oral report was also compiled and delivered by CBRM Returning Officer Bernie White to the same election post-mortem that Halifax participated in, which was sponsored by SNSMR and AMANS and included all 54 municipalities in Nova Scotia. Internet voting and its effects was one of the main topics of focus addressed at the November 23, 2012, session. Election officials had a particular interest and focus on reviewing stories from across the province from the fifteen Nova Scotia communities that used e-voting technology in the October 2012 municipal elections. Comments regarding the implementation of the new technology were mostly positive. Official minutes were taken, creating a historical record (White, December 17, 2012).

Town of Truro

In the 2012 local elections, the Town of Truro offered remote Internet and telephone voting to its 10,000 electors for a ten-day period (from October 11 to 20). Internet voting was the only option available to electors, and paper ballots were no longer used. The Town of Truro was drawn to electronic voting because officials saw an opportunity to showcase themselves as progressive and technological leader in the area. Since four other Nova Scotia municipalities had used Internet voting in the previous (2008) election, Truro officials waited to observe the experiences in those communities before deciding to try it themselves. Election organizers in Truro wanted the town to be one of the first municipalities in their area to offer Internet and telephone voting in place of paper ballots until and on election day, and to make their mark on Internet voting development in Canada. Aside from the attraction of being seen as a technological leader, the primary motivation for pursuing electronic voting was to increase voter turnout. Improving convenience and accessibility for electors as well as reaching the younger demographic were also considerations that led to its adoption.

Elections officials consulted with other municipalities that had trialled Internet voting prior to adopting it. In particular, they consulted extensively with the Town of Cobourg, Ontario, because the two communities are approximately the same size and Cobourg had also offered I-voting only in the 2010 municipal elections (Pearson, October 17, 2012). The two towns communicated by conference calls throughout the e-voting development process. This included a call in the early stages while preparing the e-vote proposal for council, another as the public education campaign was formulated and another call before the voting period began. Studying and learning from the approach used in Cobourg was very influential for Truro. In addition to holding consultations with other jurisdictions, and discussions of the proposal with Truro Town council, Truro officials actively sought advice from the local media (Henderson, May 21, 2013).

Immediately after gaining approval from council to introduce Internet and telephone voting, Returning Officer Jud Pearson approached the local newspaper for feedback. Pearson and the newspaper engaged in discussions about the nature of Internet voting, how it would work and what it would mean for the electoral process in Truro. Initially, there was some resistance, but through discussion and explanation the newspaper agreed to allow Pearson to write a regular column once a month for the six months leading up to the election. This column educated the citizens of Truro about the importance of elections, voting, Internet voting, and the potential to make Truro a leader in digital technology in Nova Scotia. During the election, Pearson continued to work with the local media, particularly the newspaper, by allowing them to publish turnout statistics by ward at the end of each voting day. Though not strictly speaking consultation, these communications helped to get the town population interested in the election and motivated electors to give Internet voting a try (Henderson, October 17, 2012; Pearson, October 17, 2012).

Officials also sought to engage the public through social media by setting up Facebook and Twitter pages, where citizens could comment, ask questions, or obtain information. The Truro election team also went to the local farmers' market for a weekend, where about one third of the town's population congregates, and set up a booth to talk about Internet voting with electors (Pearson, October 17, 2012). While the decision to proceed with Internet voting had been made and these forums were educational in nature, they served an important engagement and public feedback function. In terms of candidates, those not in office did not have a direct say in whether to implement Internet voting, but were provided information early on. Many made use of this knowledge and became advocates for Internet voting, educating electors through their own campaign pamphlets. Overall, consultations about whether or not to proceed with Internet voting were undertaken with council, other jurisdictions, and the media, but there was a robust and consistent effort to make contact with election stakeholders to educate them, provide information, and allow for comments, questions, or feedback. Election officials were keen to implement a thorough education and awareness campaign early on to ensure that a wide range of election stakeholders felt as though they were part of the process and part of something special.

Since increased turnout was the primary goal of the adoption of Internet voting, evaluation of the Truro election, officials explained, would predominately rest on whether voter participation notably increased. In fact, the turnout rose 140 percent from the 2008 election, increasing from 19 percent to 47 percent. This difference is rather remarkable given the older populace, many of whom learned to use a computer for the first time. Part of this success may be attributed to the fact that four special public access points were set up in high-traffic areas around the town: at the library, recreation centre, visitor information centre, and a computer kiosk in Millbrook, a First Nation community. Staff at these locations were trained and educated about the system so that they were able to help electors with the online voting process.Footnote 4 Portable computer kiosks were also taken to nursing homes, where staff members were officially sworn in as deputy returning officers so that they could go to an elector's bedside and assist with the voting process. With respect to achievement of goals, officials noted that evaluation largely rests on the turnout numbers (Henderson, October 17, 2012; Pearson, October 17, 2012).

Aside from turnout, officials conducted further evaluation of the election by collecting articles from newspapers and hearing comments from the Chief Administrative Officer. They also discussed the possibility of using the online survey service Survey Monkey or focus groups to gain greater public feedback. Many comments came on their own, however, through phone calls, e-mails, or cards. A meeting was also held with candidates to collect their thoughts and those workers in nursing homes and at other public access points who had been trained to assist voters. Feedback from these actors was deemed a priority for election officials to ensure online voting had improved accessibility for electors (Pearson, October 17, 2012).

The audit portion of the election worked much the same way as in the Cape Breton Regional Municipality. Discussions took place with the auditor and a report was issued regarding the evaluation of election security and its general operation (Pearson, October 17, 2012). One issue that would likely need to be discussed further as part of evaluation was the quality of the election lists obtained from the Province of Nova Scotia, which is maintained using data from Elections Canada, the SNSMR's motor vehicle registrations, vital statistics and other government sources (Henderson, April 5, 2013). Election staff seemed keen to expand and further develop election evaluation as a consequence of Internet voting and discussed some ideas, especially about obtaining public feedback. No plan was concretely outlined, however.

In addition to the evaluation carried out in the Nova Scotia communities highlighted in this report, the province hosted two evaluation sessions of its own. These sessions did not specifically address Internet voting per se, but they related to issues that impact the operation of Internet voting. For example, Elections Nova Scotia orchestrated one session, which focused on the gathering of information, maintenance, and improvement of existing voters' lists. The election office of Service Nova Scotia and Municipal Relations held the second meeting that discussed recommendations and suggestions for potential revisions to the Municipal Elections Act (Henderson, April 5, 2013).

City of Edmonton

The City of Edmonton has not trialled Internet voting in a binding election, but, in co-operation with the University of Alberta's Centre for Public Involvement, it undertook a rigorous consultation process and pre-trial evaluations to help formulate a decision about whether or not to incorporate Internet voting as an alternative voting method in future elections. To date this process represents the most comprehensive public consultation initiative conducted on Internet voting. The decision to consider an Internet voting option resulted from a desire to make the electoral process more accessible and reduce barriers to voting (Sinclair, November 25, 2012).

Another motivator for city officials, very much in line with the contextual consideration in European jurisdictions, was the desire to be recognized as an international leader with respect to Internet voting and e-government (Sinclair, November 25, 2012). In Canada, the City of Markham has stood out as a community that is taking a leadership role with respect to online voting and adopting principles of e-government. Edmonton wanted to make a name for itself much as Markham has, and possibly do better (Kamenova, September 9, 2012). Enough ground has been broken in Canada that deploying Internet voting is not treading uncharted waters, but the concept is still novel enough that there is opportunity for communities from various areas to sign on and develop unique approaches or refined systems and make a name for themselves as leaders in online voting, Internet service delivery, and e-government.

Although there were other technical elements, the bulk of Edmonton's approach in the consideration of Internet voting for future elections was focused on public consultation. This included questionnaires, a mock Jellybean Election where electors could trial an Internet voting system, roundtable advisory meetings with stakeholders and the first-ever Citizens' Jury on Internet Voting in Canada whose verdict and recommendations were presented directly to elected representatives. The rationale for developing such a robust consultation process was twofold. For one, preliminary research conducted by the city suggested that for Internet voting to be truly successful and accepted by citizens, they needed to be meaningfully engaged beforehand. A high rate of Internet connectivity did not guarantee public acceptance and support, so a commitment was made to not only inform residents, but also solicit their opinions (Kennedy, January 4, 2013).

For the past several years, the city had been working to innovate and learn in the area of public involvement. In 2009 it jointly established the Centre for Public Involvement (CPI) with the University of Alberta, which was to "be a hub of excellence in the theory and practice of public involvement" (Cavanagh, January 8, 2013). The presence of the Centre was a factor in deciding on an extensive public consultation process, in that councillors were encouraging its use. The Centre had organized several previous public consultation programs in the form of citizens' panels. In this case, the decision was made to develop a Citizens' Jury, a method that seemed better suited for the complex topic of Internet voting because it provided participants with more evidence-based education (Cavanagh, January 8, 2013).

The Centre recruited and worked with a Research Committee of professors who helped craft survey items and assisted in the design of the Citizens' Jury process. One member and co-author of this report, Nicole Goodman, composed an Issues Guide, which provided an assessment of key issues surrounding the topic of Internet voting based on available literature and the experiences of various jurisdictions. This guide was used as the primary document to inform the Jury. A Citizens' Jury Advisory Committee was also created independent of the Research Committee to oversee the decisions being made with regard to the Jury process.

Six questionnaires were created in total by the Research Committee, which contained attitudinal questions relating to Internet voting, elections, past voting participation, and demographic items. Two of these surveys (a pre- and post-citizens' survey) were written for the general public to complete online and designed to gauge general attitudes of the public toward Internet voting. The pre-survey was made available from September 1, 2012, to December 9, 2012. Links were provided on the CPI website, the City of Edmonton website, and an Edmonton Journal article on the topic. The results of the pre-survey were made available to the city for its records and evaluation after the Citizens' Jury proceedings. The post-survey was sent out via e-mail to respondents about six weeks later to see if opinions had shifted in light of the Jury outcome or other information that had been printed in media sources or websites. A brief summary of the responses and questions compiled by CPI Postdoctoral Fellow and Research Director, Kalina Kamenova, is included in Appendix 2 (Kamenova, January 14, 2013).

Two similar survey instruments (of the six) were created specifically for stakeholders (including the public), or special groups of citizens, with whom consultation roundtables were organized. The city designated these special groups of citizens to be: the elderly, persons with mobility issues, and persons with disabilities (Kamenova, January 14, 2013). Participants completed one survey before participating in the roundtable and another afterwards. The final two questionnaires were designed for the Citizens' Jury. The first acted as a recruitment questionnaire, to ensure that chosen participants had a distribution of attitudes similar to that of the general Edmonton public. The second was a post-survey that participants were asked to submit upon completion of the Jury (Cavanagh, January 8, 2013; Kamenova, September 9, 2012).

An initial test of the Edmonton Internet voting system was made in a novel way, as a non-binding election trial to choose a favourite candy colour. The Jellybean Election served as a test for the public to evaluate a prospective Internet voting system and assess its usability, functionality, security, auditability, and voter privacy. The election was open to any citizen. Participants were required to register prior to voting just as Albertans would in a regular election since the province does not have a pre-existing register of electors. The registration period ran from October 8, 2012, at 8:00 a.m. to November 1, 2012, at 12:00 p.m. and required prospective voters to complete a registration form and upload proof of identification. Applications to vote were reviewed by a registration officer and this process took about a day to complete. The voting period went from 8:00 a.m. October 22 until 12:00 p.m. November 2, 2012, and, while over 900 people registered, a total of 497 voted (Cavanagh, January 8, 2013; City of Edmonton 2012).

Although the Jellybean Election served as part of the public consultation process, it also evaluated the shape of a potential future alternative voting system. To more fully evaluate the security of the technology, the city hired an independent company to launch a denial of service attack, penetration and vulnerability testing and review of business processes on the vendor's technology. A security report was issued documenting that the attempt(s) had been unsuccessful (Cavanagh, January 8, 2013; Crutchlow, November 29, 2012).

One of the final steps of the consultation process was the Citizens' Jury meeting itself. The Jury was composed of seventeen randomly selected eligible electors, designed to demographically and attitudinally reflect the Edmonton public.Footnote 5 Jurors convened at the University of Alberta from 2:30 p.m. on November 23, 2012, to 5:00 p.m. on November 25, 2012, to hear evidence-based testimony from selected expert witnesses, engage in deliberative exercises, and determine whether Internet voting should be used as an alternative voting method in future municipal and school board elections. "Witnesses" giving presentations and answering questions from the Jury included academics, practitioners, and municipal and provincial officials, who addressed issues such as security, authentication, auditability, voter privacy, proprietary and open-source code, the history of Canadian elections, and the experiences of other jurisdictions that have used Internet voting. The Jury reached a consensus verdict, recommending that the city proceed with Internet voting in the forthcoming elections. Though one juror was opposed to the idea of Internet voting he expressed that he could "live with it" to deliver a consensus verdict. The verdict, along with recommendations decided by the Jury, Footnote 6 was put into a report that was passed on to the city (CPI 2012; City of Edmonton 2012).

In addition to the Citizens' Jury, three citizen roundtable advisory meetings were also organized. Two were held at, and organized in collaboration with, senior centres and a third was held at the University of Alberta for the general public. Altogether 60 people attended the roundtables, although only two residents came to the general public meeting. Participants were recruited through the senior homes, or because they had seen information on the city website, CPI website, or Twitter. Roundtable participants completed the pre-survey for stakeholders and were given the Issues Guide and information about the city's plan. Afterward, they also completed a stakeholder post-survey addressing Internet voting. All issues and concerns raised by citizens were recorded and included in a summary report that was compiled by CPI; however, summaries and not the entire report were provided to elected members for review. City staff prepared a report to council, which included summaries of the reports prepared by CPI.

Edmonton City Council was scheduled to meet on January 23, 2013, to determine whether to proceed with Internet voting in 2013 in light of the Citizens' Jury results, but postponed this decision until February 6, 2013, given that a member of the public had made a request to speak to council (Cavanagh January 8, 2013; Kamenova, September 9, 2012). On January 28, 2013, an Executive Committee composed of six councillors heard from Edmonton computer programmer, Chris Cates, and two Citizens' Jury participants. While the two former jurors made presentations supporting the decision to adopt Internet voting, Mr. Cates, a public opponent of electronic voting, made the argument that the online voting method is unsafe and poses a potential threat to local democracy. He alleged that he was able to vote twice in the mock election, but did not explain how he accomplished this (Kamenova, May 22, 2013).

Privacy and security of the vote had been the primary goals of the election test, but city administration had wanted people from all over the world to test the Internet voting system so they did not put onerous restrictions on registration. As a consequence, Mr. Cates was able to register to vote twice and therefore cast two ballots. While not directly having bearing on the security of the voting system, this testimony did raise concerns among the councillors. In particular, registration caused some worry given that there is no voters' list for the local and school board elections of Alberta municipalities.

Additional questions about the technology were raised and city administration provided answers to the best of their ability, but no experts were present to contribute. Councillors were provided with the Jury verdict and recommendations, but not the full report that CPI had prepared about the Jury process. This may have contributed to negative orientations surrounding Internet voting and some misunderstandings among council (Kamenova and Goodman 2013). In the end, Edmonton City Council voted 11–2 against introducing an Internet voting option in 2013.

This decision was somewhat controversial given the resources that had gone into facilitating such a robust public participation campaign, which had in general supported Internet voting's introduction. This decision to not support the public wisdom is also a reminder that democratically elected officials have the final say on policy matters. Despite the ending, however, the approach developed in partnership between the city and CPI is a consultation and engagement model possessing aspects worthy of consideration by other jurisdictions seeking public input regarding Internet voting or other policy matters.

Province of British Columbia

The Province of British Columbia is actively researching and assessing the possibility of offering online ballots in future elections. Internet voting became an issue in BC as municipalities developed an interest to engage in it. The impetus began in 2008 when Nanaimo City Council requested permission from the province to use Internet voting. It grew more intense, however, in May 2011, when Vancouver City Council passed a resolution to allow the use of online voting in its November 2011 elections, an action that also required provincial approval prior to implementation. However, the province did not grant the approval to proceed. News media reported that the province lacked confidence that Vancouver, in particular, had a sufficient amount of time to prepare and develop a "rock solid" plan (CBC News 2011).

Around the same time, like she had during her leadership campaign for the British Columbia Liberal Party, Premier Christy Clark made some public comments supporting online voting in principle and encouraging an examination of its potential use in future BC elections. Partly as a result of these comments, the provincial elections agency, Elections BC (EBC), published a discussion paper on Internet voting (Atcheson, December 10, 2012; Hillsdon 2011). Also, a November 2011 report of recommendations for legislative change by EBC Chief Electoral Officer, Keith Archer, included a recommendation to allow the trial of new voting technologies. These factors supported pursuing research into online voting in BC. Premier Clark made a commitment to request that EBC put together a non-partisan expert panel to review best practices associated with Internet voting in other jurisdictions and possible issues associated with the implementation of online ballots in BC. Attorney General Shirley Bond formally made this request to Archer, in August 2012, also asking if such a panel, chaired by Archer, would assess the use of Internet voting at provincial and municipal levels of government given recent local interest (Bond, August 7, 2012).

The appointment of a panel and the assessment of Internet voting the province is undertaking is a form of consultation with elements of evaluation. An independent group has been convened to review and make recommendations regarding the possible use of Internet voting for provincial and local government elections in BC. The consultation role of the panel involves hearing from and meeting with people who have expertise relating to Internet voting, such as academics and practitioners, including Internet voting vendors. The panel will also release an interim report inviting public commentary. In this way, the panel hopes to engage the public in a type of consultation that is more informed than a general request for input (Atcheson, December 20, 2012).

The evaluation component of the upcoming report rests on the fact that members of the panel are required to make an assessment of whether Internet voting is a suitable alternative method of voting in BC elections at either the provincial or local level of government. In doing so, the panel will bring its own judgments on the issues to bear, rather than relying solely on consultations with others.

The panel was officially created in September 2012 and has held five official meetings; it took a three-month break while EBC prepared for and carried out a general election. A total of five members were appointed, including Archer, who is also the panel chair. The five-member size was selected because it was seen as a small enough group to meet regularly, but a large enough group to allow for a variety of expertise to be represented. Participants were selected based on their expertise and experience, with careful consideration given to ensure appropriate gender and geographical distribution. All panellists live and work in BC (Archer, March 8, 2013). Additionally, it was a requirement that panel members be non-partisan and not have publicly spoken out regarding Internet voting or publicly stated their opinion as a proponent or critic. Given that EBC is a non-partisan agency of the provincial legislature, it was expected the panel would also embody the principle of independence (Atcheson, December 10, 2012). In addition to the chair, the members are Dr. Konstantin Beznosov, Associate Professor of Electrical and Computer Engineering at the University of British Columbia; Dr. Valerie King, a Professor in the Department of Computer Science at the University of Victoria; Lee-Ann Crane, Chief Administrative Officer, East Kootenay Regional District; and the former Auditor General of British Columbia, George Morfitt. Through this membership the panel represents cryptography, technical expertise, the local government, and an independent former legislative officer (Atcheson, December 10, 2012).

The panel will continue to convene on a monthly or bi-monthly basis until members are in a suitable position to fulfill their mandate and make recommendations regarding Internet voting in BC. While there is not a rigid time frame for the project, to ensure that the process is not rushed, the panel was working toward completing an interim report by the summer of 2013 and a final report in the fall of 2013. Panel meetings are closed to the public, but notes are being taken and published on the panel's website. Along with this, other information about the panel, such as participant biographies, official government letters that established the panel, and other relevant details are publicly available on the EBC website. The work plan is focused on monthly meetings wherein the panel evaluates the opportunities and challenges of Internet voting based on available literature and consultation with experts and practitioners. The panel's final recommendations on whether it would be appropriate to introduce Internet voting in provincial and local government elections in BC will be presented to the provincial legislative assembly sometime after the final report is tabled (Atcheson, December 10, 2012).

The process being used in BC takes a different approach from previous research, deliberation, and decisions regarding whether to use an online voting system. Other approaches have predominantly centred on decision making from government officials and have not adopted such a comprehensive consultation and assessment phase. While the BC example is not as consultative of the public as in the case of Edmonton, the phase of gathering of information phase, through literature and input from experts and practitioners is much more robust than previous consultation models used in Canada. Assembling an independent group to consider Internet voting in the context of BC's political and social climate presents a useful precedent for other jurisdictions seeking a decision-making strategy based on its incorporation of field experts' opinions together with a public contribution.

Footnote 4 Staff could not help with the actual voting beyond helping voters complete the online requirement of DOB because they were not officially sworn in as deputy returning officers.

Footnote 5 Eighteen participants were recruited, but one selected individual cancelled at the last minute so only seventeen residents actually participated (Kamenova, January 14, 2013).

Footnote 6 The Jury made some recommendations, including "simplified registration process; mobile friendly platform; fourteen consecutive days of Internet voting; proprietary software as a short term solution and open source software in a long run; telephone voting by 2017; services in different languages; increased security; and strong evaluation and research component" (Kamenova, November 29, 2012).