Site MenuAppendix A: Quantitative methodology – Survey of Official Agents following the 44th General Election
Background and research objectives
Elections Canada (EC), headed by the CEO, an agent of Parliament, is an independent, non-partisan agency that reports directly to Parliament. EC exercises general direction and supervision over the conduct of elections and referendums at the federal level.
EC required the services of a public opinion research supplier to conduct a Survey of Official Agents following the 44th federal General Election. This survey aimed to assess official agents' views and experiences with the 44th General Election (GE) with regards to the financial obligations of the candidate campaign. An official agent is the person responsible for administering the candidate's campaign financial transactions and for reporting those transactions to Elections Canada as required by the Canada Elections Act.
Use of research: The information gained through this public opinion research will assist in the evaluation of EC's administration of the political financing regime. It will also assist in identifying areas where EC's various products and services may be improved.
Methodology
Environics conducted a survey of official agents who submitted an electoral campaign return following the 44th GE, through a self-administered online questionnaire.
Elections Canada provided Environics with contact information (emails) for 1,486 (unduplicated) Official Agents (a few OAs had two different emails; these were kept as it could not be confirmed which they might be monitoring). A total of 548 Official Agents completed the survey, representing 37 percent of the records provided. The online survey was open from February 16 to March 26, 2023.
The participating respondents have the following regional distribution:
| Breakdown/Location | Total | Atlantic | Quebec | Ontario | MB/SK | Alberta | BC/Terr |
|---|---|---|---|---|---|---|---|
| Number of completed interviews | 548 | 54 | 101 | 217 | 42 | 55 | 79 |
| % of completed interviews | 100% | 10% | 18% | 40% | 8% | 10% | 14% |
| Number of OA unduplicated contacts | 1,486 | 130 | 331 | 553 | 110 | 160 | 202 |
| % of total OA contacts | 100% | 9% | 22% | 37% | 7% | 11% | 14% |
This is an attempted census rather than a sample survey so a margin of sampling error does not apply.
Note on timing: Although official agents are required to file campaign returns four months after election day, many campaigns requested extensions to submit the return, and some official agents were still completing their activities at the start of the survey. Nonetheless, those official agents who filed the campaign return within the prescribed timeframes may have experienced an 18-month gap between when they filed and the start of the survey, which could contribute to recall issues. As well, a few activities included in the survey (opening a campaign account, training materials and use of other services available during the campaign) occurred over 20 months prior and thus recall may be an issue for some.
Instrument design
Environics and EC designed a questionnaire that addresses the study objectives, largely based on the questionnaire of the Survey of Official Agents following the 44th General Election. Environics also drafted an advance email from Elections Canada and the invitation email and reminders. The communications included all required information. EC was responsible for translation of all instruments.
The questionnaire median length was 11.4 minutes, and the average was 16.5 minutes. All research instruments are in Appendix B.
Pre-test
EC sent out an advance broadcast email advising OAs of the upcoming data collection. Environics sent out 50 English and 50 French pre-test invitations on February 16, 2023 from which 12 interviews (6 English, 6 French) were completed. The pre-test included standard, government-accepted probing questions on comprehension and appropriateness of language at the end of the survey, and no issues were identified.
Fieldwork
The survey was conducted according to the following steps:
- Environics programmed the questionnaire using state-of-the-art survey software, hosted on a secure server. Environics ensured the data were stored on Canadian servers located and only accessible in Canada, and physically independent from all other databases, directly or indirectly, that are located outside Canada.
- At the client's request, Environics emailed a bilingual letter to 1,947 unduplicated contacts on behalf of EC explaining the survey was imminent and requesting cooperation. Following this broadcast, nine contacts asked to be removed from the list.
- Bilingual invitation emails were then sent to 1,838 official agents; the URL link directed respondents to a bilingual landing page to offer an active survey language choice. The invitations allowed potential respondents to opt out of the survey and unsubscribe from future reminders.
- Two reminder emails were sent to any remaining non-respondents to the initial invitation, to maximize response rate. The invitation and reminder emails also provided an option to opt-out of the survey.
- Environics provided technical support to survey respondents as required. Steps were taken to assure (and also guarantee) complete confidentiality and anonymity of survey responses.
- Environics kept the project authority advised on the status of data collection on a regular basis throughout the field period.
- Environics electronically captured all survey responses as they were submitted, and created an electronic data file to be coded and analysed.
All respondents were offered the opportunity to complete the survey in their official language of choice. All research work was conducted in accordance with the Standards for the Conduct of Government of Canada Public Opinion Research — Online Surveys, as well as applicable federal legislation (Personal Information Protection and Electronic Documents Act, or PIPEDA). Environics registered the survey with the Research Verification Service (RVS) of the Canadian Research Insight Council (CRIC), to allow participants to verify the legitimacy of the survey, register a complaint, get information about the survey industry or ask technical questions about the survey.
Following data collection and prior to analysis, data analysts performed a data-cleaning and validation process, in accordance with the highest industry standards. Open-ended question data were coded and Environics designed banner tables in consultation with the project authority. Data tables were submitted in Excel and CSV format; verbatim comments were made available in an Excel document.
Completion results
This survey consisted of 548 interviews with Official Agents. The response rate for the survey is 37 percent. The disposition of all contacts is presented in the following table:
| N | ||
|---|---|---|
| Total invitations | (c) | 1,486 |
| Total completes | (d) | 548 |
| Qualified break-offs | (e) | 77 |
| Disqualified | (f) | 9 |
| Not responded | (g) | 908 |
| Quota filled | (h) | 0 |
| Contact rate = (d+e+f+h)/c | 42.7 | |
| Response rate = (d+f+h)/c | 37.4 |