open Secondary menu

Appendix A: MethodologyPublic Opinion Research Study on Electoral Matters - Wave 1

Printable PDF version

Quantitative Methodology

Quantitative research was conducted using online surveys and computer-assisted web interviewing technology.

As a member of the Canadian Research Insights Council, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards for the Conduct of Government of Canada Public Opinion Research—Series D—Quantitative Research.

Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix B.

Sampling Procedure

Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,582 respondents participated in the survey. The exact distribution is presented in the following section. Participants were selected randomly from the Leger Opinion online panel.

Léger operates a proprietary Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled based on certain socio-demographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.

Data Collection

Fieldwork for the survey was conducted from April 1, 2021, to April 11, 2021. The participation rate for the survey was 16%. A pretest of 65 interviews was completed on April 1, 2021.

To achieve data reliability in all subgroups, a total sample of 2,582 Canadians who are eligible voters were surveyed, in all regions of the country.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error for this survey cannot be calculated. Respondents were selected from among those who have volunteered to participate or registered to participate in online surveys. The results of such surveys cannot be described as being statistically projectable to the target population. The data were weighted to reflect the demographic composition of the target population. Because the sample was based on those who initially self-selected for participation, no estimates of sampling error could be calculated.

Based on data from Statistics Canada's 2016 national census, Léger weighted the results of this survey by age and gender in each region of the country.

The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.

Table A1: Regional Distribution of Respondents
Region Number of respondents
Atlantic 362
Quebec 412
Ontario 721
Prairies 362
Alberta 344
British Columbia 336
Territories 45
Total 2582

Participation Rate

Table A2 breaks down the calculation of the web survey's participation rate. The overall response rate for this study was 16%. The participation rate is calculated using the following formula: participation rate / response rate = R (U + IS + R).

Table A2: Participation Rate Calculation
Invalid cases 153
Invitations mistakenly sent to people who did not qualify for the study 153
Incomplete or missing email addresses 0
Unresolved (U) 14,191
Email invitations bounce back 128
Email invitations unanswered 14,063
In-scope non-responding units (IS) 114
Non-response from eligible respondents 0
Respondent refusals 0
Language problem 0
Selected respondent not available (illness; leave of absence; vacation; other) 0
Early breakoffs 114
Responding units (R) 2,626
Surveys disqualified – quota filled 44
Completed surveys disqualified for other reasons 0
Completed interviews 2,582
Potentially eligible (U + IS + R) 16,931
Participation rate 16%

Typical participation rates for web surveys are between 20% and 30%. A response rate of 16% may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely among panel members to achieve our objectives; this had an impact on the participation rate.

Unweighted and Weighted Samples

A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

Table A3 presents the geographic distribution of respondents before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions, such as the Atlantic region and the Prairies, to obtain a sufficient sample. Therefore, the weighting minimized the weight of those regions and slightly increased the weight of Quebec and Ontario.

Table A3: Unweighted and Weighted Sample Distribution by Province
Region Unweighted Weighted
Atlantic 362 176
Quebec 412 604
Ontario 721 989
Prairies 362 168
Alberta 344 289
British Columbia 336 349
Territories 45 8
Total 2,582 2,583

Tables A4 and A5 present the demographic distribution of respondents according to gender and age.

First, regarding gender, we can see that weighting slightly adjusted the proportion of men and women. These adjustments are minor, and there is no reason to believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.

Table A4: Unweighted and Weighted Sample Distribution by Gender
Gender Unweighted Weighted
Men 1,257 1,242
Women 1,314 1,328
Total 2,571 2,570

Regarding age, the weighting process corrected some minor discrepancies. The distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduced a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.

Table A5: Unweighted and Weighted Sample Distribution by Age Group
Age Unweighted Weighted
Between 18 and 34 657 704
Between 35 and 54 953 880
55 and over 972 998
Total 2,582 2,582

There is no evidence from the data that having achieved a different age or gender distribution before weighting would have significantly changed the results for this study. The relatively small weighting factors and differences in responses among the various subgroups suggest that data quality was not affected. The weighting that was applied corrected the initial imbalance for data analysis purposes, and no further manipulations were necessary.

Tables A6 and A7 present the weighting factors applied to the database according to the different respondent profiles.

Table A6: Weighting Factors by Profile
Label Weighting
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 18-240.0072
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 25-340.0109
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 35-440.0102
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 45-540.0117
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 55-640.0117
British Columbia, Yukon and Male, Or please specify., Prefer not to say and 65+0.0141
British Columbia, Yukon and Female and 18-240.0068
British Columbia, Yukon and Female and 25-340.0111
British Columbia, Yukon and Female and 35-440.0108
British Columbia, Yukon and Female and 45-540.0125
British Columbia, Yukon and Female and 55-640.0125
British Columbia, Yukon and Female and 65+0.0162
Alberta and Male, Or please specify., Prefer not to say and 18-240.0065
Alberta and Male, Or please specify., Prefer not to say and 25-340.0116
Alberta and Male, Or please specify., Prefer not to say and 35-440.0106
Alberta and Male, Or please specify., Prefer not to say and 45-540.0099
Alberta and Male, Or please specify., Prefer not to say and 55-640.009
Alberta and Male, Or please specify., Prefer not to say and 65+0.0083
Alberta and Female and 18-240.0062
Alberta and Female and 25-340.0114
Alberta and Female and 35-440.0104
Alberta and Female and 45-540.0098
Alberta and Female and 55-640.0089
Alberta and Female and 65+0.0096
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 18-240.004
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 25-340.0058
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 35-440.0053
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 45-540.0055
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 55-640.0055
Manitoba, Saskatchewan, Northwest Territories and Male, Or please specify., Prefer not to say and 65+0.0059
Manitoba, Saskatchewan, Northwest Territories and Female and 18-240.0038
Manitoba, Saskatchewan, Northwest Territories and Female and 25-340.0058
Manitoba, Saskatchewan, Northwest Territories and Female and 35-440.0053
Manitoba, Saskatchewan, Northwest Territories and Female and 45-540.0055
Manitoba, Saskatchewan, Northwest Territories and Female and 55-640.0056
Manitoba, Saskatchewan, Northwest Territories and Female and 65+0.0072
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 18-240.0223
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 25-340.0306
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 35-440.0292
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 45-540.0346
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 55-640.0316
Ontario, Nunavut and Male, Or please specify., Prefer not to say and 65+0.0363
Ontario, Nunavut and Female and 18-240.0212
Ontario, Nunavut and Female and 25-340.0315
Ontario, Nunavut and Female and 35-440.0319
Ontario, Nunavut and Female and 45-540.0365
Ontario, Nunavut and Female and 55-640.0338
Ontario, Nunavut and Female and 65+0.0441
Quebec and Male, Or please specify., Prefer not to say and 18-240.0121
Quebec and Male, Or please specify., Prefer not to say and 25-340.018
Quebec and Male, Or please specify., Prefer not to say and 35-440.0189
Quebec and Male, Or please specify., Prefer not to say and 45-540.0203
Quebec and Male, Or please specify., Prefer not to say and 55-640.0211
Quebec and Male, Or please specify., Prefer not to say and 65+0.0239
Quebec and Female and 18-240.0118
Quebec and Female and 25-340.0181
Quebec and Female and 35-440.0189
Quebec and Female and 45-540.0203
Quebec and Female and 55-640.0217
Quebec and Female and 65+0.0295
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 18-240.0034
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 25-340.0046
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 35-440.0049
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 45-540.0061
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 55-640.0065
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Male, Or please specify., Prefer not to say and 65+0.0076
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 18-240.0033
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 25-340.0047
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 35-440.0053
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 45-540.0065
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 55-640.0069
New Brunswick, Newfoundland, Prince Edward Island, Nova Scotia and Female and 65+0.0089
Table A7: Weighting Factors by Province and Territory
Label Weighting
Newfoundland0.015257
Prince Edward Island0.004092
Nova Scotia0.027016
New Brunswick0.021775
Quebec0.234007
Ontario0.38285
Manitoba0.035029
Saskatchewan0.029936
Alberta0.111797
British Columbia0.135321
Yukon0.001011
Northwest Territories0.001112
Nunavut0.000797