open Secondary menu

Appendix 1: MethodologyPublic Opinion Research Study on Electoral Matters - Wave 2

Printable PDF version

Quantitative Methodology

Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.

As a CRIC Member, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research—Series D—Quantitative Research.

Respondents were assured of the voluntary, confidential, and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix A2.

Sampling Procedure

Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,504 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo's online panel.

Léger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled on different sociodemographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone during the past decade, making the panel's composition very similar to the actual Canadian population on many demographic characteristics.

Data Collection

Fieldwork for the survey was conducted from April 25 to May 4, 2022. The participation rate for the survey was 15 percent. A pre-test of 49 interviews was completed on April 26, 2022.

To achieve data reliability in all subgroups, a total sample of 2,504 Canadians who are eligible voters were surveyed, in all regions of the country.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents were selected from among those who have volunteered or registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.

Based on data from Statistics Canada's 2016 national census, Léger weighted the results of this survey by age and gender within each region of the country.

The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.

Table A1: Regional Distribution of Respondents
Region Number of respondents
Atlantic 350
Quebec 401
Ontario 701
Prairies 349
Alberta 343
British Columbia 327
Northern Territories 33
Total 2,504

Participation Rate

The overall participation rate for this study is 15 percent.

Below is the calculation of the web survey's participation rate. The overall response rate for this study is 16 percent. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.

Table A2: Participation Rate Calculation
Invalid cases 74
Invitations mistakenly sent to people who did not qualify for the study 74
Incomplete or missing email addresses 0
Unresolved (U) 17,935
Email invitations bounce back 11
Email invitations unanswered 17,924
In-scope non-responding units (IS) 163
Non-response from eligible respondents 0
Respondent refusals 51
Language problem 0
Selected respondent not available (illness; leave of absence; vacation; other) 0
Early breakoffs 112
Responding units (R) 3,111
Surveys disqualified – quota filled 943
Completed surveys disqualified for other reasons 0
Completed interviews 2,168
Potentially eligible (U + IS + R) 21,209
Participation rate 14.67%

Typical participation rates for web surveys are between 20 and 30 percent. A response rate of 15 percent may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely through the panel to achieve our objectives, which had an impact on the participation rate. The participation rate is similar to that of the first wave of the study that took place in 2021.

Unweighted and Weighted Samples

A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions, including the Atlantic region and the Prairies, in order to have a sufficient sample in these regions. Therefore, the weighting minimized the weight of these regions that had been voluntarily inflated and slightly increased the weight of Quebec and Ontario.

Table A3: Unweighted and Weighted Sample Distribution by Province
Region Unweighted Weighted
Atlantic 350 168
Quebec 401 576
Ontario 701 967
Prairies 349 161
Alberta 343 278
British Columbia 327 348
Northern Territories 33 7
Total 2,504 2,504

The following tables present the demographic distribution of respondents according to gender and age.

First, regarding gender, we can see that weighting has adjusted slightly the proportions of men and women. The adjustments made by weighting are minor, and in no way do we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.

Table A4: Unweighted and Weighted Sample Distribution by Gender
Gender Unweighted Weighted
Men 1,250 1,220
Women 1,243 1,275
Total 2,504 2,504

Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduced a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.

Table A5: Unweighted and Weighted Sample Distribution by Age Group
Age Unweighted Weighted
Between 18 and 34 583 668
Between 35 and 54 963 806
55 and over 958 1,030
Total 2,504 2,504

There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results of this study. The relatively small weight factors (see section below) and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.

The following tables present the weighting factors applied to the database according to the different respondent profiles.

Table A6: Weighting Factors by Profile
Label Weighting
British Columbia + Territories and Male and 18-240.7036
British Columbia + Territories and Male and 25-341.2050
British Columbia + Territories and Male and 35-441.1420
British Columbia + Territories and Male and 45-541.0586
British Columbia + Territories and Male and 55-641.1747
British Columbia + Territories and Male and 65+1.6115
British Columbia + Territories and Female and 18-240.6627
British Columbia + Territories and Female and 25-341.1949
British Columbia + Territories and Female and 35-441.1755
British Columbia + Territories and Female and 45-541.1361
British Columbia + Territories and Female and 55-641.2613
British Columbia + Territories and Female and 65+1.8419
Alberta and Male and 18-240.6049
Alberta and Male and 25-341.0033
Alberta and Male and 35-441.0819
Alberta and Male and 45-540.9115
Alberta and Male and 55-640.8983
Alberta and Male and 65+0.9914
Alberta and Female and 18-240.5662
Alberta and Female and 25-341.0078
Alberta and Female and 35-441.0900
Alberta and Female and 45-540.9042
Alberta and Female and 55-640.9135
Alberta and Female and 65+1.1221
Manitoba, Saskatchewan and Male and 25-340.3823
Manitoba, Saskatchewan and Male and 18-240.5589
Manitoba, Saskatchewan and Male and 35-440.5456
Manitoba, Saskatchewan and Male and 45-540.4782
Manitoba, Saskatchewan and Male and 55-640.5303
Manitoba, Saskatchewan and Male and 65+0.6584
Manitoba, Saskatchewan and Female and 18-240.3501
Manitoba, Saskatchewan and Female and 25-340.5518
Manitoba, Saskatchewan and Female and 35-440.5532
Manitoba, Saskatchewan and Female and 45-540.4834
Manitoba, Saskatchewan and Female and 55-640.5437
Manitoba, Saskatchewan and Female and 65+0.7759
Ontarioand Male and 18-242.1192
Ontarioand Male and 25-343.3074
Ontarioand Male and 35-442.9929
Ontarioand Male and 45-542.9760
Ontarioand Male and 55-643.2772
Ontarioand Male and 65+4.0270
Ontarioand Female and 18-241.9659
Ontarioand Female and 25-343.2647
Ontarioand Female and 35-443.1814
Ontarioand Female and 45-543.1905
Ontarioand Female and 55-643.4634
Ontarioand Female and 65+4.8329
Quebec and Male and 18-241.0850
Quebec and Male and 25-341.7944
Quebec and Male and 35-441.8835
Quebec and Male and 45-541.7539
Quebec and Male and 55-642.0655
Quebec and Male and 65+2.6936
Quebec and Female and 18-241.0393
Quebec and Female and 25-341.7768
Quebec and Female and 35-441.8885
Quebec and Female and 45-541.7373
Quebec and Female and 55-642.1039
Quebec and Female and 65+3.1964
Atlantic regionand Male and 18-240.3232
Atlantic regionand Male and 25-340.4643
Atlantic regionand Male and 35-440.4638
Atlantic regionand Male and 45-540.5166
Atlantic regionand Male and 55-640.6304
Atlantic regionand Male and 65+0.8496
Atlantic regionand Female and 18-240.2995
Atlantic regionand Female and 25-340.4650
Atlantic regionand Female and 35-440.4964
Atlantic regionand Female and 45-540.5484
Atlantic regionand Female and 55-640.6678
Atlantic regionand Female and 65+0.9837
Table A7: Weighting factors by provinces and territories
Label Weight
British Columbia13.8783
Alberta11.0950
Saskatchewan2.9165
Manitoba3.4955
Ontario38.5984
Quebec23.0182
New Brunswick2.1532
Nova Scotia2.6997
Prince Edward Island0.4218
Newfoundland1.4340
Northwest Territories0.1044
Yukon0.1080
Nunavut0.0770