Appendix 1: Methodology – Public Opinion Research Study on Electoral Matters - Wave 2
Quantitative Methodology
Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.
As a CRIC Member, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research—Series D—Quantitative Research.
Respondents were assured of the voluntary, confidential, and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.
The questionnaire is available in Appendix A2.
Sampling Procedure
Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,504 respondents participated in the survey. The exact distribution is presented in the following section. Participant selection was done randomly from Leo's online panel.
Léger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled on different sociodemographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone during the past decade, making the panel's composition very similar to the actual Canadian population on many demographic characteristics.
Data Collection
Fieldwork for the survey was conducted from April 25 to May 4, 2022. The participation rate for the survey was 15 percent. A pre-test of 49 interviews was completed on April 26, 2022.
To achieve data reliability in all subgroups, a total sample of 2,504 Canadians who are eligible voters were surveyed, in all regions of the country.
Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents were selected from among those who have volunteered or registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.
Based on data from Statistics Canada's 2016 national census, Léger weighted the results of this survey by age and gender within each region of the country.
The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.
Region | Number of respondents |
---|---|
Atlantic | 350 |
Quebec | 401 |
Ontario | 701 |
Prairies | 349 |
Alberta | 343 |
British Columbia | 327 |
Northern Territories | 33 |
Total | 2,504 |
Participation Rate
The overall participation rate for this study is 15 percent.
Below is the calculation of the web survey's participation rate. The overall response rate for this study is 16 percent. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.
Invalid cases | 74 |
---|---|
Invitations mistakenly sent to people who did not qualify for the study | 74 |
Incomplete or missing email addresses | 0 |
Unresolved (U) | 17,935 |
Email invitations bounce back | 11 |
Email invitations unanswered | 17,924 |
In-scope non-responding units (IS) | 163 |
Non-response from eligible respondents | 0 |
Respondent refusals | 51 |
Language problem | 0 |
Selected respondent not available (illness; leave of absence; vacation; other) | 0 |
Early breakoffs | 112 |
Responding units (R) | 3,111 |
Surveys disqualified – quota filled | 943 |
Completed surveys disqualified for other reasons | 0 |
Completed interviews | 2,168 |
Potentially eligible (U + IS + R) | 21,209 |
Participation rate | 14.67% |
Typical participation rates for web surveys are between 20 and 30 percent. A response rate of 15 percent may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely through the panel to achieve our objectives, which had an impact on the participation rate. The participation rate is similar to that of the first wave of the study that took place in 2021.
Unweighted and Weighted Samples
A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).
The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions, including the Atlantic region and the Prairies, in order to have a sufficient sample in these regions. Therefore, the weighting minimized the weight of these regions that had been voluntarily inflated and slightly increased the weight of Quebec and Ontario.
Region | Unweighted | Weighted |
---|---|---|
Atlantic | 350 | 168 |
Quebec | 401 | 576 |
Ontario | 701 | 967 |
Prairies | 349 | 161 |
Alberta | 343 | 278 |
British Columbia | 327 | 348 |
Northern Territories | 33 | 7 |
Total | 2,504 | 2,504 |
The following tables present the demographic distribution of respondents according to gender and age.
First, regarding gender, we can see that weighting has adjusted slightly the proportions of men and women. The adjustments made by weighting are minor, and in no way do we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.
Gender | Unweighted | Weighted |
---|---|---|
Men | 1,250 | 1,220 |
Women | 1,243 | 1,275 |
Total | 2,504 | 2,504 |
Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduced a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.
Age | Unweighted | Weighted |
---|---|---|
Between 18 and 34 | 583 | 668 |
Between 35 and 54 | 963 | 806 |
55 and over | 958 | 1,030 |
Total | 2,504 | 2,504 |
There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results of this study. The relatively small weight factors (see section below) and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.
The following tables present the weighting factors applied to the database according to the different respondent profiles.
Label | Weighting |
---|---|
British Columbia + Territories and Male and 18-24 | 0.7036 |
British Columbia + Territories and Male and 25-34 | 1.2050 |
British Columbia + Territories and Male and 35-44 | 1.1420 |
British Columbia + Territories and Male and 45-54 | 1.0586 |
British Columbia + Territories and Male and 55-64 | 1.1747 |
British Columbia + Territories and Male and 65+ | 1.6115 |
British Columbia + Territories and Female and 18-24 | 0.6627 |
British Columbia + Territories and Female and 25-34 | 1.1949 |
British Columbia + Territories and Female and 35-44 | 1.1755 |
British Columbia + Territories and Female and 45-54 | 1.1361 |
British Columbia + Territories and Female and 55-64 | 1.2613 |
British Columbia + Territories and Female and 65+ | 1.8419 |
Alberta and Male and 18-24 | 0.6049 |
Alberta and Male and 25-34 | 1.0033 |
Alberta and Male and 35-44 | 1.0819 |
Alberta and Male and 45-54 | 0.9115 |
Alberta and Male and 55-64 | 0.8983 |
Alberta and Male and 65+ | 0.9914 |
Alberta and Female and 18-24 | 0.5662 |
Alberta and Female and 25-34 | 1.0078 |
Alberta and Female and 35-44 | 1.0900 |
Alberta and Female and 45-54 | 0.9042 |
Alberta and Female and 55-64 | 0.9135 |
Alberta and Female and 65+ | 1.1221 |
Manitoba, Saskatchewan and Male and 25-34 | 0.3823 |
Manitoba, Saskatchewan and Male and 18-24 | 0.5589 |
Manitoba, Saskatchewan and Male and 35-44 | 0.5456 |
Manitoba, Saskatchewan and Male and 45-54 | 0.4782 |
Manitoba, Saskatchewan and Male and 55-64 | 0.5303 |
Manitoba, Saskatchewan and Male and 65+ | 0.6584 |
Manitoba, Saskatchewan and Female and 18-24 | 0.3501 |
Manitoba, Saskatchewan and Female and 25-34 | 0.5518 |
Manitoba, Saskatchewan and Female and 35-44 | 0.5532 |
Manitoba, Saskatchewan and Female and 45-54 | 0.4834 |
Manitoba, Saskatchewan and Female and 55-64 | 0.5437 |
Manitoba, Saskatchewan and Female and 65+ | 0.7759 |
Ontarioand Male and 18-24 | 2.1192 |
Ontarioand Male and 25-34 | 3.3074 |
Ontarioand Male and 35-44 | 2.9929 |
Ontarioand Male and 45-54 | 2.9760 |
Ontarioand Male and 55-64 | 3.2772 |
Ontarioand Male and 65+ | 4.0270 |
Ontarioand Female and 18-24 | 1.9659 |
Ontarioand Female and 25-34 | 3.2647 |
Ontarioand Female and 35-44 | 3.1814 |
Ontarioand Female and 45-54 | 3.1905 |
Ontarioand Female and 55-64 | 3.4634 |
Ontarioand Female and 65+ | 4.8329 |
Quebec and Male and 18-24 | 1.0850 |
Quebec and Male and 25-34 | 1.7944 |
Quebec and Male and 35-44 | 1.8835 |
Quebec and Male and 45-54 | 1.7539 |
Quebec and Male and 55-64 | 2.0655 |
Quebec and Male and 65+ | 2.6936 |
Quebec and Female and 18-24 | 1.0393 |
Quebec and Female and 25-34 | 1.7768 |
Quebec and Female and 35-44 | 1.8885 |
Quebec and Female and 45-54 | 1.7373 |
Quebec and Female and 55-64 | 2.1039 |
Quebec and Female and 65+ | 3.1964 |
Atlantic regionand Male and 18-24 | 0.3232 |
Atlantic regionand Male and 25-34 | 0.4643 |
Atlantic regionand Male and 35-44 | 0.4638 |
Atlantic regionand Male and 45-54 | 0.5166 |
Atlantic regionand Male and 55-64 | 0.6304 |
Atlantic regionand Male and 65+ | 0.8496 |
Atlantic regionand Female and 18-24 | 0.2995 |
Atlantic regionand Female and 25-34 | 0.4650 |
Atlantic regionand Female and 35-44 | 0.4964 |
Atlantic regionand Female and 45-54 | 0.5484 |
Atlantic regionand Female and 55-64 | 0.6678 |
Atlantic regionand Female and 65+ | 0.9837 |
Label | Weight |
---|---|
British Columbia | 13.8783 |
Alberta | 11.0950 |
Saskatchewan | 2.9165 |
Manitoba | 3.4955 |
Ontario | 38.5984 |
Quebec | 23.0182 |
New Brunswick | 2.1532 |
Nova Scotia | 2.6997 |
Prince Edward Island | 0.4218 |
Newfoundland | 1.4340 |
Northwest Territories | 0.1044 |
Yukon | 0.1080 |
Nunavut | 0.0770 |