Secondary menu

Survey of Electors – Whitby–Oshawa / Yellowhead By-elections

Appendix A: Detailed Methodology

On behalf of Elections Canada, Nielsen conducted a random telephone survey with 1,503 eligible electors in the electoral districts of Whitby–Oshawa in Ontario and Yellowhead in Alberta after the November 17, 2014 by-elections. The objective of the study was to evaluate the electors' opinions, attitudes and knowledge of the agency's services and their experience with the electoral process.

Only eligible voters, Canadian citizens 18 years of age or older on the day of polling and who had been residing in one of the ridings between October 15 and November 17, 2014, were invited to participate. Below is a detailed methodology of this project.

Sampling Strategy

Elections Canada was looking for a survey that was representative of eligible voters in both ridings. A stratified random sampling technique based on age and gender was applied and resulted in a similar composition on both locations. The goal was to survey a minimum of 750 eligible electors from the general populations on each electoral riding.

Table A: Sample frame for each subject riding
Quota group Desired # of completes
Yellowhead Whitby–Oshawa
Male 18 - 34 75 75
Male 35 - 54 150 150
Male 55 and older 150 150
Female 18 - 34 75 75
Female 35 - 54 150 150
Female 55 and older 150 150
Total # of completes n=750 n=750

The final result was similar to the actual population distribution in both ridings and only minor adjusts were made via weighting. Below is the actual respondents' distribution.

Table B: Respondents distribution
Quota group # of completes
Yellowhead Whitby–Oshawa
Male 18 - 34 107 102
Male 35 - 54 153 154
Male 55 and older 126 106
Female 18 - 34 101 103
Female 35 - 54 145 165
Female 55 and older 118 124
Total # of completes n=750 n=753

Questionnaire Design

Elections Canada provided a draft questionnaire which was used in previous waves of research and worked together with Nielsen team to create a final questionnaire that would maximize the project objectives. Although some changes were made, most questions were maintained in order to allow for comparisons across elections and by-elections.

The final questionnaire was composed of 45 questions and 13 demographic questions and took an average of approximately 12 minutes to complete.

Pre-Test

Nielsen conducted a pre-test using random digit dialing (RDD) to call residents of Whitby–Oshawa and Yellowhead ridings on the evening of December 17 from 5:30 pm onwards. The objective of the pre-test was to observe the respondents' experience during the survey, look for any inconsistencies or wording issues that may confuse the respondent and assess overall functionality of the survey instrument.

A total of 20 interviews were observed throughout the evening. A pre-test report was presented to the client group one day later and once EC reviewed the results, only minor changes were made to the questionnaire and therefore, results from the pretest were used in the final data.

Survey Launch and Data Collection

Interviews were administered by telephone between December 17 and 30, 2014, using random digit dialling technology and Confirmit Horizons software. Potential participants were asked qualification questions before being invited to participate in the survey. Respondent qualification was determined using a series of questions about citizenship status, age and residency of the respondent.

Interviews lasted an average of 12.04 minutes. A total of 1,503 valid survey completions were obtained. The margin of error associated with the combined sample size for this survey is ±2.5% while for the two individual ridings is ±3.6% 19 times out of 20 (95% confidence).

Weighting Scheme

At the completion of data collection, the data was weighted to reflect the information from the 2011 census. We used a standard procedure for calculating weighting factors, based on established methodological standards. This procedure involved calculating the actual population within each segment (in this case age and gender within region), and the true proportion of the sample that would fall into each segment if the survey were conducted on strictly a random basis (based on the most current population statistics available). Into this number was divided the actual segment subsample to produce a weighting factor that was then used to "weight" the data for that segment.

In both ridings there was a surplus of respondents aged 55 and older and an underrepresentation of respondents aged 18 to 34. The table below presents the weights applied according to the stratification plan.

Table C: Weight scheme – Whitby–Oshawa
Whitby–Oshawa # Population % Population # in Survey % in Survey WEIGHT
Males 18-34 15,030 13.56% 67 8.90% 1.524542001
Females 18-34 15,155 13.68% 74 9.83% 1.391808347
Males 35-54 22,615 20.41% 152 20.19% 1.01113285
Females 35-54 24,235 21.87% 152 20.19% 1.08356421
Males 55+ 15,570 14.05% 154 20.45% 0.687104998
Females 55+ 18,195 16.42% 154 20.45% 0.802946399
TOTAL 110,800 100% 753 100% 1


Table D: Weight scheme – Yellowhead
Yellowhead # Population % Population # in Survey % in Survey WEIGHT
Males 18-34 10,740 14.30% 63 8.40% 1.701812097
Females 18-34 10,080 13.42% 74 9.87% 1.359805167
Males 35-54 15,325 20.40% 151 20.13% 1.013144545
Females 35-54 14,550 19.37% 153 20.40% 0.949334878
Males 55+ 12,590 16.76% 153 20.40% 0.821451967
Females 55+ 11,845 15.77% 156 20.80% 0.757981038
TOTAL 75,130 100% 750 100% 1

Impact of weighting

The table below outlines the unweighted and weighted counts by age and gender in both ridings. It can be noted that the biggest adjustments were required to males in 18-34 age group in both ridings, which is a common observation among random probability surveys conducted in Canada via landline interviewing. The weighting scheme accounts for this shortfall and adjusts the final sample so that it represents the general population distribution for both regions.

Table E: Impact of weight on sample composition
Riding Sample group Sample composition
Unweighted Weighted
Whitby–Oshawa Male 18-34 67 102
Male 35-54 152 154
Male 55+ 154 106
Female 18-34 74 103
Female 35-54 152 165
Female 55+ 154 124
Yellowhead Male 18-34 63 107
Male 35-54 151 153
Male 55+ 153 126
Female 18-34 74 101
Female 35-54 153 145
Female 55+ 156 118
TOTAL   1,500 1,500