Site MenuYouth Engagement and Mobilization in the 2010 Toronto Municipal Election
Appendix A – Quasi-experimental Analysis
Apathy is Boring's original intention, as described in the Research Design and Methodology section of this report, was to conduct a quasi-experimental analysis using data collected from both the survey and Toronto Elections' partner organizations.
In a true experiment, researchers assign subjects to treatment and control groups, typically through random assignment. In a quasi-experiment, researchers are observing an existing phenomenon. Groups are identified as "treatment" or "control" based on their exposure to the phenomenon in question. In both cases, the analysis is then conducted by comparing results from the treatment and control groups.
The initial survey questionnaire collected postal codes from respondents. Our goal was to conduct a quasi-experimental analysis by combining this with the mobilization data, using respondents' proximity to mobilization activities as a proxy for exposure. However, as explained below, this form of analysis was not possible for youth mobilization in the 2010 municipal election. The report therefore presents a simpler analysis of the data.
Limited Scale of Mobilization Activity
Due to the scale of youth mobilization during the election, it is impossible to develop a treatment scheme for a quasi-experimental analysis. This problem is best explained with a specific example: consider Ward 42 in the city of Toronto. This ward had the largest amount of recorded mobilization activity before the election. In total, approximately 1,952 voting-age residents of the ward – though not exclusively youth – were approached by volunteers or attended election-related events organized by partner organizations.
A back-of-the-envelope calculation demonstrates the analytical challenge. In 2010, there were 44,136 registered voters in Ward 42 (City of Toronto 2010). For argument's sake, let us make two generous assumptions about the mobilization activity being analyzed: that there was no duplication of contact, and that everyone person contacted was a registered voter.
Given these assumptions, 4.4% of registered voters in Ward 42 either attended an event or encountered a volunteer. The survey sample includes 27 respondents from the ward, 18 of whom completed the follow-up questionnaire. This creates the challenge of using a sample with fewer than 20 respondents to assess the impact of a campaign that contacted less than one twentieth of the population from which they are drawn.29
This issue occurs despite the tremendous amount of activity in Ward 42, where almost a third of all recorded contact in our dataset took place. Whatever the effectiveness of youth mobilization in the 2010 Toronto municipal election, its overall scale is too small for a quasi-experimental analysis.
29 Changing the unit of analysis does not resolve this issue. Some neighbourhoods have higher contact rates than the wards in which they are located, but they also have fewer survey respondents.