Effectiveness of School Based Recruitment Procedures and Modular Data Collections

Purpose: The School Health Action, Planning and Evaluation System (SHAPES) is a school-based data collection and knowledge exchange system designed to improve the health of youth. This paper outlines the design of the SHAPES study, examines the impact of different school recruitment models on participation rates, and examines the impact of using two different research modules during data collection on the prevalence of core behaviours being measured. Methods: In total, 76 schools were recruited from seven health regions and data were collected using the SHAPES Tobacco (TM) and Physical Activity Modules (PAM). Results: It was found that school recruitment rates were higher when both, the researchers and the health unit, worked together to recruit schools. Significant differences were found between students who completed the TM and students who completed the PAM with respect to body mass index, smoking susceptibility, the number of friends who smoke, and the number of active friends. Conclusions: This paper provides valuable real-world insight for future researchers interested in performing population-level, school-based studies of youth risk behaviours. Our experience suggests that a modular approach to data collection is feasible and that recruitment rates are improved when researchers work in collaboration with health unit staff who have existing relationships with schools.


Introduction
Human Services, 1999Services, , 2004)), but they are linked to an increased risk of developing other chronic diseases, including cardiovascular disease and diabetes (U.S.Department of Health & Human Services, 1999, 2004).Considering that tobacco use and physical activity patterns tend to be established during childhood and adolescence (Centers for Disease Control & Prevention, 1997; U.S. Department of Health and Human Services, 1994), it is critical to promote smokefree and physically active lifestyles among the school-aged population.
In 2006, the Ontario Ministry of Health Promotion (MHP) launched provincial strategies in the areas of youth tobacco control and physical activity promotion (as described on http://www.mhp.gov.on.ca).Recognizing that the vast majority of youth spend a significant amount of time at school (Manske, Cameron & Brown, 1997), these strategies included schoolbased prevention actions led by secondary schools and public health units.However, while schools are increasingly pressed to address smoking and physical inactivity through prevention initiatives, much of the evidence available to guide their programming decisions are derived from artificially controlled research that is not aligned with the realities of 'real world' school practice (Boyle, Purciel, Craypo, Stone-Fransisco & Samuels, 2004;Dusenbury & Hansen, 2004;Green, 2006;Ringwalt, Ennett, Vincus, Thorne, Rohrbach & Simons-Rudolph, 2002).The disconnect between the research evidence and the needs of teachers, school administrators and/or school health practitioners, may explain why fewer than 30% of schools implement evidence-based prevention interventions (Ringwalt, et al., 2002).Research has demonstrated that evidence tends not to be utilized unless it is applicable to local circumstance (Green, 2001;Green & Mercer, 2001).Accordingly, program providers in Ontario require locally relevant data to guide their intervention planning and actions associated with the new provincial prevention strategies.
The School Health Action, Planning and Evaluation System (SHAPES) is a school-based, data collection system which can be used to provide school stakeholders with the locally relevant data required to inform prevention planning (Cameron, et al., 2007;Leatherdale, Manske, Wong, & Cameron, 2009) Taking into consideration the mutual needs of school stakeholders and researchers, the SHAPES research platform has been designed to: • engage local health and education systems in planning, tailoring, and evaluating school health initiatives based on evidence; • engage researchers in using aggregated data to assess environmental influences on youth behaviour; and • provide a platform to study the processes and structures required for effective knowledge transfer and exchange in school settings (refer to www.shapes.uwaterloo.ca for additional information).
Research tools have been developed and tested to collect tobacco use and physical activity/inactivity at the student-and school-levels (Wong, Leatherdale & Manske, 2006).Knowledge exchange tools (i.e., customized school-specific feedback reports) have also been developed to translate the student-and school-level data for school stakeholders (Cameron, et al., 2007;Leatherdale, et al., 2009).These feedback reports have been designed to provide schools (administrators, staff and students) and community partners (e.g., public health officers, parents) with school-specific understanding of what is happening at the school (i.e., student behaviours, existing programs and policies), and offers contextually appropriate suggestions for interventions designed to improve the health behaviours of their students.In essence, SHAPES creates an innovative linkage between research and practice by providing school stakeholders with the evidence they need, when they need it, in a form that is useful and understandable for guiding and evaluating school-based prevention programming; valuable insight for the emerging MHP provincial strategies.
School stakeholders often have their own agendas and ways of prioritizing current issues related to youth health.For example, while one school may want to learn more about the frequency of physical activity among their own students to address obesity issues associated with their student population, another school may require more information about the prevalence of smoking among their students in order to advance their tobacco control programming.
The SHAPES system has been specifically designed to address such issues by using a modular approach to data collection (Figure 1).Having different survey modules to address different health behaviours (i.e., SHAPES-Tobacco Module [TM] questionnaire or the SHAPES-Physical Activity Module [PAM] questionnaire) allows the SHAPES tools to be implemented in whatever manner best addresses the needs of school stakeholders.Depending on the needs of stakeholders, students in a school can complete either the TM, the PAM, or both (randomly sampling students in a school where half complete the TM and half complete the PAM).Since the TM also contains the core physical activity measures and the PAM also contains the core smoking measures, this stratified sampling approach would allow schools to have data on the smoking and physical activity behaviour of all their students, while having sufficient data about the correlates of each behaviour from half of the school population for each behaviour.This multi-modular approach can substantially reduce the data collection burden placed on schools.However, there is a need to examine the impact that a modular approach to data collection may have on student responses (i.e., are the prevalence rates for the core behaviours similar across both modules) in a large applied study sample.

Measures within the Tobacco and Physical Activity Modules
Tobacco Module Only: Smoking behaviours of parents and older siblings, participation and awareness of smoking related programs, quitting behaviour, preferred places to smoke, when and where smoking occurs, the school environment with regards to smoking and advertising of cigarettes.

Physical Activity Module Only:
Participation in physical activity and various sedentary activities, parental participation and attitudes towards physical activity, the school environment with regards to physical activity and self perception of athletic ability, self perceived body weight.
Both Modules: Age, gender, grade, peer smoking and physical activity, smoking behaviour, susceptibility to smoking, school connectedness, height and weight, and physical activity.
The SHAPES-Ontario (SHAPES-On) project is a recent large scale application of using a modular approach to data collection using the SHAPES tools.SHAPES-On was purposefully designed to inform aspects of the MHP provincial tobacco control and physical activity promotion strategies.As such, data were collected from students in participating schools using both the TM and PAM.At the request of the MHP, SHAPES-On data were not only collected to provide school stakeholders with data to inform their prevention activities, but they were collected to provided regional public health planners with data about the smoking and physical activity behaviours of youth in their public health region to facilitate the implementation of local public health interventions.Linking with public health in the SHAPES-On provides another opportunity to learn from an applied large scale data collection because some of the participating public health units (PHU) in regions requested to be part of the school recruitment process and others did not participate in the school requirement process.Considering the challenges associated with recruiting schools to participate in research studies, this allows us to examine the impact that partnering with public health stakeholders can have on school recruitment.
The present paper: 1) outlines the methodology and descriptive statistics for the SHAPES-On study; 2) examines the impact that different school recruitment models had on school and student participation rates; and, 3) examines the prevalence of different behaviours in the study sample depending on which SHAPES module students completed.

School Board Recruitment
As requested by the MHP, the six lead health units for the Ontario Tobacco Strategy [Tobacco Control Area Networks (TCANs)] were selected to participate.An additional health unit requested to participate due to previous involvement in a SHAPES project (refer to Leatherdale, Sparks & Kirsh, 2006).All public and Catholic school boards within the sampled health unit regions were eligible to participate.Independent schools and boards (including private and religious schools) were excluded.The outcomes for school board recruitment are shown in Table 1.

School Recruitment
Once participating school boards in each region were identified, we then approached schools within those boards in each region to participate.Procedures to approach schools varied across the different regions.In Region 1, the research team randomly approached 15 schools to participate.Since there were less than 15 schools in Region 2, all schools were approached to participate.In Region 3, the research team randomly selected 15 schools to participate but the PHU provided additional support so that the research team could approach an additional seven schools.In Region 4, the PHU worked with the research team to approach 19 schools that the PHU had previously established working relationships with.In Region 5, at the request of the PHU, the research team approached all 28 schools in the region to participate.In Region 6, the PHU worked with the research team to approach nine schools in the region which were purposefully selected by the PHU.In Region 7, the research team approached all 12 schools since there were less than 15 schools in the region.The outcomes for school recruitment are shown in Active information with passive parental permission was used for ascertaining consent from students and parents.This consent approach involved mailing information letters to parents which contained project details, project staff contact information and a 24-hour, tollfree number to call to withdraw permission for their son/daughter.These letters were prepared at the University of Waterloo, folded, stuffed and sealed in envelopes, and stamped with postage.These letters were then couriered to schools in time to allow parents at least two weeks to withdraw permission prior to survey administration.School staff produced the address labels for letters to be mailed to parents.The use of this procedure 1) protected student/parent privacy as the addresses were not released to UW, as well as 2) increased the likelihood of parental receipt of the letter.Students could refuse to answer any question, or the entire survey without prejudice.
In each participating school, classes within each grade were randomly assigned to have consenting students complete either the TM questionnaire or the PAM questionnaire.On the day of data collection, teachers administered the survey during a designated class period according to the provided instructions.Students completed the questionnaires in their classrooms.The survey took about 15 to 20 minutes to complete.To protect confidentiality, teachers were asked not to circulate among students as they were filling out the questionnaire or refer to students' copies of the questionnaire.

Measures
Vigorous physical activity (VPA) was defined as jogging, team sports, fast dancing, jump-rope, and other physical activities that increase the individual's heart rate and cause deep breathing and sweating.Moderate physical activity (MPA) was defined as lower intensity physical activities such as walking, biking to school, and recreational swimming.Reponses for VPA and MPA were provided by indicating the number of hours (0-4h) and 15-min increments (0-45 min) that each type of physical activity was performed for each day of the previous week.Detailed information pertaining to student activity intensity classification can be found in Wong, et al (2006).Body Mass Index (BMI) was calculated as self-reported weight (kg) divided by height (m) squared (BMI=kg/m2).
The smoking stage categories used were consistent with existing research (Leatherdale et al., 2005); with current smokers having smoked at least once in the 30 days preceding the survey, and ever smokers having smoked previously but not in the last 30 days.Smoking susceptibility among never smokers (never smoked a cigarette, not even a puff), was derived by three measures which asked students: a) "do you think in the future you might try smoking cigarettes?" b) "if one of your best friends were to offer you a cigarette, would you smoke it?"and c) "at any time during the next year do you think you will smoke a cigarette?" Students responded to these questions on a 4-point Likert scale and students who answered 'definitely not' to all three questions were considered non-susceptible; they were considered susceptible if they gave any response except 'definitely not' to at least one of the items listed above (Pierce, Choi, Gilpin, Farkas, & Merritt, 1996).

Analysis
For hypothesis testing we used t-test or Chi-square test based on the variable format.We based statistical significance on 95% confidence intervals or a p-value < 0.05.

Results
Table 1 shows the recruitment of schools by who recruited the school, the health unit (HU), the researchers (University of Waterloo (UW)), both the health unit and researchers working together (joint), or other options (board sent out information approving the project to principals, and the school recruited themselves before they had received a package; recruitment by the researchers for spring 2005 and health unit recruitment for spring 2006).Region 5 has the highest recruitment rate of all regions excluding Region 4 and 27 of the 28 schools that were approached in Region 5, were approached jointly.Region 4 is an exception because the health unit contacted the researchers and requested to participate.The region with the lowest recruitment rate, Region 6, did not approach any schools jointly.From these rates, it is apparent that for this project schools were more likely to be recruited when both the researchers and the health unit worked together to recruit the schools.This may be due to the fact that researchers' strengths complement health units' strengths with regards to recruiting schools.When these strengths are combined it follows that the best results would be achieved.
Abbreviations: HU = health unit UW = University of Waterloo Joint = both the health unit and researchers (University of Waterloo) working together Column 1 of Table 2 shows the total number of student enrolled in all secondary schools in the recruited boards.The enrolment data were obtained from the Ministry of Education in Ontario.
The second column of Table 1 is the total enrolment of eligible classes in recruited schools.Column 3 and 4 reports the total number completed survey and response rate respectively.Student level response rates were derived from student eligibility and classroom questionnaire completion totals, where eligibility was based on the presence of 1 or more students, in a given grades 9 through 12 class, on the day of survey administration.Region 5 has a relatively high student response rate, which may be related to the joint recruitment efforts of the researchers and the health unit.Region 6 has the lowest student and school response rate where none of the schools were recruited jointly.As expected Region 4 has the highest student response rate similar to school recruitment rates.
A total of 66,357 students were eligible for the survey and among them 51,739 completed the survey with a response rate of 78.6%.Students from ineligible grades (5-8) were excluded (n=316).As such, the final data set for SHAPES-Ontario consisted of data from 51,739 grade 9 to 12 students; 26,344 completed the Tobacco Module and 24,990 completed the Physical Activity Module.
Table 3 highlights differences in the core behavioural measures between the TM and PA samples.Students who completed the TM were significantly different from students who completed the PA Module with respect to BMI, the susceptibility score, the number of friends who smoke and the number of active friends.Among the males who completed core sections within each module, students were found to be significantly different with respect to most demographic variables, with the exception of the number of friends who smoke, whether the student had ever smoked and whether the student smoked in the last 30 days.Among the female responses to core questions, students were significantly different with respect to MVPA, the number of friends who smoke and the susceptibility score.The males who completed the TM reported a higher MVPA than the males who completed the PA Module.Conversely, the females who completed the TM reported a lower MVPA than the females who completed the PA Module.The sequence of questions in a questionnaire does matter since it is different between the two modules.The physical activity questions are at the end of the TM and similarly, the smoking questions are at the end of the PA Module.

Discussion
In Canada, schools are experiencing unprecedented requests for data collection with their students.National and provincial surveys have been unable to collect any data in certain major metropolitan areas because of the response burden being placed on schools (Health Canada, 2005).Clearly, coordination and collaboration among surveillance agencies is one part of the solution.But another part is being relevant to the education system.The modular design of SHAPES (i.e., TM and PAM) also helps to make our school-based research less demanding on the participating schools.Using classroom-based cluster-randomized sampling, we are able to collect representative data from an entire school pertaining to physical activity and tobacco while maintaining a 20-minute limit on total time to complete the questionnaire.By using a modular approach, we can more economically and efficiently collect data for a wider variety of behaviours without increasing the burden on participating schools or their student population.
Our results suggest that although there is some minor variation in the responses to core measures across the two groups of students who completed the different modules, this variation was generally smaller than what would have been expected if there were significant differences between the groups.
Another important issue such as participation of local public health staff also played an important role in school recruitment.Our results suggest that although randomly selecting schools to participate resulted in acceptable response rates, the school recruitment process was most successful when partnering with a HU who had existing working relationships with schools.Interestingly, the worst school recruitment rates were when we partnered with a HU who did not have past experience working in partnership with schools.This insight is important for school based researchers for two reasons.
• First, considering the substantial costs associated with recruiting schools (both financial and time costs), there may be substantial benefits from partnering with a PHU (or similar regulatory body with responsibilities to work with schools) who has strong working relationships with schools in instances where budgets are limited or there are time constraints.
• Second, considering the potential benefits for future research, it may be wise for researchers to work as brokers for helping to foster relationships between HU and school stakeholders where strong relationships do not yet exist.
This may not only benefit future research projects in relation to having access to schools, but it may also have the larger benefit of fostering links between schools and the HU who could be providing prevention services and resources within the schools.
An important issue faced by researchers working in applied school-based projects is the balance between internal and external validity.In the past, researchers have typically focused their efforts on issues associated with internal validity (e.g., designing studies as randomized controlled trials); however, the lack of 'real world' effectiveness resulting from such overcontrolled studies has led to a conceptual shift for increased attention to be paid to external validity (Glasgow, Bull, Gillette, Klesges, Dzewaltowski, 2002;Green & Glasgow, 2006).As such, a critical first step is the need for a greater practical understanding of the impact of recruitment procedures on external validity in applied research studies.For instance, an applied study such as SHAPES-Ontario, which utilizes a non-randomized sampling procedure, faces a challenge in overcoming selection bias.However, in conducting applied research in collaboration with practitioner organizations, practical issues may trump "pure science."This is particularly true for studies that involve government, education and public health organizations which are often required to comply with a relatively short turnaround period between the receipt of funding and associated deadlines for data collection (i.e., want to quickly see the impact of particular programs targeted to particular school districts).Designs that compromise aspects of external validity may have increased 'real world' value to the practice organizations, since they have results that are pertinent to their direct needs.

Conclusion
Much research which deals with youth and school environment lacks the credibility of their estimate due to low rates of recruitment of schools and participation from students.We believe this paper provides practitioners and school personnel with valuable information about the risk behaviour profile of a large youth population, while also providing researchers with novel research designed to improve recruitment procedures for future large-scale, school-based data collections.Finally, research collaboration with local public health was found to increase the likelihood of school participation compared to sole researcher recruitment strategies.

Table 2 .
Data Collection School board and school recruitment began in February 2005.Data collection was conducted in all 76 schools starting in May 2005 and ending in April 2006.Prior to implementation, all protocols and materials were approved by the University of Waterloo's (UW) Office of Research Ethics.