Staff Engagement and Camper Outcomes at a Low Ropes Course

The purpose of this study was to explore potential relationships between staff engagement scores and camper outcomes at a low ropes course for youth with serious illnesses. Two measures were created for this study: the Participant Adventure Outcomes Scale measured campers’ personal challenge and teamwork, and the Staff Engagement Observation Rubric was completed by low ropes course facilitators on “auxiliary staff” including volunteers and counselors not directly responsible for facilitating the low ropes course. Results showed no relationships between camper outcomes and auxiliary staff engagement. Implications for the measures and programming are discussed.


Introduction
While most adventure programs and research studies on challenge courses aim to see if there are effects on participants, few focus on the potential connections between what staff do and participant outcomes.Investigating the "black box" of program theory, we created and implemented a rubric to measure engagement of "auxiliary staff" while campers were at a low ropes challenge course ("Adventure"), and explored potential relationships between staff engagement scores and camper outcomes.Auxiliary staff members were defined as adults supervising and supporting youth participants at Adventure (i.e., volunteers and summer camp counselors) but who were not directly responsible for facilitating the low ropes course.Many adventure and experiential education programs include auxiliary staff to serve as additional safety monitors and to effectively communicate with participants using the auxiliary staff Journal of Youth Development | http://jyd.pitt.edu/| Vol. 13 Issue 1-2 DOI 10. 5195/jyd.2018.493Auxiliary Staff Engagement at a Low Ropes Course 241 members' pre-existing relationships with youth.For example, teachers who bring their students to a ropes course program would expect program staff to directly facilitate the program activities.Program staff would expect teachers to be engaged and alert for psychological or physical safety risks, drawing on their existing knowledge of their students.
The idea for this study came from Adventure facilitators who observed that some auxiliary staff members would frequently interact with campers during their Adventure experiences, but others would stand back and observe, or occasionally even act disinterested in the program.
Adventure staff wondered what the effects of different levels of staff engagement would be on camper outcomes.Identifying engaged behaviors can be useful for future staff and volunteer trainings.This study builds on previous research on setting-level factors and their potential influence on youth outcomes.For example, Roark and colleagues found that programming for specific outcomes such as friendship skills and teamwork could elicit these outcomes in participants (Roark, Ellis, Wells, & Gillard, 2010;Roark, Gillard, Evans, Wells, & Blauer, 2012;Roark, Gillard, Wells, Evans, & Blauer, 2014).
Developmental Systems Theory (DST; e.g., Lerner, 2015;Lerner et al., 2014) was used in this study to consider the processes between campers and their context in Adventure.That is, according to DST, the nature of the systems in which campers are embedded likely has bearing on their development.Youth development involves changing relations between developing youth and their shifting contexts, and acknowledges that youth exist in a larger social context (Lerner & Castellino, 2002).A key element of DST is fit: activities and experiences that are developmentally appropriate, interesting, and engaging, and that provide support via interactions with caring others and opportunities for building skills.For this study, we explored how auxiliary staff potentially shaped the context of the Adventure program and if staff engagement had any bearing on camper outcomes.

Setting
There were eight sessions of camp in 2016, with 20-25 campers aged 13-15 per Adventure group.Campers were youth living with serious illness such as cancer, sickle cell, HIV/AIDS, metabolic disease, hemophilia, and rare diseases.Except for two sessions only for youth with sickle cell and one session only for siblings of youth with serious illnesses, the remaining sessions were a mix of youth with different serious illnesses.Due to the high medical needs of the campers, the ratio of auxiliary staff to campers was typically 1:3 or 1:4.

Measures
The Participant Adventure Outcomes Scale was created for this study as a 7-point agreement scale (to allow finer parsing of responses and avoid floor and ceiling effects) to measure "personal challenge" and "teamwork" goals of the Adventure program.A response of one indicated "strongly disagree" and seven indicated "strongly agree.The Staff Engagement Observation Rubric (Table 1) was developed for this study in early 2016 through a focus group with two former Adventure staff, the program coordinator, and the author.The 7-point observation rubric ranged from red (7 = too engaged) to green (4 = just right) to violet (1 = too passive) to indicate level of engagement at Adventure and reflected the standards and expectations for staff engagement during Adventure.For each of the 15 groups (two groups per session except Session 5 which had only one group), Adventure facilitators used the rubric to observe and rate 184 cabin staff, program staff, and volunteers, that is, the auxiliary staff or adults who were not Adventure staff members.

Analysis
Data from the rubric were linked to camper data for each group, indicating the average auxiliary staff engagement level during each camper's group.Data were analyzed using descriptive statistics, exploratory factor analysis, and analysis of variance.

Camper Outcomes
The average camper outcomes score was 5.4 out of 7, indicating that campers mostly agreed with the questions in the survey.Cronbach's alpha for the 9-item scale was .819.Inter-item correlations ranged from .08 to .59. Figure 1 shows the average score campers gave to each item.Two factors of the scale emerged: feeling physically and mentally challenged, and everything else.When the two "challenge" questions were analyzed on their own, results showed that campers averaged 4.4 out of 7 for feelings of challenge, a full point lower than the average.This result indicates that campers did not feel very challenged during Adventure.
Otherwise, campers reported moderate or strong agreement with feelings of teamwork and enjoyment, and especially that their counselors were interested in them during Adventure.Campers reported lower feelings of challenge compared to other outcomes of Adventure.

Auxiliary Staff
Figure 2 shows the staff engagement scores for auxiliary staff on the six areas of evaluation in the rubric.The average auxiliary staff engagement score was 4.1 out of 7, indicating that staff engagement was green with a dash of yellow.For the auxiliary staff engagement rubric, 4 was the best -neither too engaged (red) nor too passive (violet).Cronbach's alpha for the six-point scale was .735,but would be .870if "other staff/volunteer interactions" was removed.Interitem correlations (including "other staff/volunteer interactions") ranged from -.06 to .80.Interactions with other staff, volunteers, and Adventure facilitators were close to perfect (4.0).

Connecting Camper Outcomes to Staff Engagement
The auxiliary staff engagement scores were averaged for each group and then added to the corresponding group of campers' survey responses to indicate the "climate" of auxiliary staff engagement each camper experienced.Correlation, regression, and analysis of variance analyses (including using group as a covariate) were conducted, but no statistically significant relationships emerged between camper outcomes and staff engagement.Both individual items and total averages of camper outcomes and staff engagement were analyzed, and no statistically significant relationships were found with either approach.
1. Perhaps the high quality of staff engagement (green with a dash of yellow) was sufficient and appropriate to influence camper outcomes.That is, there were so few instances of outliers of red and violet scores that these staff scores were not enough to negatively affect camper outcomes.While it can be disconcerting to learn that the behaviors of auxiliary staff did not affect camper outcomes, this study provides three opportunities.First, many questions were generated by this study that lead to further investigations and contemplation of programming, training, and structure.Second, the findings suggest to staff that auxiliary staff engagement is excellent and that camper outcomes are strong (although "challenge" could be strengthened).Third, this study provides two tools that can be used in other adventure-like youth camp settings in which non-facilitator adults are present.In conclusion, more research should be done on the effects of adults in youth development settings such as camp.
" The scale was reviewed and edited by the program coordinator in collaboration with the author.During all eight summer sessions in 2016, 331 campers completed the Participant Adventure Outcomes Scale during Adventure.Campers completed surveys immediately after they finished the course, and used their responses to participate in the final debriefing activity conducted by Adventure facilitators.

Figure 1 .
Figure 1.Average Scores for Camper Survey Items.

Figure 2 .
Figure 2. Average Scores for Staff Engagement Rubric Items.
surprising because dozens of rigorous research studies and decades of practice have shown that the adults in a youth development setting have an important influence on youths' Journal of Youth Development | http://jyd.pitt.edu/| Vol. 13 Issue 1-2 DOI 10.5195/jyd.2018.493

2.
Perhaps the most important adults in the Adventure setting are the Adventure facilitators and the influence of auxiliary staff on campers is more distant and therefore less important.3. Perhaps something other than adult behaviors in Adventure drives camper outcomes.For example, maybe outcomes are more influenced by peer interactions or by previous experience than by auxiliary staff.Still, the low-ropes Adventure program offered campers opportunities to develop teamwork and experience enjoyment, although results indicated that campers could experience more physical and mental challenges during the program.Further, the vast majority of auxiliary staff exhibit positive behaviors in line with what is expected by Adventure staff.Future research could examine other areas of camp to see if the same patterns hold true or if auxiliary staff might have more of an influence on camper outcomes in program areas other than Adventure.The auxiliary staff engagement rubric could be used in other camp settings.Due to its negative influence on the reliability of the rubric scale, the category of "other staff/volunteer interactions" should possibly be deleted because Adventure staff in this study seemed to vary considerably on how they assessed this category of auxiliary staff behavior.The rubric could be used to train volunteers and cabin and program staff.For example, during orientation, the rubric could be shared with new staff as examples of what to do and what to avoid.Finally, the rubric could be used as part of a program improvement process in which activity facilitators use the rubric to assess auxiliary staff members.Areas of strength and challenge could be assessed and targeted interventions made with individual staff or by adjusting directions for auxiliary staff at thebeginning of and during the program.One challenge to using the rubric is the amount of time needed for facilitation staff to assess auxiliary staff.However, investing the time can support facilitation staff members' deeper engagement in their work.Another challenge to using the rubric is the potential discomfort of staff evaluating other staff.However, this challenge can be addressed by positioning the effort as a way to name and explain the features of a positive youth development setting, and as a way to train and support auxiliary staff for their roles, which will ultimately benefit youth.