Process Evaluation Examining the Implementation of a Sport-Based Positive Youth Development Program

This study examined perceived staff implementation of program elements related to key programmatic design features of a community sport-based positive youth development (PYD) program for socially vulnerable youth. Previous research supports the ability of sport-based PYD programs to promote positive developmental outcomes, such as life skills. However, much remains unclear about the processes and factors that contribute to such outcomes. Specifically, there is a lack of understanding related to the implementation of key programmatic design features. Over the course of a 4-week sport-based PYD program, data were collected from 54 staff members at 60 time points. The degree of implementation related to key programmatic design features, including the program climate, curriculum usage, and quality of instruction were examined. Additionally, perceived implementation was compared across type and setting of sport activity. Analyses revealed that there was no significant difference in reported implementation for program climate, curriculum, and structure related to the type of sport (i.e., contact vs. non-contact sport). However, there was a significant difference in reported implementation of program climate and curriculum across the setting of sport activity (i.e., indoor vs. outdoor facilities). Implications for sport-based PYD program planning and management are discussed.


Introduction
The development of life skills is critical for all youth, especially youth who are socially vulnerable, lack social support, and have limited access to resources (Hermens et al., 2017;Newman, 2020;Super et al., 2018). From the perspective of social vulnerability theory, youth who are socially vulnerable are recognized as being confronted with and experiencing a culmination of negative risk factors that impede their overall healthy development, such as systemic racism, poverty, and neighborhood disorder (Vettenburg, 1998). Life skills are defined as "social-emotional learning, emotional intelligence, positive psychology, resilience, and character" (Hodge et al., 2012(Hodge et al., , p. 1127. In other words, life skills are abilities and behaviors that can be used in a variety of contexts and situations which enable individuals to manage the stressors, challenges, and demands of everyday life. As a learning context, sport is positioned to promote the development and transfer of life skills, particularly among youth who are socially vulnerable (Newman, 2020;Newman & Anderson-Butcher, in press). For example, research has indicated that among sport-based positive youth development (PYD) programs serving youth of color from economically disadvantaged urban communities, participation was associated with life-skill outcomes, including self-control, effort, teamwork, social responsibility, social competence, and transfer of learning (Anderson-Butcher et al., 2018;Newman, Anderson-Butcher, & Amorose, 2018).
While research supports the ability of sport-based PYD programs to promote life-skill outcomes, much remains unclear about the contributing processes and factors (Fraser-Thomas & Côté, 2006;Gould & Carson, 2008). At the program level, research has highlighted several antecedents of life-skill development. For instance, programmatic design features related to a program's climate, curriculum usage, and quality instruction are often believed to be most pertinent for life-skill development (Anderson-Butcher et al., 2014;Bean et al., 2016;Camiré et al., 2012;Newman & Anderson-Butcher, in press). However, recent research has demonstrated that sport program characteristics (e.g., sport type, setting of sport) can influence what youth sport leaders (e.g., program staff) deem important, particularly related to PYD . Thus, several important questions remain. For instance, to what degree do staff working in sport-based PYD programs implement the program elements related to key programmatic design features? And further, does staff implementation of program elements differ based on the sport activity characteristic? To advance this area of research, the current pilot study examined reported staff implementation of key program elements within a sport-based PYD program designed for youth who are socially vulnerable.

Sport-Based PYD Program Design Features and Program Staff
To promote the development of life skills among youth, staff and program administrators must intentionally design and facilitate sport-based PYD programs. Several key programmatic design features have been identified within effective programs. For instance, Eccles and Gootman (2002) suggested that program activities with embedded curricula and authentic instruction are key components of effective programs. Similarly, Anderson-Butcher et al. (2012) advanced that critical design features of community sport-based PYD programs include the program climate, curriculum, and instruction. Ultimately, program staff are responsible for ensuring that these key programmatic design features are implemented effectively through their intentional practices.

Program Climate
Effective sport-based PYD programs are characterized by a caring and supportive program climate that promotes task mastery. A positive program climate should focus on both physical and psychological safety (Eccles & Gootman, 2002;Perkins & Noam, 2007). Through their model of PYD through sport, Holt and colleagues (2017) forwarded that a PYD climate refers to the social environment, specifically relationships with key social agents (e.g., program staff) that enable youth to gain experiences promoting PYD outcomes. Research supports the importance of fostering a positive program climate. For instance, Gould et al. (2012) found that the more adult program staff created a caring, mastery-oriented climate, the more likely youth developed life skills related to initiative, teamwork, and social skills. Therefore, programs need to provide ample opportunities for staff to have positive interactions and foster meaningful relationships with youth.

Program Curriculum
A sport-based PYD program's curricular model also is believed to be imperative for the program to successfully develop youth life skills through sport. Several prominent sport-based PYD curricular models have been established in the literature. For example, Kendellen et al. (2017) integrated a life-skill curriculum in the sport of golf to teach skills promoting intrapersonal and interpersonal development. Using golf as an educational medium and teaching tool, each lesson was associated with a curriculum focused on one particular life skill. Similarly, the Girls on the Run program features a three-stage curriculum that focuses on developing self-care, connectedness, and empowerment across 24 distinct lessons . Additionally, curricular models, such as the Teaching Personal and Social Responsibility (TPSR), have been Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 6 DOI 10.5195/jyd.2020.890 Process Evaluation of a Sport-Based PYD Program 73 implemented in a variety of sport-based PYD programs. The TPSR curricular model, which is a guide to programming for youth-sport leaders, aims to help youth "take more responsibility for their well-being and be more sensitive and responsive to the well-being of others" (Martinek & Hellison, 2016, p. 9). As such, well-designed sport-based PYD programs, which are grounded in a curricular model, often have the dual priority of enhancing sport skills, as well as intentionally promoting life-skill outcomes.

Program Instruction
Research also suggests that program staff should provide structured instruction and facilitation during activities (Camiré et al., 2011;Newman & Alvarez, 2015;Pierce et al., 2018). For instance, program instructors should provide opportunities for youth to learn, develop, and then practice new skills (Fraser-Thomas et al., 2005). Activities also must be intentionally facilitated through the appropriate framing of skill lessons, the use of active facilitation during activities, and debriefing of critical learning experiences . Moreover, lessons should be framed to provide an overview of the upcoming activity to help youth understand how certain skills may be useful (Newman & Anderson-Butcher, in press). When facilitating an activity, naturally occurring teachable moments should be actively identified through the use of direct feedback, keywords, and metaphors. The process of debriefing should be used at the end of an activity or experience to promote the transfer of learning from sport to other life domains. Therefore, the intentional instruction of program staff plays a critical role in enhancing the ability of youth to develop life skills that are transferrable to other diverse contexts.
Although there is support for these programmatic features (i.e., program climate, curriculum, and instruction), much remains unknown about the degree to which program staff implement these strategies in practice (Kramers et al., 2019). Even well-designed programs may not promote positive youth outcomes if key design features are not implemented appropriately (Whitley et al., 2019). Given the importance of a program's climate, curriculum, and instruction, an investigation of program staff implementation is imperative.

Implementation in Sport-Based PYD Programs
Implementation is a process-oriented construct that can be used to further understand the design and delivery of sport-based PYD programs. Assessment of program implementation helps to illuminate the extent of adherence to aspects of a program (Fagan et al., 2008;Tucker & Blythe, 2008). Within sport-based PYD programming, implementation issues are just Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 6 DOI 10.5195/jyd.2020.890 Process Evaluation of a Sport-Based PYD Program 74 beginning to be discussed. For example, Iachini and colleagues (2014) identified barriers of program implementation, including program design features and program staff practices.
Moreover, when comparing profiling patterns of congruence among youth sport coaches, Kramers and colleagues (2019) found that intentionality is critical. The authors noted that adult program staff (i.e., coaches) "must be encouraged to link their intentions to their actions in manners that are both congruent and intentional" (p. 15). In other words, both the design features of programs and the practices related to the implementation of those features are critical. However, sport characteristics can often support or impede intentional implementation of key programmatic design features.
For instance, the type of sport can influence program staff's implementation of prescribed programmatic design features and life-skill outcomes. When considering distinct sport characteristics, the level of physical contact associated with the sport (i.e., contact vs. noncontact) has been found to have varying effects on youth outcomes due in part to sport-specific norms, such as aggression and violence (Newman et al., in press). For example, aggressive sport norms, often inherent within contact sports, can conflict with the efforts of program staff to cultivate a caring climate and reinforce prosocial behaviors (Lemieux et al., 2002).
Additionally, the setting of the sport (i.e., indoor versus outdoor) is also associated with differences in cognitive performance and understanding (Fernando et al., 2013). When considering facility characteristics, indoor multipurpose gyms with only a divider curtain separating different activities may contain divergent sounds which can disrupt activity facilitation (Fried, 2010). Outdoor facilities, however, pose the opposite challenge for program staff as sound does not carry long distances (Calvert, 2003). Moreover, a concern in outdoor facilities is weather, as rain or lightning can create significant risk and disruption (Seidler, 2013). Therefore, the activities offered by sport-based PYD programs are believed to influence the staff implementation of program elements.
As a whole, there continues to be little research investigating program staff implementation of key programmatic design features of sport-based PYD programs. Recognizing that such factors warrant further investigation, the current study examined staff implementation of program elements related to key programmatic design features of a community sport-based PYD program designed for youth who are socially vulnerable. Specifically, the study explored the degree to which program staff reported implementing strategies to promote a caring climate, aligning activities with curricula, and using effective instructional techniques. Additionally, this Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 6 DOI 10.5195/jyd.2020.890 Process Evaluation of a Sport-Based PYD Program 75 study examined the degree to which program staff implementation varied based on sportspecifically the type and setting of activity-as previous research has proposed that such features may influence staff implementation and PYD outcomes. The ability to enhance understanding of the processes and factors within sport-based PYD programs will allow for more effective programming and practices when serving youth.

Method
The program under investigation is operated at a large Midwestern university in the United States. All procedures were approved by the Institutional Review Board at the university. The community sport-based PYD summer camp, LiFEsports Summer Camp, integrated key PYD program design features with sport-based activities as a way to promote life-skill development of youth participants. The program was created for youth in the local community, many of whom were youth of color from economically disadvantaged communities. All youth (boys and girls) who participated in the program were between the ages of 9 and 15 years. Further, 83.5% of youth identified as Black and/or African American, followed by multi-racial (11.3%), White/Caucasian (4.4%), and "some other race" (0.9%). Additionally, 30.4% of youth participants lived in a household below the federal poverty line, and 69.8% of youth reported eligibility for free and/or reduced-price school lunch. Youth also stated that their household demographics were diverse (e.g., 43.1% lived with both parents, 42.1% lived with only their mother, 2.1% lived with only their father). Youth participants were randomly assigned into agestratified groups (Groups 1-24), with 25-30 youth in each group.
Adult program staff were responsible for fostering a caring climate, using curricula to teach sport sessions, and providing structured instruction and facilitation. Each of these programmatic design features was created to intentionally promote the development and transfer of key life skills, including self-control, effort, teamwork, and social responsibility. Specifically, each agestratified group of youth was led by one program staff member, who traveled with the group to each sport activity (throughout the duration of the camp). Additionally, at each sport activity, one program staff member (who was permanently stationed at the activity) was responsible for implementing the activity. Thus, the LiFEsports Summer Camp provided an opportune context to study staff implementation of program elements related to climate, curriculum, and instruction.
Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 6 DOI 10.5195/jyd.2020.890 Process Evaluation of a Sport-Based PYD Program 76 Over the course of the 19-day summer camp, the program was organized into 15 days of curricula that built towards a 4-day culminating event. Each day of curricula involved four 60minute sessions: three sport sessions designed to foster life-skill development by infusing lifeand sport-skill instruction, and one classroom-based education session designed to support lifeskill development through play-based activities. By the conclusion of the program all youth participated in the program's nine different sports and each sport's unique curriculum, including basketball, dance, football, health and fitness, lacrosse, soccer, softball, swimming, and volleyball. For more information about the LiFEsports Summer Camp program, please visit: osulifesports.org.

Participants
Data were collected from adult program staff who implemented the sport sessions. The sample consisted of 26 program staff who completed the implementation measure. Of the 26 program staff, 83.3% were new to the program, whereas 16.7% were returning. Among returning program staff, one reported having worked for the program for 2 years, one had previously worked for 3 years, and one had previously worked for 4 years.
The program staff consisted of 15 females (57.7%) and 11 males (42.3%), with ages ranging from 19 years to 27 years, and a majority of staff reporting a Caucasian (57.7%) or African American (19.2%) ethnicity (23.1% did not report ethnicity). Nearly all program staff reported high school athletics participation (95.7%), with 65.2% identifying as current college studentathletes. Additionally, a majority of staff were current college students (80.8%), with the other 19.2% college graduates.

Procedures
All program staff received training in how to complete the pilot version of the implementation The Session Log was designed to take approximately 5 minutes to complete. For data collection, program staff were provided with four copies of the tool at the beginning of the day and instructed to complete a Session Log after each 60-minute activity session. Therefore, the program staff completed 4 Session Logs for each of the 15 days of curriculum, for a total of 60 Session Logs per program staff (please note: 5 program staff were substitutes for staff who were unable to attend an entire day of camp). In total, 1,260 Session Logs were collected throughout the course of the program.

Measures
In order to assess perceived implementation of key programmatic design features across sport activities in a sport-based PYD setting, data were collected from two sources: the pilot version of the Session Log tool, and documentation of sport characteristics. The Session Log was a selfreflexive tool designed to measure perceived program implementation among program staff related to the program's climate, curricula, and instruction. Sport characteristics documented by the program were completed with regard to the context in which the Session logs were completed (i.e., type of sport and the setting of sport). A description of each measure follows.

Self-Reflexive Implementation Tool
A self-reflexive evaluation tool ("Session Log") was developed by the research team to measure the degree of perceived program implementation among program staff in relation to three key programmatic design features. Broadly, the use of self-report instruments to assess program implementation has been supported in the literature (Bean et al., 2018;Kramers et al., 2019).
In order to develop a tool specifically for use in a sport-based PYD program, existing implementation tools and available research pertaining to curriculum implementation, intervention, and treatment integrity were reviewed (e.g., Fagan et al., 2008). Researchers with expertise in youth development, sport management, and scale construction were consulted throughout the development of the tool.
Preliminary testing of the Session Log was conducted to determine the optimal number of subsets for grouping items and the internal consistency of the grouped items. Principal component analysis (PCA) was used to reduce the instrument without losing item information.

Program Climate
Program climate assessed the creation of a caring, mastery-oriented climate. The program's climate was measured by 14 items within the "Climate" and "Interaction with Youth" sections of the tool. Example items included: "Emotionally safe climate created" and "Opportunity for youth to make at least one decision."

Program Curriculum
Program curriculum assessed the use of the LiFEsports curriculum that was designed to promote life-skill development through the use of sport-based activities. Adherence to the program's curriculum was measured by six items within the "Curriculum" section. Example items included: "Use of curriculum lesson for the day" and "Emphasis on teamwork."

Program Instruction
Program instruction assessed adherence to the program's framing, facilitating, and debriefing framework. The program's instruction was measured by 14 items within the "Introduction," "Overview of skill," "Activities," and "Debriefing" sections. Example items included: "Introduction and demonstration of sport skill" and "Asking open-ended debriefing questions." For data organization purposes, staff recorded the date and which of the four daily sessions was being evaluated (first to fourth) on each Session Log. Sport Characteristics.
Additional information was collected about the context in which the Session Logs were completed, including type and setting of sport. Such information allowed for comparative analysis to determine which program characteristics relate to implementation of a sport-based PYD program.

Type of Sport
The type of sport was determined on the basis of the level of intentional or unintentional physical contact within the sport activity. Basketball, football, lacrosse, and soccer were categorized as contact sports, whereas, dance, health and fitness, softball, swimming, and volleyball were non-contact sports.

Setting of Sport
Setting of sport was determined based upon the facilities used. Basketball, dance, swimming, and volleyball were categorized as indoor sports, whereas football, health and fitness, lacrosse, soccer, and softball were categorized as outdoor sports.

Data Analysis
Data were entered into SPSS statistics v.25 software for analysis. Prior to analysis, the data were screened for missingness. Upon review, four program staff had four sessions of missing data as they were unable to attend 1 day of camp, for which substitute staff filled in for the missing staff and completed Session Logs accordingly. Further, four program staff had one session of missing data each as they were unable to implement the session due to rain. These systematic missing data were not treated (e.g., deleted, replaced) because no data were reported due to the missing session(s). Upon review of the remaining missing data, no patterns were discerned, suggesting the data were missing by chance (Kline, 2011)

. A Markov Chain
Monte Carlo (MCMC) data treatment technique was employed to treat all data missing at random (MAR) in order to reduce the amount of missing data. This technique assumes a MAR data loss pattern and takes advantage of the information in the data to yield a more accurate estimate (Kline, 2011).
Upon preliminary testing of the Session Log to determine the optimal number of subsets for grouping items, the data were reduced to allow for comparison of program implementation across programmatic characteristics. More specifically, the staffs' individual item responses were summed across their Session Logs. Subsequently, the item summations were added across the items associated with an individual program component (i.e., program climate, curriculum, instruction) for each staff member. Therefore, each staff member had a total program climate, program curriculum, and program instruction implementation score reflecting his or her perceived program implementation across the summer camp. For the purposes of readability, percentages were calculated based upon the total possible score (perfect implementation) for Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 6 DOI 10.5195/jyd.2020.890 Process Evaluation of a Sport-Based PYD Program 80 each program component taking into consideration systematically missing data. A score of 100% represented perfect implementation of that program component.
In light of the pilot study's limited sample size (i.e., 26 program staff), use of a non-parametric test was deemed appropriate as they are more robust to small sample sizes (Pallant, 2016). To examine the difference in program implementation across sport characteristics, a Mann-Whitney U Test was employed. Specifically, differences in the median program implementation percentage for program climate, program curriculum, and program instruction were tested across the type of sport and setting of sport. The Mann-Whitney U Test assumes independent observations (Pallant, 2016), which was met as all observations from program staff were computed into a single percentage for each respective staff member (per program component), representing a single independent (self) observation. To determine statistical significance, a p < .05 level of significance was adopted.

Implementation Subscales
Overall implementation of the program climate, curriculum, instruction was examined prior to considering differences across sport characteristics. In relation to the three program staff Whereas the majority of items related to program climate had high self-reported implementation and the program curriculum items had moderate implementation, a pattern emerged among the program instruction items. That is, implementation of the items reflecting instruction of the present day's sport and life skill were relatively high (mean scores ranging 3.49 to 3.63), whereas items reflecting review of past or preview of future sport/life skills had low to moderate implementation (mean scores ranging from 2.62 to 3.01).

Type of Sport
The degree of physical contact inherent within sport was considered in relation to the ability of staff to implement the prescribed program. A Mann-Whitney U Test revealed no significant differences in staff's reported implementation of the program climate (U = 84.00, z = 1.70, p = .096, r = 0.33), program curriculum (U = 67.00, z = 0.57, p = .601 r = 0.11), and program structure (U = 60.50, z = 0.13, p = .896, r = 0.01) across contact and non-contact sports (Table 1). Although not statistically significant, higher perceptions of program implementation were consistently reported for non-contact sports compared to contact sports. For both contact and non-contact sport activities, program climate had the greatest degree of perceived implementation, whereas program curriculum the lowest degree of perceived implementation.

Setting of Sport.
The sport-based PYD program under investigation utilized both indoor and outdoor facilities. A Mann-Whitney U Test revealed a significant difference in staff's reported implementation of the program climate (U = 12.00, z = -3.45, p < .001, r = 0.68) and program curriculum (U = 32.00, z = -2.29, p = .022, r = 0.46) across indoor and outdoor facilities, with program implementation significantly higher when sessions were implemented indoors (Table 1).
However, there was no statistically significant difference in staff's self-reported implementation of the program instruction (U = 38.50, z = -1.91, p = .06, r = 0.37) across indoor and outdoor facilities. Program climate had the greatest degree of perceived implementation for indoor activities, whereas program instruction had the greatest degree of perceived implementation for outdoor activities; program curriculum had the lowest degree of perceived implementation for both settings.

Discussion
Previous research suggests that sport-based PYD programs promote positive developmental outcomes, such as life skills (Anderson-Butcher et al., 2014;Bean et al., 2016;Camiré et al., 2012;Newman, 2020). However, much remains unclear about the processes and factors that contribute to such outcomes (Fraser-Thomas & Côté, 2006;Gould & Carson, 2008). Therefore, the purpose of this pilot study was to examine perceived implementation of key sport-based PYD programmatic design features related to program climate, curriculum, and instruction.
Additionally, the study aimed to explore the differences of perceived implementation across different types of sports. Taken collectively, findings from the current study support previous research that identifies implementation of program elements related to key programmatic design features as achievable processes for participant development (Anderson-Butcher et al., 2012;Eccles & Gootman, 2002;Fraser-Thomas et al., 2005;Perkins & Noam, 2007).
With respect to the type of sport (i.e., contact vs. non-contact sport), findings from the current study indicated that there was no significant difference in self-reported implementation for program climate, curriculum, and structure. However, whereas no statistically significant differences were found, there are several reasons why non-contact sports were consistently associated with a higher degree of implementation than contact sports. For example, previous research has demonstrated that youth, especially males, who participate in sports involving high levels of contact are more likely to engage in aggression and violence (Newman et al., in press).
Program staff must keep in mind norms that exist in sport outside of the context of sport-based PYD programs (Veliz & Shakib, 2012). As such, sports perceived as more physically aggressive may be more difficult to adapt in programs that promote life-skill development and a caring climate. However, the non-significant finding of the current study suggests that contact sports may not be as much of a risk factor as previous believed; and therefore, can be considered as a possible context for PYD. Future research may consider further exploring this misalignment.
Related to implementation of program climate and program curriculum, there were statistically significant differences between sports implemented in indoor versus outdoor facilities.
Specifically, program implementation was significantly higher when sessions were implemented in indoor facilities. To understand why indoor sports were reported to have higher, more consistent staff implementation of program elements than outdoor sports, the design of the program under investigation should be reviewed. Outdoor sports took place on one large, shared area that changed depending on weather. Indoor sports occurred separately from each  Rogerson and colleagues (2016) found that outdoor sports had increased social interaction during the activity, compared to indoor sports. Thus, inclement weather aside, outdoor sport participation may be more distracting in nature than indoor sports. If indoor sports offer greater degrees of control and consistency, this may explain why program staff reported higher levels of implementation within indoor sports and why perceptions of these sessions were less variant than outdoor sports.
Further, based on staff perceptions, program design features associated with program climate were implemented with the greatest degree of perceived implementation, whereas program curriculum design features were implemented with the lowest levels. Creating a caring, motivational climate has been shown to contribute to positive youth outcomes within sportbased PYD programs (Gano-Overway et al., 2009;Gould et al., 2012). Most sport-based PYD programs are designed so that all components (i.e., climate, curriculum, instruction) are mutually supportive and simultaneously implemented (Anderson-Butcher et al., 2012;Perkins & Noam, 2007). In practice, however, establishing a physically and emotionally safe climate through rapport-building and behavior management strategies might sequentially precede implementation of prescribed sport activities. Given the importance of creating a caring climate for achieving youth outcomes, the fact that program staff felt particularly competent in this aspect of the program is meaningful.

Implications for Practice
The current study has several implications related to program planning and management of sport-based PYD programs that aim to promote the development of life skills. Given research shows the practices of program staff are important, staff development and management (i.e., training and support) should focus on improving implementation of program instruction and curriculum. Ultimately, program staff must implement all critical sport-based PYD programmatic design features with high implementation in order for the program to have the greatest impact.
To support program staff to implement our program with high fidelity, we now incorporate multiple sport-based curriculum demonstrations, reviews youth development and behavior management principles, and break-out sessions for the individual program staff roles. Indeed, to enhance implementation, program administrators must provide a clear framework of the program model, curriculum, as well as staff roles and responsibilities during training (Tucker & Rheingold, 2010). The sport characteristics that may influence program implementation are also important to consider. The present study examined the type and setting of the sport. Findings revealed that program staff report the most difficulty adhering to the program climate, curriculum, and instruction when implementing the program in outdoor facilities. For example, program staff working outdoors might be in particular need of training on how best to maximize the use of space, given that there are not natural confines to offer structure and control. Program staff should consider this information as they make decisions about program design and management. For programs with an established model, program staff must consider program characteristics that may impede implementation and help to overcome obstacles in different settings. This support may manifest in greater staffing, targeted training, strategic curriculum development, and/or defining the program philosophy (Tucker & Rheingold, 2010).
With respect to design, programs that have the flexibility to choose from multiple sports should weigh the decision to use contact sports. Although the study findings suggest staff have the capacity to implement program elements in contact sports with fairly high fidelity, the literature suggests this particular context, potentially due to adherence to aggressive norms within certain sports, may be more difficult to mold to a sport-based PYD approach (Newman et al., in press).
Sport-based PYD programs that are already designed around a specific contact sport might benefit from preparing staff to handle unique challenges (and opportunities) that arise within these sport contexts. Similarly, extra preparation and supports could be built into management strategies aimed at staff working within any of the conditions shown to inhibit implementation.
Finally, the Session Log tool-which was used to assess perceived implementation among program staff in relation to climate, curriculum, and instruction-was found to be a reliable instrument. The Session Log tool was designed to be a brief and easy-to-use resource for program administrators and staff. Other forms of tools designed to assess implementation are often observational in nature, thus relying on a third party to consistently observe staff and report back, which can be both time consuming and costly. Due to the self-reflexive nature of the tool, program implementation can be evaluated efficiently by a single person, rather than utilizing valuable time and human resources. The Session Log can also be used as a capacitybuilding tool, as program staff reflect and evaluate their implementation through completion of the tool, creating opportunities for staff to identify areas for improvement.

Limitations
Altogether, this study illustrated how program staff perceived implementation in relation to prescribed programmatic design features and sport characteristics. However, several limitations must be considered when interpreting the findings. First, the study investigated a single sportbased PYD program that served youth who were recognized as being socially vulnerablespecifically youth of color from economically disadvantaged communities-limiting the generalizability of the results. Future research should explore the influence of sport characteristics on implementation in other contexts (e.g., physical education, interscholastic sport) and among youth populations (e.g., indigenous youth, youth with disabilities).
Additionally, implementation was captured only during structured time of the sport-based PYD program (i.e., program sessions). Anderson-Butcher et al. (2003) indicated that program staff implement strategies during unstructured time as well (e.g., travel between sessions).
Researchers should capture program implementation during both structured and unstructured time in future studies.
Measurement-related limitations also were evident. Preliminary testing of the Session Log tool provided guidance only for the reduction of the instrument. Since the completion of the pilot study, the Session Log tool has been refined, with construct validity and reliability tested . Further testing is needed to establish discriminant and convergent validity of the tool with external measures. Further, researchers may consider testing the tool's factorial invariance across staff of different genders and ethnicities, as the items may be interpreted differently based on the staff member's background (e.g., Caucasian program staff may be unaware of macroaggressions that impede a psychologically safe environment). The researchers were constricted in their statistical approach due to the limited sample size, therefore, future research should seek a larger sample necessary for parametric tests which are more rigorous and sensitive than non-parametric tests (Pallant, 2016). With a more robust sample, researchers could account for the nested structure of the data and examine potential staff effects. Future research should seek to confirm these results through investigation of a larger population, aiming to demonstrate statistical significance for greater implications and generalizability. In relation to the Session Log, only program staff self-perceptions were measured. Future studies should consider use of a third party to objectively evaluate implementation of staff practices or employ triangulation to verify the accuracy of the selfassessed implementation scores.

Conclusion
This study examined the perceived staff implementation of program elements related to the programmatic design features of a community sport-based PYD summer camp for socially vulnerable youth. In the end, the perceived implementation of three critical design features were described, including the program's climate, curriculum, and instruction. Staff implementation of program elements was compared across sports in varied settings (i.e., indoor and outdoor) and of varied types (i.e., contact and non-contact). Study findings can guide future program planning and management of sport-based PYD programs designed to promote positive developmental outcomes, particularly when serving youth who are socially vulnerable and in greatest need of these types of programs.