The Evaluation Skill-a-Thon: Evaluation Model for Meaningful Youth Engagement

Active engagement of youth participants in the evaluation process is an increasingly sought out method, but the field can still benefit from new methods that ease youth participatory evaluation implementation. Meaningful youth engagement in the evaluation process is particularly advantageous under the 4-H thriving model because of its potential to contribute to positive youth development, foster relationship building, enhance evaluation capacity, and improve program quality through improved evaluations. This program sought to facilitate actively engaging youth in the evaluation process by breaking it up into clear and manageable steps including evaluation design, data collection, data interpretation and analysis, reporting results, and outlining programmatic change. To achieve this aim, program staff designed the Evaluation Skill-a-Thon, a set of self-paced, experiential evaluation activities at various stations through which youth participants rotate. Actively involving youth participants in the evaluation process using the Skill-a-Thon model resulted in youth being able to identify and design programmatic changes, increased participation and response rates in other evaluations, and several youth participants’ gaining an interest in evaluation and working to design evaluations in later years. The Evaluation Skill-a-Thon holds promise for actively engaging youth participants in the entire evaluation process, easy implementation, and increasing evaluation capacity.


Introduction
In the past decade, the field of program evaluation has evolved considerably with increasing attention turned towards improving program quality and involving stakeholders through participatory evaluation (Arnold & Cater, 2011). The developing approach of youth participatory evaluation (YPE) is particularly pertinent to youth-centered 4-H programs because of its potential to (a) contribute to positive youth development (Checkoway & Richards-Schuster, 2003;White et al., 2012;), (b) build organizational evaluation capacity (Arnold & Cater, 2011), and (c) improve program quality (Brown, 2006;Lewis et al., 2019;O'Connor & Zeldin, 2005).
However, incorporating evaluation into the many tasks involved with program management remains challenging and more methods to bring youth into active roles within program evaluation are needed (Arnold & Gandy 2019). This paper aims to contribute to YPE literature by introducing the Evaluation Skill-a-Thon as one such new methodology which incorporates a participatory design, connects to the 4-H thriving model, and scaffolds the evaluation process into manageable steps which can promote youth and organizational evaluation capacity building. Inspired by the Skill-a-Thon model-an event which was originally designed to assess knowledge/skill development using experiential activity stations that program participants rotate through (Ullry & Wright, 2019)-the 4-H Healthy Living Ambassadors (HLA) program developed the "Evaluation Skill-a-Thon" to incorporate youth in the evaluation process.

Review of the Literature
Generally, participatory methods differ from traditional data collection in that participatory data collection is done with and/or by stakeholders as opposed to for them (Guijt, 2014). The aim of participatory data collection is to foster partnership as much as possible with the underpinning idea that stakeholders are uniquely able to provide insights which help evaluators identify why data should be collected, what data to collect and how, who should provide and collect the data, how to interpret and utilize findings, and more. Additionally, participatory approaches have been demonstrated to be effective in increasing the use of evaluation findings and teaching evaluation logic and skills (Patton, 1997). In the case of 4-H, youth are the key stakeholders uniquely positioned to benefit from and be an asset to the evaluation process.
Incorporating participatory methods into 4-H programming may help (Zukoski & Luluquisen, 2002) (a) ensure that the most relevant data is captured and interpreted correctly, (b) provide an opportunity for youth to reflect and consider next steps, (c) develop a sense of ownership Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 1 DOI 10. 5195/jyd.2021.968 The Evaluation Skill-a-Thon 102 among youth over data and findings, and (d) build youth's capacity to think analytically and utilize data.
Engaging youth in program evaluation is particularly pertinent to the 4-H thriving model because of YPE's possibilities for cultivating the four ingredients of high-quality programs identified under the model: (a) facilitation of youth sparks (self-identified & motivating interests/skills), (b) developmental relationships, (c) program quality standards, and (d) youth program engagement (Arnold, 2018). Previous YPE projects have found that encouraging youth to lead research projects helped them establish developmental relationships with the adult leaders they worked alongside (Powers & Tiffany, 2006). Importantly, engaging youth in the evaluation and modification of their programs gives them a designated space to feel heard by adult leaders. Furthermore, YPE could ease continuous improvements for higher program quality with the highly insightful assistance provided by youth. Finally, involving youth in participatory evaluation could prompt them to engage more deeply with 4-H programming due to increased ownership over evaluation results fostering stronger connections to programs and how they evolve (Zukoski & Luluquisen, 2002).
Evaluation skills are increasingly critical in fulfilling 4-H educator responsibilities but are not always something in which adult leaders have been trained, a tension which can cause apprehension, doubt, and reluctance surrounding program evaluations (Arnold, 2006). However, YPE programs can provide a facile entry point for both adult leaders and youth participants to strengthen their evaluation capacity if (a) the evaluation process is broken down into "concrete, manageable steps" (Fox & Cater, 2011, p. 2) and (b) evaluation competencies are structured into each step. While evaluation involves a vast breadth of components, YPE methods could focus on evaluation competencies such as identifying indicators to measure program success, forming evaluation questions, selecting/creating data-collection instruments, collecting data, analyzing and interpreting data, communicating results, thinking analytically, etc. (Arnold, 2006;Silliman & Guin, 2012). YPE methods should also cultivate evaluation knowledge and perception such as evaluation terms and concepts, qualitative vs. quantitative data, data validity and reliability, evaluation ethics, evaluation importance, evaluation impact, etc. (Preskill & Boyle, 2008). Considering these competencies and designing YPE methods to incorporate them in each evaluation step will make carrying out YPE more approachable and easier to implement while also facilitating evaluation capacity building for all those involved in the process.

Study Purpose
During the last community club meeting of the year, in 2018 and 2019, the 4-H members participated in an "Evaluation Skill-a-Thon." In 4-H, a Skill-A-Thon refers to a series of activity stations through which participants rotate (Ullry & Wright, 2019). The purpose of this event was to adapt the existing Skill-a-Thon model to design a YPE. This Skill-a-Thon aimed to (a) collect data for the end-of-year evaluation of the 4-H Healthy Living Ambassadors (HLA) program, (b) involve youth in the evaluation process, and (c) begin building the youth participants' evaluation capacity. Importantly, the Evaluation Skill-a-Thon also worked to show the youth participants that the adult leaders care about how they view their own experiences and that they are dedicated to providing engaging and meaningful experiences.
The Evaluation Skill-a-Thon was designed to collect data to answer the following targeted c. Increased interest in leadership roles.
3. What overall impact do youth feel the Health Living Ambassadors program has on them and in general?
The intended use of the end-of-year evaluation was to make judgements about which activities to carry forward in the next program cycle and to identify improvements to continuing activities.
Adult leaders also hoped for "process use": changes in individuals' thinking and behaviors, or changes in organizational procedures and practices which happen as a result of learning that occurs during the evaluation process (Patton, 1997). Specifically, leaders wanted participating youth to gain a deeper appreciation for evaluation and begin changing organizational practices.

104
The purpose of this paper will be to (a) share the Evaluation Skill-a-Thon (e.g., the activities implemented), (b) describe how information from the event was used, and (c) begin exploring how Evaluation Skill-a-Thons can be used as the foundation of an evaluation cycle which actively engages youth in every step of the evaluation process, building organizational and youth evaluation capacity and improving program development. In sum, this paper will focus on examining the merit of Evaluation Skill-a-Thons using Patton's (1997) guidance on evaluating an evaluation to consider the extent to which the desired uses of the evaluation described above came about.

Participants
The Evaluation Skill-a-Thon was carried out with and by youth participating in the Pima County 4-H Healthy Living Ambassadors program, a 4-H club at Tucson Village Farm that provides middle and high school-aged youth (ages 12 to 18) from around the county with the opportunity to learn about the importance of maintaining a healthy lifestyle and how to promote healthier habits within their own communities. Participating youth learn about concepts targeted around nutrition, healthy food preparation, fitness, gardening, sustainability, mental health, outdoor adventures, leadership, and service learning. The program follows a school year cycle; however, optional summer programming and volunteer opportunities are also provided. Monthly community meetings, topic-specific projects, and community service are required elements for participation in the program. These activities vary year-to-year and change based on the interests determined during the end-of-year evaluation.

Participatory Evaluation Skill-a-Thon Design
The evaluation process was broken down into the following steps: (a) evaluation planning, (b) data collection, (c) data analysis and interpretation, (d) sharing results, and (e) planning programmatic change. In the initial year (2018), project personnel worked with an external evaluation specialist to carry out Step 1 by designing activities that collected quantitative and qualitative data from youth using a participatory approach. The Skill-a-Thon model was selected to design the YPE method because of its engaging characteristics and ability to answer the endof-year evaluation questions. Steps 2 through 4 of the evaluation process were carried out together by adult leaders and youth participants during activities in designated sections of the Evaluation Skill-a-Thon.
Step 5 occurred casually in conversations at the end of the Evaluation Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 1 DOI 10.5195/jyd.2021.968 The Evaluation Skill-a-Thon 105 Skill-a-Thon event and formally during program planning over the summer. Particularly unique features of this event are that (a) youth were structurally included in most steps of the evaluation process and (b) the event served as a stepping stone to involve youth in every step of the evaluation process through changes to the Evaluation Skill-a-Thon design in subsequent years (participating youth were more engaged in the evaluation planning in 2019 and had organized to be formally involved through an Evaluation Skill-a-Thon team for the 2020 evaluation, but this was postponed due to COVID).
Activity stations in the Evaluation Skill-a-Thon were designed to gather information similar to that of survey questions but were remodeled to appeal to youth participants and to encourage more engaged responses. For example, the commonly used Likert scale was refashioned to mimic the A-F grading system with which most school-aged youth are familiar. Examples of a more engaged response style include answering questions through creating videos or newspaper headlines. The self-directed nature of the Skill-a-Thon enabled youth participants to feel a sense of ownership over the evaluation process as they were able to participate in the evaluation activity stations at their own pace and according to their own priorities. Furthermore, the Skill-a-Thon design required minimal oversight from adults, enabling them to act mostly in a supporting role with youth participants taking the lead. The data analysis and interpretation in the Skill-a-Thon design also facilitated increased involvement of youth participants-as opposed to a survey model which typically requires program personnel to aggregate, analyze, and interpret the data after participants have merely filled out the survey.
In total, the Evaluation Skill-a-Thon, which asked participating youth to evaluate the 4-H Healthy Living Ambassadors program, consisted of 14 activities, summarized in Table 1

Participatory Procedures
At the last club meeting, 4-H HLA youth were divided into two groups in which they carried out the evaluation activities, which were facilitated in two separate locations.

Data Collection
Group A carried out Activities 1 through 7 in Room 1 and Group B carried out Activities 8 through 14 in Room 2. Activities were self-guided, at their own pace, and carried out in any order according to each youth participant's individual priorities. There was one staff member in each room. Each group was given approximately 20 minutes to complete all of the activities. The Evaluation Skill-a-Thon 108 Data Analysis Once the adults observed that youth participants had completed all corresponding activities of the data collection portion, groups switched locations and were given instruction on the data analysis portion. In this section, each group analyzed the data collected by the other group.
Youth participants were allowed and encouraged to analyze the data from each of their corresponding activity sets. For each activity, youth participants were provided with writing utensils, the necessary instructions to analyze the data, and calculators for additional assistance. Instructions were kept simple and covered whether the analysis required counts, sums, or averages-along with the corresponding calculation formulas. For dot voting activities, youth participants were asked to analyze data by determining the top three activities listed. For the headline, drawing/writing, and video testimonials activities, youth participants were asked to analyze data by identifying common themes. For the grading and bar chart activities, youth participants were asked to analyze data by creating tallies to determine results. A full description of the analysis instructions can be found in Appendix B. The groups were allotted 12 minutes for the data analysis portion.

Data Interpretation
After completing the data analysis portion, youth participants then interpreted the data. Within the same groups and rooms, youth participants were given 12 minutes to select one station and delve into what data from that activity indicated about its corresponding targeted evaluation question. The discussions were framed by the following prompts: 1. What does this data tell you about the HLA program overall?
2. What does this tell you about how the HLA program helps youth build selfconfidence?
3. What does this tell you about how the HLA program helps youth build goal-setting abilities?
4. What does this tell you about how the HLA program helps youth increase their interest in leadership roles?
5. Based on the data provided, what should be done to improve the HLA Program?
Youth participants were also asked to record the results of their discussions by answering the above prompts on a sheet of paper which was collected at the end of the Evaluation Skill-a-Thon. Upon finishing the data interpretation portion, both groups of youth participants came back together in a central location to debrief the results they found through the evaluation process.
Youth participants self-selected which activity they wanted to report on and were called on to answer the following questions: 1. What did you learn today?

What surprised you?
At the closing of the evaluation Skill-a-Thon, staff members thanked the participating youth for their participation and shared what the next steps of the evaluation would be.

Results
In 2018 Project," in which youth participants will be able to help design, plan, and implement the next Evaluation Skill-a-Thon.
The Evaluation Skill-a-Thon was an effective method to conduct an end-of-year program evaluation and seemingly accomplished (to some extent) all of its intended purposes. Adult leaders and youth participants were able to use the collected, analyzed, and interpreted data to decide which activities to continue and identify how best to improve them. Since the Evaluation Skill-a-Thon, youth have become increasingly active participants in the evaluation processinitiating evaluation activities themselves, revising data collection activities, and responding to more data collection requests-pointing to an increased appreciation for evaluation by participating youth.

Conclusion and Implications
As evidenced by the results of the 4-H HLA Evaluation Skill-a-Thons, this method of YPE holds promise as an easily replicable model which could be brought to not only other 4-H programs, but any youth serving organization to evaluate and improve program quality, contribute to positive youth development through evaluative competency development, and build organizational evaluation capacity. The Evaluation Skill-a-Thon has a foundational base (as outlined in the study Methods) which can be implemented by other programs, eliminating the need to plan an evaluation program from scratch. Because program evaluation is broken into five steps, program practitioners can easily modify the Evaluation Skill-a-Thon to fit their program needs by simply making changes as necessary at whichever evaluation step needs adjustment. For example, practitioners can change the guiding evaluation questions to measure different program outcomes, change individual station activities to collect different data, change the data analysis and interpretation activities to include more or less complicated analysis, add Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 1 DOI 10.5195/jyd.2021.968 The Evaluation Skill-a-Thon 111 activities for youth to brainstorm program changes, and more. This adaptability of the Evaluation Skill-a-Thon is shown in the way it has evolved just within the years it has been carried out by the 4-H HLA program such as with the addition of an evaluation project for youth to plan subsequent Evaluation Skill-a-Thons and the changes to data collection activities. The development of an evaluation project within the 4-H Healthy Living Ambassadors also supports the findings of Goodyear (2003), that participatory evaluation allows youth to share ownership of the evaluation process-traditionally held solely by adults-diminishing the power differential that can separate youth, program stakeholders, and decision-makers. Together, the experiences of the 4-H HLA program indicate that the Evaluation Skill-a-Thon may be a useful tool for other organizations and 4-H programs to evaluate and improve program quality by including the active participation of youth in the evaluation process, enabling them to share their experiences, knowledge, interpretation, and ideas for program improvement.
The Evaluation Skill-a-Thon also contributes to YPE research as a method that supports positive youth development by providing a structure which is fun and easy to both grasp and plan, making it an ideal tool to interest youth in program evaluation and assist their transition to a coordinating role in the evaluation process. This too is facilitated by breaking the evaluation process into clear steps, allowing adult leaders-and the youth themselves-to ask where else youth can be incorporated and what competencies can be developed at each step. Of these five steps, the 2018 Evaluation Skill-a-Thon was initially structured to include youth participants in the last four steps but evolved towards including them in all five, a process which happened organically at both the initiative of youth participants and adult leaders by asking how to bring youth involvement to the first step of evaluation planning. The Skill-a-Thon model for participatory evaluation also helped to build the capacity of youth participants in handling and interpreting data-highly important skills for all citizens-from start to finish. By actively participating in the evaluation, youth participants were able to practice collecting data, analyzing it, interpreting its significance, and creating relevant responses-a process which they otherwise may have been exposed to only in piecemeal forms (such as in math class) and without a concrete, meaningful application. This shows how participating in the evaluation Skilla-Thon, which required simple data analysis of both qualitative and quantitative data, offers an entry point for youth to develop skills in evaluation which they can then progressively improve over time. The evaluation competencies and broader life skills that were and could be included in the Evaluation Skill-a-Thon program are detailed in Table 2. The Evaluation Skill-a-Thon

113
The Evaluation Skill-a-Thon model also has promising implications as a method which 4-H programs can use to improve evaluation culture in general. In the experience of the 4-H HLA program, the Evaluation Skill-a-Thon was an enjoyable and easily planned evaluation program that enabled sustainable, yearly evaluation of the program. Participating in a meaningful evaluation program and having their voices heard may also have made youth participants more receptive to other evaluation efforts; a subsequent evaluation of a different program sent out to youth participants over email during summer vacation-a previously experienced difficult period to gain participation from teenagers participating in our programs-received a response rate of 100%. This high response rate may indicate that youth participants were more willing to engage in evaluation because of a greater appreciation for the role evaluation plays in creating programming that is important to them derived from their prior participation in the Evaluation

Future Developments and Implications
Although there seems to be evidence to suggest that participating youths' appreciation for evaluation has increased after engaging in the Evaluation Skill-a-Thon, data could be more systematically gathered to (a) ensure that this evidence is not circumstantial, (b) assess how much the event contributed to their increased interests, and (c) more deeply explore implications for the wider youth population. For example, adult leaders could collect data through surveys or focus groups to specifically explore if participating youth do have an increased appreciation for evaluation, how the event may have contributed to this increased interest, what evaluation skills and competencies youth participants specifically gained from the event, and how to further build their evaluation capacity.
In the future, the possibility to adapt the evaluation Skill-a-Thon model to evaluate different types of programs should also be explored. The 4-H HLA Evaluation Skill-a-Thon was designed to evaluate a long-term program taking place over the course of a year among a consistent group of youth participants and adult leaders. What role could this model play in evaluating Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 1 DOI 10.5195/jyd.2021.968 The Evaluation Skill-a-Thon 114 short-term programs or singular events which may not have the required amount of time to facilitate the Skill-a-Thon as designed in this article? Further development of the program could also include exploration of additional creative ways to collect data. As it stands, the Skill-a-Thon contains diverse forms of data collection which can better relate to different modes of thinking or communication styles, but this could benefit from further investigation. The staff at Tucson Village Farm are currently working to continue developing more Skill-a-Thon activities, incorporate technology, and teach others how to implement their own evaluation Skill-a-Thons.