Youth Can ! Grow Healthy : A Formative Evaluation of a Positive Youth Development Program

This paper presents a formative evaluation of an afterschool program that combined positive youth development and school garden curricula. Novel approaches were used to teach elementary school children about gardening and nutrition, and to engage them in advocacy for healthy community physical activity and nutrition environments. The youth development curriculum included sessions on team building, community pride, healthy eating, physical activity, and advocacy. Photovoice methods were used to allow participants to assess their community and communicate findings with community leaders. The school garden curriculum included nutrition and gardening lessons. Formative evaluation was conducted for each session. Themes of the evaluation were: successful methods for engaging youth, issues in the social environment, and implications for program management. Evaluation results are discussed in relationship to relevant youth development literature to provide recommendations that will strengthen future programs.


Introduction
Obesity rates in children and adolescents ages 6-19 years tripled between 1980 and 2002 (Hedley, et al., 2004).Since 2002, this rate has remained stable, with about 35% of children and adolescents considered overweight or obese (> 85 th percentile of age-and gender-specific body mass index [BMI]) (Ogden, Carrol, & Curtin, 2010).This sustained high prevalence of childhood overweight and obesity is a public health priority because of the negative social-and health-related outcomes (American Dietetic Association, 2008;Han, Lawlor, & Kimm, 2010;Margery, Daniels, Boulton, & Cockington, 2003;US Department of Health and Human Services, 2010).
In response to this epidemic, numerous school-based interventions have been created and implemented to prevent childhood and adolescent obesity.Increased fruit and vegetable (FV) intake and physical activity are two primary goals for obesity prevention programs (U.S.Department of Health and Human Services, 2000).Traditionally, school-based programs that aimed to improve FV intake have used Social Cognitive Theory (SCT), a theoretical model that has been effective in improving health behaviors by addressing individual and environmental factors hypothesized to affect behavior ( Bandura, 2004a, Bandura, 2004b;Kelder, Hoelscher, Barroso, Walker, Cribb, & Hu, 2004;Reynolds, Hinton, Shewchuk, & Hickey, 1999;Reynolds, Yaroch, Franklin, & Maloy, 2002).However, two reviews of school-and SCT-based studies that aimed to increase FV consumption among youth found mixed results (French & Stables, 2003;Howerton, et al., 2007).Additionally, SCT has been used to develop garden-based nutrition education programs for youth that aim to increase FV preferences, self-efficacy to consume FV, and FV intake (Robinson-O 'Brien, Story, & Heim, 2009).Although promising, the results of these programs are mixed as well (McAleese & Rankin, 2007;Morris & Zidenberg-Cherr, 2002; O 'Brien & Shoemaker, 2006;Poston, Shoemaker, & Dzeqaltowski, 2005;Ratcliffe, Merrigan, Rogers, & Goldberg, 2011) and demonstrate the need for additional methods to improve FV consumption among youth.
The success of youth development programs in changing behaviors provides examples that could be used to develop school-based obesity prevention programs (Bloomberg, Ganey, Alba, Quintero, & Alcantara, 2003;Tebes, et al., 2007).Youth development programs focus on positive development, thereby creating an empowering environment where youth are allowed to make decisions and guide the program, and providing opportunities for youth to develop interests, build skills, and be recognized for their work by school and community members (Bloomberg, et al., 2003;Roth, 2004).These programs have been used successfully for smoking cessation and reduction of drug and alcohol use (Bloomberg, et al., 2003;Roth, 2004;Tebes, et al., 2007), but scant research exists on positive youth development programs that targeted nutrition and/or physical activity behaviors (Jones, Spence, Hardin, Miller, & Schoch, 2011;Bruening, Dover, & Clark, 2009;Story, Lytle, Birnbaum, & Perry, 2002).
Youth Can! Grow Healthy (YCGH), is an after-school program to promote physical activity and FV consumption and to teach students about advocating for healthy nutrition and physical activity environments.The curriculum includes integrated SCT, a garden-based nutrition education curriculum, and positive youth development.This program allowed elementary school children to engage in advocacy efforts to improve their community.As part of the YCGH pilot study, a formative evaluation was conducted.This paper provides a description of the program and the methods and outcomes of the formative evaluation with suggestions for improvements of intervention design, delivery, and program management that is useful for future youth development and empowerment studies.
The program was implemented over 16 weeks, meeting twice per week for one hour after school.Parents provided demographic information along with consent.Baseline anthropometric and demographic characteristics of participants are displayed in Table 1.Surveys were completed by participants during the first three group sessions and the two sessions prior to the final session.Height and weight measurements were obtained using standard protocol by trained researchers at the same time as survey administration.Participants received one day of garden-based nutrition education activities per week and one day of youth development activities per week.The principal investigator (PI), a graduate student researcher, was present for all sessions and led sessions under the guidance of the coprincipal investigator (co-PI), a trained youth development researcher and school-based nutrition interventionist.Additional support for curriculum delivery was provided by graduate students in the University of Tennessee's Public Health Nutrition Program and community partners, including a master gardener and extension agent.

Table 1
Baseline anthropometric and demographic characteristics of participants, 2010 The garden-based nutrition education curriculum was based on the Junior Master Gardener series (Seagraves, et al., 2005), and the youth development curriculum was adapted from the Youth Can! curriculum (Jones, et al., 2011).
Photovoice, which is an investigative method that allows participants to document and discuss their environment using photographs (Wang, 1994), was part of the youth development curriculum as a way to engage participants in assessing their community nutrition and physical activity environments and advocating for change.The SHOWeD method was used to guide discussions of participants' perspective on the photographs they took during Photovoice (Hergenrather, Rhodes, & Bardhoshi, 2009;Shaffer, 1980).This technique uses a series of questions about photographs to generate discussion.

Formative Evaluation
Formative evaluation of the pilot program was conducted using session evaluation forms completed by the PI.An existing tool for documenting and evaluating afterschool programs was used with permission from a local urban demonstration farm located in Knoxville, Tennessee.Information collected at each session included the name and associated organization of the session leader, the number of volunteers, the number of participants in attendance (as recorded at the start of each session), and a summary of the activities implemented, including what was effective and ineffective about each activity, and how the session outcomes were evaluated.Effective activities were judged to be those in which participants readily participated, were able to complete the activity in the allotted time, and required little or no disciplinary action from activity leaders.Activities were deemed ineffective if they lacked these characteristics.Field notes and qualitative reflections on why each session was effective or ineffective were listed on the back of each evaluation form.The PI completed an evaluation form after each session.
At the end of each session, the effectiveness of each activity was determined by the PI based on the criteria listed above and discussed at weekly meetings with the Co-PI.The evaluations and subsequent discussions were used to tailor the lessons for participants throughout the pilot intervention.At the conclusion of the pilot, evaluations of the youth development and gardenbased nutrition education curricula were entered into a spreadsheet and categorized as effective or ineffective components of each curriculum.These notes then were condensed to form summary statements about each activity's effectiveness.These summary statements were placed on two new overall lists, one of effective and one of ineffective activities, with activities from both curricula combined on the lists.Subsequently, the lists were inspected for themes and examined for common participant behaviors or lesson components that related to the level of lesson effectiveness.Modifications made to the delivery of the curricula during this program were taken into account.Trends in attendance data were analyzed with respect to recruitment strategies, conflict with other afterschool programs, family mobility, and behavioral issues.

Results
Analysis of the formative evaluation summary lists revealed three emergent themes: 1) successful strategies for engaging participants, 2) issues within the social environment, and 3) program management strategies.
A summary of these results follow.

Theme: Successful Methods for Engaging Youth
Creative Activities Activities that allowed participants to express themselves creatively were more successful in engaging participants than traditional didactic methods, such as when an adult delivered a lesson through lectures accompanied by handouts.Strategies that allowed creative expression included writing, drawing, taking photographs, acting, and singing.Creative activities were integrated into the curriculum whenever possible to give participants the opportunity to express themselves.

Missions
Giving participants fact-finding "missions" engaged them in finding information that supported program goals.This method initially was used when participants identified community leaders to invite to a meeting by searching in phone books and on the Internet.Participants were excited about finding basic information, such as addresses or phone numbers, and recording them for later use.Additionally, participants were given "missions" for both gardening and nutrition lessons.These "missions" often were phrased as "detective work," which resonated well with participants.

Working in Dyads
Working in dyads was a built-in component of some activities, such as Photovoice.The success of this strategy led to its use in other parts of the curriculum.Projects in which participants were asked to work alone often took more time than allotted.Therefore, on several occasions individual projects were combined into dyads or triads.

Experiential Activities
Providing experiential examples of concepts from lessons greatly increased participant interest in lessons.For example, in a lesson on plant parts, participants first reviewed the basic edible parts, and then real food examples of these parts were displayed for participants to identify.Also, participants enjoyed lessons in which they were allowed to go outside and plant, weed, water, or do other activities in the garden.These types of activities engaged the entire group in lessons and limited behavioral problems; and therefore, were incorporated as often as possible.

Cognitive Appropriateness
The analysis and planning components of Photovoice were challenging for many participants.The process of developing themes and a plan for improvement was adapted to the cognitive level of the participants in two main ways.First, during analysis of the photographs, questions were asked in context of "things that made it easy or hard for kids to grow up healthy," focusing on the nutrition and physical activity environments.SHOWeD questions about causation, such as "Why does this situation exist?" or "Why does this happen?,"were tested, but participants did not understand them, so these questions were dropped.Secondly, during the prioritizing and planning phase of Photovoice, participants had difficulty prioritizing themes to develop a plan.Therefore, participants were asked to set deadlines for each important issue to be resolved, which helped make the activity more cognitively appropriate for participants.

Theme: Social Environment Issues
Behavioral Problems Behavioral issues included off topic conversations, teasing, and difficulty focusing on lessons.Progress was slow on lessons that were not planned or adapted using the successful methods mentioned above.Despite reiteration of the ground rules to all new participants, those who joined the group after the initial team building sessions exhibited more behavioral issues than those who attended sessions from the beginning.Allowing long periods (30 minutes or more) for participants to work on creative projects or other unstructured activities led to increased teasing and off-topic conversations and activities.

Shyness
During the initial Photovoice sessions, participants described their photographs and provided rich perspectives on them.Some participants were reluctant to share in small groups and had to be encouraged to give their opinions on the photographs they took.Subsequently, participants developed a presentation for community leaders so that they could share their photographs and plan for improving the community, but were reluctant to speak during the actual meeting.
Gender Subgroups/Competition Gender subgroups formed when participants divided into self-selected groups for gardening tasks, classroom activities, scavenger hunts, Photovoice, etc. Subsequently, several lessons evolved into competitive games between the boys and girls.This friendly competition between gender sub-groups increased participation in activities, allowed projects to be completed in the allotted time, and limited behavioral problems.

Theme: Program Management Issues Attendance
To maintain engagement and facilitate the completion of group activities, students were allowed to join the group and participate in activities on an open basis.However, pre-and post-data were collected from only the 13 participants who provided parental consent and student assent (enrolled participants).Nine additional students attended group sessions and participated in all activities except BMI screenings and responding to surveys (non-enrolled participants).Attendance data are reported here for enrolled participants only.Average attendance of enrolled participants was 5, or approximately 38%.The highest attendance was 11 enrolled participants and the lowest was 2. Retention at the final session of the pilot was 23%.Most of the non-enrolled participants were recruited by the initial enrolled participants or their social networks.For example, a group of participants saw a friend during the community assessment using Photovoice.Subsequently, this student later joined the program and attended regularly.Another participant began gardening at home with materials received during a garden-based nutrition education session.Shortly after she began this project, several friends from the participant's neighborhood began attending YCGH sessions after helping her work in her garden at home.

Order of Activities
After the school day, participants enjoyed active or creative activities to begin sessions, but were not engaged by lessons that required reading or writing for more than a few minutes or adult-led lessons and presentations.Participants requested active projects to begin sessions and were much more willing to participate in more traditional lessons after active or less structured time.

Volunteer Training
Along with the PI, Co-PI, and graduate students, volunteers from four organizations assisted with sessions throughout the program.Because all of these volunteers had prior experience leading school garden or afterschool programs, there was no formal volunteer training before YCGH.It became apparent during the Photovoice sessions that some volunteers were not familiar with positive youth development techniques, which sometimes led to the compromise of the empowering aspect of the intended youth environment.This included times when volunteers gave participants suggestions for peer-led projects or provided answers to questions the participants were meant to develop on their own.The PI and Co-PI were able to shift the environment back to one of youth development and empowerment in most instances.

Team Building Activities
The first four sessions of the youth development curriculum were team building sessions designed to improve communication in the group, help group members get to know each other, and learn how to work with one another.These sessions were all well received by participants with full active participation throughout.Students who joined the program after these sessions lacked this experience and did not work as well with the group or follow expectations set by the participants during the first two sessions.

Survey Administration
The initial version of the pre-post survey developed to assess program impact was determined to be too long for participants to complete in a reasonable time.The baseline survey was administered over two sessions.After working on surveys for 30 minutes at the first session, surveys were collected and participants played a team building game.Surveys then were completed at the following session.After the two sessions for baseline survey administration participants commented that the program was not what they expected and that it was boring.For the posttest, the survey was modified to a shorter version, which participants were able to complete in one 30 minute session.

Discussion
Theme: Successful Methods for Engaging Youth Methods that were found to engage youth were creative activities, working in pairs, hands-on activities, and sending them on "missions" to find information.These findings are supported by others' suggestions to incorporate hands-on activities into every session of youth programs (Roth, & Brooks-Gunn, 2003;Strack, et al., 2004;Wilson, et al., 2007).The PI's completion of an evaluation form after each session helped to discover these successful methods.
Subsequently, this provided immediate feedback used to modify future sessions.This program incorporated more creative, hands-on, paired and "mission" activities after their successes were noted initially.These activities were appropriate for this group, but should not be generalized to all youth in this age group.For future projects, staff should complete their own formative evaluations to determine which methods work best with their target group (D.Wilson, et al., 2006).
The SHOWeD questions in the Photovoice activity were useful in eliciting meaningful responses from participants, but not all were appropriate for participants in YCGH.Participants were most responsive to the theme questions of "What do you see here?","How does this affect our lives?"and "What can we do?" (Hergenrather, et al., 2009).Participants were divided into small groups during this discussion to allow sufficient time to review all of the photographs.Each group was given responsibility for half of the photographs and then brought together to share findings.Strack and colleagues recommend dividing participants into small groups based on their developmental stage to make sessions appropriate for participants of all capacities (Strack, et al., 2004).Although Wilson and colleagues (2007) found that adolescents in their study had difficulty responding to the question "How does this affect our lives?"participants in YCGH easily were able to describe how their photographs affected their lives.This may have been because photographs were framed in advance as places where children could grow up healthy, focusing on places to eat and places to play or be active.With this added context, participants in this study easily were able to describe the implications of each photograph.However, they did have difficulty with the question, "Why does this situation exist?" even after it was framed in several ways.For this reason, this question was dropped from the Photovoice discussion.

Theme: Social Environment Issues
Behavioral Problems Some behavioral problems encountered in YCGH were off topic conversations, teasing, and difficulty focusing on lessons.Wilson and associates (2007) suggest setting clear standards for behavior and consequences for violating those standards.Strack and colleagues (2004) also recommend setting standards early by reviewing them during the recruitment process.Setting standards for behavior was moderately effective in this study.To adapt this practice to a youth development program, participants should be allowed to identify consequences for violating expectations for behavior for use in extreme cases of problem behaviors, although this was not used in this pilot-test.
For addressing behavioral problems, the use of positive reinforcement in response to desirable behaviors and extinction (planned ignoring) of problem behaviors is recommended (Ban, 1979;Shinn & Yoshikawa, 2008;Webster-Stratton, Reid, & Stoolmiller, 2008).The use of a visual display of behavior along with positive reinforcement, such as putting a check mark on the board when participants behave in accordance with expectations, can shape social norms in the classroom through the participants' desire to receive a reward for meeting expectations (Shinn & Yoshikawa, 2008).Researchers conducting Photovoice with youth have found that participants were preoccupied with peer approval and establishing social hierarchies during group sessions, which kept the researchers from having serious discussions about the students' photographs (Wilson, et al., 2007).Establishing rewards for the desired behaviors can shape social norms in the classroom by giving participants incentive to correct their peers' behavior (Shinn & Yoshikawa, 2008).
One common struggle in youth development programs is balancing discipline with empowerment (D.Wilson, et al., 2006;Wilson, et al., 2007).When facilitators focus on creating an empowering atmosphere and don't enforce expectations for behavior, participants may take advantage of the relaxed atmosphere by "acting out."Rather than blaming participants for their behavior, researchers recommend additional staff training in classroom management (Wilson, et al., 2007;N. Wilson, et al., 2006).Additionally, the use of positive reinforcement with participants can contribute to empowerment and has been shown to reduce behavioral problems among elementary participants in a randomized trial (Shinn & Yoshikawa, 2008;Webster-Stratton, et al., 2008).

Shyness
The issue of shyness was not realized until the later sessions of this program, so no steps were taken to reduce shyness of participants.Practice in presenting information or role play may be helpful for future interventions to encourage participants to share their findings with others.In conducting interviews with third to fifth grade participants for a formative evaluation of an obesity prevention program, researchers found that interviewing participants in pairs alleviated the shyness of participants (Gittelsohn, et al., 1998).The current study found that working in dyads allowed participants time to think while the other in the pair was talking, which stimulated discussion.This method could be used for a presentation on a Photovoice project by allowing participants to work in dyads to develop and deliver the presentation of their findings.
Gender Subgroups/Competition Activities can capitalize on gender differences by allowing youth to work in single-gender groups (D.Wilson, et al., 2006).In this study, allowing youth to self-select between choices of activity created gender subgroups, which helped forge new friendships within the subgroups.Borden and associates found that girls ranked self-improvement and community-improvement as the most important reasons for participating in a youth development program, while boys ranked games and social activities as the most important reasons for participation (Borden, et al., 2006).This shows that while both genders find participation in a youth development program important, reasons for participation, expectations, and preferred activities differ across gender groups (Borden, et al., 2006).
Channeling the energy of participants into healthy competition was a successful way to reduce problem behaviors and increase participation.Participants were highly motivated by games and competition to guess correct answers.Future interventions could incorporate competitive games into team building activities.Tests of knowledge on information taught during the program could establish a pattern where participants are motivated to learn by the desire for success in subsequent games.Freeman and Mathison (2009) suggest games as ways to improve group cohesiveness and provide a desirable experience to youth.

Theme: Program Management Issues
Attendance This program experienced a high attrition rate.This was in part due to other afterschool programs occurring at the same time.Although the PI consulted with school administrators about other afterschool programs and activities, new programs and activities were introduced after this consultation and information about some ongoing programs was incorrect or omitted.One suggestion for avoiding such conflict is to consult with school administrators, other afterschool program staff and teachers to create a chart of all afterschool programs, including ongoing and upcoming tutoring, sports, arts, and afterschool day care programs and then select days for the intervention with the least conflict with other programs.When attendance was low during the study, phone calls were made to participants' parents to check on participants' health and make sure that there were no concerns regarding the program or transportation.Attendance improved at sessions following the phone calls.Periodic phone calls to parents of participants exhibiting absenteeism may help improve attendance in afterschool programs.Classroom management also was found to affect attendance.Program leaders should be observant of any conflict between participants and address it immediately to avoid the development of negative feelings between participants or toward the program.O' Brien and Shoemaker (2006) reported an average attendance of 96%.Their afterschool garden program ran 10 weeks and served a snack at the start of every session.Although they did not discuss attendance, serving a snack at each session could improve program attendance by providing participants a time to socialize and relax before starting a lesson.D. Wilson and colleagues (2006) reported 55% attendance during an 8-week afterschool physical activity program, which was higher than this study's attendance rate of approximately 38%.

Recruitment
Recruitment through participants' peer networks could provide additional participants for afterschool programs.In this study, participants recruited their peers for the program on their own initiative.Asking participants to recruit peers for a program could lead to increased participation in future programs.Encouraging peer recruitment before the first session would allow participants recruited through peers to be a part of the research study.
To reduce attrition in a Photovoice program, Strack and colleagues (2004) recommend finalizing recruitment before beginning group sessions so that meaningful activities can begin immediately.They also recommend an application process that concludes in signing a contract to introduce program expectations early on and set a standard for accountability (Strack, et al., 2004).Finalizing recruitment prior to the first session is appropriate for studies which require pre-post measures or have a strict timeline for completing activities.
Conversely, Delgado and Staples (2008) state that youth development programs should accept participants into programs continuously to ensure the sustainability of the program.When youth are positively engaged in a program, they can be an important part of recruitment for the program and legitimize it to their peers (Delgado, & Staples, 2008).Because it was a youth development program, YCGH accepted new students throughout the year, which could have contributed to attrition and behavioral problems.Participants who were not present for team building activities were noted to exhibit more behavioral problems compared to those who were present.Not having an application process could have contributed to the attrition in YCGH by allowing participants to develop a low sense of accountability to the program.
Order of Activities D. Wilson and associates (2006) found that doing homework activities at the end of an afterschool program increased productivity and decreased behavioral problems.The present study's results show further support for this finding.The order of activities in afterschool programs should be considered so that participants have time to transition after the school day.If reading or writing activities are planned, ample active and/or creative time should be allowed in advance so that participants are ready to do thoughtful work (D.Wilson, et al., 2006).

Volunteer Training
Empowerment is a trait unique to the environment of youth development programs and is critical to achieving program goals.Creating an empowering environment does not come naturally to adults who are comfortable with providing specific directions and suggesting solutions when youth are not taking the expected path in an activity.Curtis (2008) notes the difficulty adults have in relinquishing control of a program and letting youth take the lead.The importance of staff training in creating an environment true to program goals was stressed by Wilson and researchers (2006) from the Youth Empowerment Strategies and the Teens Eating for Energy and Nutrition at School programs (Story, Lytle, Birnbaum, & Perry, 2002;D. Wilson, et al., 2006;N. Wilson, et al., 2006).Researchers highlighted the importance of training staff on enforcing program rules while maintaining an empowering environment (D.Wilson, et al., 2006;N. Wilson, et al., 2006).Staff, as well as all volunteers, for youth development programs should receive training on the components of an empowering environment and how to refrain from interfering with youth problem solving activities.
Team Building Activities Team building activities provide a time for youth to experience new ways of interacting with peers and facilitates the development of a new peer group (Freeman & Mathison, 2009).Formative results of YCGH support the use of team building activities throughout afterschool program curricula.Such activities could take place at the start of each session to allow youth a break before beginning mentally taxing work.Continuous team building activities throughout the program also would facilitate integration of new members into the group in a youth development program.Strack and colleagues (2004) did not incorporate team building into their Photovoice curriculum, but recommended team building during the first few sessions to improve the quality of group discussions.Researchers of the Youth Empowerment Strategies dedicated several team building sessions at the beginning of their program and continued to do team building activities as a component of all sessions (Wilson, et al., 2007).

Survey Administration
The length of the initial survey for this program was not appropriate for this study's sample.Although all components came from age-appropriate sources, the combination of components from several surveys led to excessive length.Adjustments were made after initial administration of the survey so that it was an appropriate length for the final assessment.These adjustments included eliminating duplicate scales of self-efficacy for FV consumption and using a validated, abbreviated measure of social desirability.No reports of issues with survey administration in youth programs could be found, but other studies report pilot testing surveys before administering them in a pilot program (Graham, Beall, Lussier, McLaughlin, & Zidenberg-Cherr, 2005;Neumark-Stzainer, Wall, Perry, & Story, 2003).Pilot testing survey measures with representatives of the target population is recommended to determine appropriateness of respondent burden.

Conclusion
These findings show that in an underserved, urban population, a youth development curriculum provided a framework for integrating team building and the creation of an empowering environment into an afterschool program.Photovoice was used to engage youth in learning about their physical activity and nutrition environments and advocating for change.Evaluation after each session was critical for identifying successful strategies for working with participants and modifying the curriculum to suit their preferences.Formative results of the YCGH program agree with others in key areas of program management, strategies for engaging youth, and issues within the social environment.These results support the recommendation of using, at least, a basic formative evaluation tool in programs with youth.Completing this type of evaluation is helpful in both adjusting an ongoing program to the target population and for informing future programs.