Capturing Youth Voice to Assess Learning in Urban Youth Development Programs

This article describes a two-part evaluation method that was designed to assess the nature of the learning experiences and the learning environments in urban youth programs by capturing the often absent voice of youth. It also presents evaluation results after delivering educational youth programs in an urban setting for one year. While youth across five program sites indicated their programs had strong program planning and delivery that provided intentional learning environments, the most common challenge across programs was a need for improved participation of both youth and adults. The evaluation findings convey the perspectives of young people on their experience as learners in youth programs. Practitioners may use such data in future planning as they employ strategies to improve the overall quality of their programs.


Introduction
Formal classroom education is often viewed as the most important component in a young person's education.Yet research shows that youth programs that take place during non-school hours also play a significant role in the education of young people (McLaughlin, 2000;McLaughlin, Irby, & Langman, 1994).In fact, programs may have an advantage in meeting the needs of youth because they can quickly adapt their way of working to better fit the changing circumstances, strengths, and conditions of youth.This article describes a two-part evaluation method that aims to assess the nature of the learning experiences and the learning environments in youth programs by capturing the often absent voice of youth.It also presents evaluation results from an urban youth program that was delivered for one year.

Method Phenomenological Essays
Typically, phenomenological methods are used in qualitative research to study the phenomenon of lived experiences.We adapted a phenomenological research method (see Giorgi, 1997) and applied it to an evaluation context.The first part of the evaluation utilizes phenomenological essays to describe youth learning experiences.van Manen's (2002) use of essays to study the educational experiences of children influenced the essay method used in this evaluation, in which youth were asked to write about their learning experiences in the context of a youth program.The data analysis followed four basic procedures: a) reading the data, b) dividing data into parts, c) organizing and transforming data into disciplinary data, and d) expressing the meaning in themes (see Giorgi, 1997).
Results, in the form of themes depicting the experience of learning, were supported by corresponding text excerpts from the youth essays.Data were drawn from 40 essays written by youth ages 8-12 across five program sites.

Five-Component Survey
In the second part of our assessment, youth from the five program sites collectively responded to a 5-component survey to determine the effectiveness of their program's learning environment in terms of: a) program planning and delivery, b) intentional learning, c) safety, d) membership and inclusion, and e) relationships with adults and the community (see the survey in Figure 1) Each survey category had six elements.An adult leader in each program administered the survey in a large group setting.Youth in each program were asked to indicate if the elements were present or not in their program.Overall scores were tabulated on a scale of effectiveness from 0 (low) -20 (high).Data were analyzed separately for each program site.A total of 60 youth ages 8-12 participated in the survey.
The survey instrument is shown below.

Figure 1
Learning Environment Survey • Mark a checkpoint with an X if more than half of the survey participants raised their hands.
In case of a tie, explain the checkpoint again, make sure everyone understands, and ask youth to share their reasons for answering one way or another.Then vote again.• Follow the directions at the end of the survey to obtain a score.

Program Planning and Delivery
Program To Score: For each survey category above, count the number of checkpoints marked with an X and circle the corresponding rating: 6=Excellent; 5=Very Good; 4=Good; 2-3=Fair; 0-1=Poor.
Then use this information to fill out the tables below.
Overall Score Add products of the following.

Results
Youth Learning Experiences Analysis of the 40 essays revealed four themes depicting the experience of learning: 1) the amount of fun youth are having is proportional to how much they are learning; 2) learning thrives with challenging experiences; 3) youth are hungry for new knowledge; and 4) personal satisfaction comes with a finished product.
Following is a brief description of each theme and corresponding text excerpts from youth essays.
1.The amount of fun youth are having is proportional to how much they are learning.
This does not mean that youth do not take their learning seriously.Rather, youth see that their learning is directly related to how much fun they are having in the process.
Here is what youth had to say.
"It was so fun that I didn't want to stop because it was a good experiment." "I loved it when we made slime.We used glue, water and food coloring.People said, 'Eww' or 'gross'.I just laughed.We got to play with it." "I felt excited when we did the music project….It really taught me the joy in music."

Learning thrives with challenging experiences.
Experiential learning is an ongoing process of discovery that begins by performing a task, sharing and analyzing the results with others, connecting them with other events in life, and then applying the results to new situations.Many youth thrived in their learning when they were involved in challenging experiences like science experiments as shown below.
"The reason I liked is because it is like a high school science experience and kids like us don't get to make volcanoes very often." "My team won the race and our team won the dam competition when our dam didn't break or get smaller."

Youth are hungry for new knowledge.
An intentional learning environment has the power to ignite intellectual curiosity and foster a desire for more learning.Here, youth described their own desire to know more.
"We made landfills out of chocolate pudding, vanilla pudding, licorice, and pie crust.
We learned where garbage would go, and I didn't know that." "I liked when some people came and we made a river and learning about how rivers can make sand fall in the water.I liked that day because we made a river.I want to learn more about rivers." "It is kind of fun when you can...try the things that you make and then you can bring it home and make it something else…"

Personal satisfaction comes with a finished product.
Whether it is a finished paper, completed science experiment, or polished artistic portrait, learners receive satisfaction from seeing the results of their work.Youth excepts follow.
"I felt like I was on a volcano island….It was fun because we actually got to make a volcano and we got to make it erupt." "The reason why I liked the landfill is because we got to make a new thing out of candy, we got to eat it before dinner."

The Learning Environment
The overall scores for the learning environment survey from five program sites ranged from 11-18 (on a scale of 0-20).These scores are at the 50 th -75 th percentiles and show good to excellent results in regard to the strength of each program's learning environment (see Table 1).• group decisions are made using an agreed upon process; • adult leaders are sensitive to the unique realities of each youth and incorporate this sensitivity into how they relate to youth; • parents and other family members are encouraged to attend events.
Each of these challenges center on the lack of youth and adult participation.

Relationships with Adults/Connection with Community
Adult leaders relate to youth with respect and dignity 5 0 Adult leaders guide, rather than lead, the decision-making around program priorities and activities Parents and other family members are encouraged to attend events 1 4 Youth are provided with opportunities to connect with individuals, businesses, and organizations in the community 3 2 There were 13 elements that youth most frequently indicated as present in their programs.The strongest categories for all five programs were program planning and delivery, and the intentional learning environment.Youth indicated that their programs had educational goals that attend to youth development needs.All five programs met regularly, incorporated handson activities, gave feedback and public recognition for youth contributions, and occurred in places where youth could develop relationships with peers.The youth also indicated experiencing positive relationships with adult leaders, who related to them with respect and strived to get to know them as individuals.

Discussion
McLaughlin (2000) calls for practitioners, as well as the larger community, to rethink how they design and deliver youth programs.Not only do young people want safe places to go, but they also want challenging learning opportunities within their programs.Recognizing that youth programs offer fertile ground for building effective learning environments is one of the first steps practitioners can take to making this happen.Youth programs offer prime opportunities to foster learning.This sentiment is echoed by the urban youth who participated in this evaluation.
Results show youth are hungry for new knowledge and they thrive when their learning experiences are challenging.
In order to meet these learning needs, it is important to be intentional about crafting a learning environment, first by planning educational content that is conducive to learning.This can be done in part by ensuring that curriculum is age-appropriate, program design is culturally responsive, and educational methods are engaging (Skuza, Cogshell, & Russo, 2007).In addition, youth learn most successfully when they use and connect new knowledge to other life experiences.In this way, knowledge actually becomes a part of their experience.
Incorporating experiential learning (see Kolb, 1984) as an integral part of youth programs is one way to ensure that young people process and apply what they have learned.McLaughin (2000) points out that community based programs are often the most powerful learning environments.They offer an informal educational environment where young people can learn in a more relaxed atmosphere.These intentional learning environments offer a time for youth to be themselves, sort things pursue an interest, or make friends, often with peers who are outside their segregated friendship boundaries (Skuza, 2005).
The survey data revealed many insights into the challenges that youth indicated were present in their programs.Youth and adult involvement was the common thread strung through these challenges.Applying Hart's (1992) ladder of youth participation as a model for creating a space for young people and adults to work together may help resolve these challenges.This model illustrates how young people can increase their participation and how adults can work with and support young people by responding to their participation.This delicate balance of power ebbs and flows as relationships grow.Allowing for young people's participation enables adults and youth to consider and create new ways of encountering one another that are based on mutual trust.

Conclusion
As youth programs attempt to set up learning environments, it is helpful to have specific strategies and goals in mind for creating successful places for young people to learn.The learning environment survey and the youth learning essays provide a means of assessing how programs intentionally foster positive learning experiences for young people and elements of these experiences.Through this evaluation, young people in the urban youth programs were able to articulate their most memorable learning experiences.They were also able to share their perspectives on the effectiveness of their program's learning environment.While youth at each of the five program sites indicated that their programs had strong program planning and delivery that provided intentional learning environments, the most common challenge of all five program sites revealed a need for improved participation of both youth and adults.
One unique attribute of this evaluation is that it captures data reflecting the often absent perspective of young people on their experience as learners in youth programs.Practitioners may use such data in future planning as they employ strategies to improve the overall quality of their programs.We also hope that this evaluation method and data will add to ongoing conversations practitioners have with young people about "what they are learning" and "how they are learning" in their programs.
This survey is intended to determine how programs provide the best possible place for youth to learn.It is based on positive youth development research.

Relationships with Adults/Connection with Community Adult
Provides quality learning experiences in small groups Program reflects the diversity of the community (e.g., race, gender, socio-economic background) Accessible to all those who want to be in the program All youth are welcomed into the program Each youth is included in program activities and decision-making Adult leaders are sensitive to the unique realities of each youth and incorporate this leaders relate to youth with respect and dignity Adult leaders guide, rather than lead, the decision making around program priorities and activities Adult leaders strive to establish individual relationships with youth in their program Parents and other adults in the community share their knowledge and expertise during program sessions Parents and other family members are encouraged to attend events Youth are provided with opportunities to connect with individuals, businesses, and

Table 1
Learning Environment Survey Scores

Table 2
presents the frequency of each element within the five learning environment components.The elements that youth most commonly indicated as not present in their programs are:• youth participate in the development of program goals, plans and strategies;

Table 2
Learning Environment Survey Frequencies