Evaluation of Youth Leadership Training Programs

Results of a two-year evaluation of youth leadership programs offered within community youth development programs in Connecticut are presented. Youth involved in leadership activities were contrasted with a comparison group of youth who were not involved in leadership programming. Participants in the leadership programs reported an improved sense of support from their local communities. Leadership training also appeared to offer an added benefit to males who reported significant improvements in their social self-efficacy in contrast to females engaged in leadership programs or youth comprising the comparison group. Youth who participated in the leadership programs appeared to be a uniquely talented group of individuals, initially scoring higher than the comparison group on a variety of youth outcome measures. However, a subgroup of youth who began the leadership program at a lower level of overall functioning were more likely than youth who began the program at a higher level of functioning to report positive changes.


Introduction
In recent years, youth development scholars have called for a paradigm shift from deterrence to development.This has led to an increased emphasis on asset-building and developmental readiness as opposed to problem prevention and intervention as the desired goals of youth programs (Roth and Brooks-Gunn, 2003).Programs that subscribe to this framework seek to positively influence youth development by fostering intellectual, social and emotional competencies.These competencies then serve as protective factors that lessen the likelihood that youth will engage in harmful or destructive behaviors.
Based on a review of the youth development and leadership literature (Boyd, 2001;Church, 2001;Libby et al, 2005), it was hypothesized that involvement with leadership activities would enhance the developmental competencies of youth within four general developmental domains: • personal adjustment • social competence • supportive connections to adults • level of involvement and connections with communities and neighborhoods.

Methodology Evaluation Design
The Leadership Programs that were included in this evaluation were based in 25 communities within the State of Connecticut.All participating communities were receiving funding for youth programming from the State of Connecticut's Office of Policy and Management.Fifteen of the communities were funded to provide programs for combating underage drinking.The remaining programs received funding for Juvenile Review Boards (n=2), Community Youth Advisory Committees (n=3), or Delinquency Prevention Programs (n= 5).
Despite the obvious differences in the focus of these various programs, all programs were required to provide youth leadership training that included: (1) youth involvement in program planning and decision-making (2) training in youth leadership (3) on-going interaction with program staff and other adults (4) active participation and involvement in local community initiatives.
The evaluation design included pre-testing and post-testing of youth who were engaged in youth leadership activities in participating communities.Each program was asked to recruit a minimum of 20 youth to participate in leadership activities in their program.Programs were also asked to recruit 30 youth to comprise the comparison sample of youth who had not participated in youth leadership activities.The comparison samples were generally drawn from classes or study halls within the local high schools, or youth in cafeterias during their lunch periods on randomly selected times and days.

Sample
A total of 586 youth from the twenty-five youth leadership programs were included in the evaluation.Another 747 youth comprised the comparison group.The total sample of youth involved in the study (N = 1333) included 547 males (42.1%) and 752 (57.9%) females (34 youth did not indicate their gender).All of the youth included in the study fell in the targeted age range of 12 to 18 years of age (Mean = 16 years of age) and were, on average, 10 th graders.Youth in the sample reported doing fairly well in school with 73% reporting a B average or better.Only 3.4% of the sample reported a D to F grade point average.The total sample was mainly Caucasian (65%, 866 of 1333 youth).Among the remaining participants, 13.5% were African American, 11.8% were Hispanic, and 2.1% were Asian.American Indians comprised less than 1% of the sample and nearly 7% of youth checked "other" when asked to report their race.The majority of the sample (51%) reported living with both their biological mother and father.Almost 26% reported residing within a single-parent family and another 12.5% reported living with a biological parent and a step-parent.The remaining youth reported living with close relatives (4%), foster parents (1.3%), non-relatives (1.3%) or did not answer the question.
Comparisons between the leadership group and comparison group revealed no significant differences between youth participants in terms of sex, race, family composition, age, or grade in school.

Measures
Youth involved in the leadership programs were contrasted with the youth comprising the control group on the following indicators.
Personal Adjustment.It was hypothesized that youth leadership programming would have a positive influence on youth's self-efficacy.Self-efficacy includes both knowing what to do in a particular situation and having confidence to carry out necessary tasks.Because research has shown that measures of specific types of self-efficacy are more predictive of adjustment than are global assessments of efficacy (Bandura, 1997), this study focused on three distinct selfefficacy indicators: social self-efficacy (the ability to relate to and communicate effectively with others), self-assertive efficacy (ability to speak up for one's rights), and self-regulatory efficacy (ability to resist negative peer pressures).
Social Self-Efficacy was assessed using a brief 8-item scale developed by Muris (2001).Muris reported high alpha coefficients for the scale ranging between .85 and .88.An exploratory factor analysis also showed the "majority of the items loaded convincingly on their intended factor" (Muris, 2001, p. 146).The average alpha reliability for pre-test and post-test scores on this measure in this evaluation was .74.
Two scales developed by Bandura (2001) were used to measure self-assertive efficacy and selfregulatory efficacy.The 4-item Self-Assertive Efficacy Scale and the 9-item Self-Regulatory Efficacy Scale were found to have average reliabilities of .75 and .81respectively in this study.

Social Competence.
A social skill thought to be affected by leadership programs was the capacity for empathy with others.Leaders are expected to be able to listen well to others, show sensitivity and explain the reasons for their decisions (McCauly & Van Velsor, 2003).Empathy was assessed by a subscale taken from the Teen Conflict Survey (Bosworth & Espelage, 1995).The 5-item scale has been shown to have an internal reliability coefficient of .83(Dahlberg, Toal, & Behrens, 1998).Previous research has demonstrated a significant relationship between lack of empathy and high rates of violence and interpersonal conflict between individuals (see Barnett et al., 1997).The alpha coefficients for this scale in this study averaged .72.
Supportive Connections with Adults.Youth in leadership programs spend a lot of time engaged in meaningful activities with adults.It was hypothesized that youth participants would develop supportive relationships with adults (staff) and perceive them as resources for dealing with social and emotional experiences.This outcome was assessed using the Presence of Caring subscale from the Individual Protective Factors Index (Phillips & Springer, 1992).The scale was developed for use in a large national survey of youth by EMT Associates (Dahlberg, et al., 1998).In a previous evaluation, the 9-item scale demonstrated an internal reliability coefficient of .65 (Gabriel, 1994).In the present study, the average alpha reliability for pre-test and post-test scores .79.
Connection to Community and Neighborhood.Leadership programs that participated in this evaluation involved youth in community projects.Thus, youth engaged in these programs were expected to report a greater sense of involvement and connection to their neighborhoods and communities.The two specific outcomes included in the evaluation were neighborhood support (receiving help and protection, and a sense that people work together in the neighborhood) and neighborhood activities (the perception that there are available activities, things to do, and places to gather in the community).
These outcomes were assessed by two subscales drawn from The Neighborhood Youth Inventory (Chipuer et al., 1999).They were the 8-item Neighborhood Support subscale and the 3-item Neighborhood Activities subscale.Chipuer et al. reported high reliabilities for the neighborhood support subscale, ranging from .92 to .94, and acceptable reliabilities for the neighborhood activities subscale, ranging from .75 to .81.Average alpha reliabilities in this evaluation were .95for support and .71for activities.

Results
Preliminary tests were conducted to determine whether youth engaged in leadership training in the four different types of programs included in the evaluation differed on any of the outcome measures.Analysis of variance tests pointed to only one significant difference between participants' pre-test scores.Specifically, the empathy pre-test scores of youth who participated in delinquency prevention programs differed significantly from the empathy pre-test scores of participants in the other three types of programs.Because of the absence of widespread differences in the scores of youth from the four program types, participants in the four programs were combined in subsequent analyses.

Youth Outcomes: Total Sample of Youth
Analyses were conducted using repeated measures analysis with pre-test and post-test scores as the within subjects factor and group membership (leadership, comparison) as the between subjects factor.In addition, gender was included as a between subjects factor because t-test analyses revealed that males and females differed on several of the outcome measures included in the study.Specifically, females scored higher than males on the pre and post-test indicators of self-regulatory efficacy (t = 4.29; p < .001and 3.75; p < .001,for the pre and post test contrasts, respectively), empathy (t = 7.22; p < .001and t = 4.97; p < .001),and the presence of caring (t = 3.06; p < .05 and t = 2.14; p < .05).
The results are summarized below for each outcome indicator.The pre-test and post-test means for each group of participants on each outcome are summarized in Table 1.Social Self-Efficacy.Males engaged in leadership activities reported a statistically significant increase in their social self-efficacy over time when compared to females within the leadership group and all of the youth within the comparison group (F (1,742) = 3.91; p < .05).These findings suggest that males in the leadership group believed they were better able to relate to, and communicate effectively with, others after completing the program.
In addition, between-subjects comparisons between youth in the intervention and comparison groups indicated that youth in the leadership group tended to score higher at both the pre and post-test intervals on reported levels of social self-efficacy when compared to the youth in the comparison group (F(1,742) = 14.1; p < .001).This suggests that the youth participating in the leadership activities perceived themselves as being more socially competent than the youth within the comparison group.
Self-Assertive Efficacy.Though the amount of change was very small (only a change of about ½ point), post-test scores were significantly higher for both the leadership and control groups (F(1,740) = 10.1;p < .002).This finding suggests that over time all youth believed that their ability to stand up for themselves increased.
Self-Regulatory Efficacy.The post-test scores of youth in the leadership group and youth within the comparison groups showed no significant increases in their self-regulatory efficacy at the end of the evaluation period.However, it is interesting to note that the self-regulatory efficacy scores of the youth within the leadership group were statistically significantly higher, at both the pre-test and post-test intervals, than the youth within the control group (F(1,729) = 23.09;p < .001).
Furthermore, there was a statistically significant group x gender interaction (F(1,729) = 9.36; p< .002).The pattern of mean scores indicated that males within the leadership group reported higher levels of self-regulatory efficacy when compared to the males within the comparison group.
Neighborhood Support.Youth engaged in leadership activities reported a significant increase in the level of support they experienced in their neighborhoods compared to youth in the comparison groups (F(1,738) = 4.28; p < .03).That is, youth in leadership programs reported positive changes in support, help, protection, and people working together in their neighborhoods.
In addition, between-subjects comparisons between youth in the intervention and comparison groups indicated that the youth in the leadership group tended to score higher at both the pre and post-test intervals on reported levels of neighborhood support when compared to the youth in the comparison group (F(1,742) = 11.3;p < .001).This suggests that the youth participating in leadership programs consistently experienced higher levels of neighborhood support than youth comprising the comparison group.
Neighborhood Activities.The post-test scores of the leadership youth and youth in the comparison groups were not significantly higher than their pre-test scores.Neither the youth in the leadership or control groups reported increases in their perception of neighborhood activities during the program period.
Consistent with the general trend of the findings reported previously, however, youth who participated in leadership activities reported a higher level of neighborhood activities than the youth within the control group, at both the pre-test and post-test intervals (F(1,749) = 4.54; p<.03).The leadership and control groups differed from the outset of the program in their perception of available neighborhood activities and these differences remained constant over time.
Empathy.The post-test scores of the leadership youth and the youth within the comparison groups were not significantly higher than their pre-test scores.Over time, neither the youth within the leadership or control groups reported increases in their reported levels of empathy.
Again, youth who participated in leadership activities reported higher levels of empathy than the youth within the control group, at both the pre-test and post-test intervals (F(1,749) = 19.8;p<.001).The leadership and control groups differed from the outset of the program in their perceived capacity to be empathic to others.
Furthermore, the pattern of mean scores indicated that the females within the sample scored consistently higher than the males in their reported levels of empathy (F(1,749) = 50.9;p<.001).
Presence of Caring.The post-test scores of the leadership youth and the youth within the comparison groups were not statistically higher than their pre-test scores.Neither the youth within the leadership or control groups reported increases in the presence of caring adults in their lives during the program period.
Consistent with the general trend of the findings reported previously, tests for differences between the youth in the leadership and comparison groups revealed that the youth participating in leadership activities reported higher presence of caring scores when compared to the youth within the control group, at both the pre-test and post-test intervals (F(1,735) = 5.3; p < .02).The leadership group participants differed from control group participants from the outset of the program, and throughout the program, in their perceptions of caring received from other non-family adults.
The pattern of mean scores suggested that the females within the sample consistently reported higher presence of caring scores when compared to the males within the sample (F(1,735) = 11.7;p < .001).
In sum, the results indicated that youth who participated in leadership activities reported significant improvements in two areas compared to youth who comprised the comparison group.Youth engaged in leadership programs reported greater neighborhood support than other youth.In addition, males in leadership programs reported improvements in their social self-efficacy compared to other youth.Males in the leadership group also reported significantly greater change in self-regulatory efficacy compared to males in the comparison group.

High and Low Functioning Youth
The findings to this point highlight a consistent difference between the youth participating in leadership programs and the youth making up the control group.On all of the outcome indicators, with the exception of the measure of self-assertive efficacy, youth participating in leadership programs scored higher, at both the pre-test and post-test, than the youth within the control groups.This pattern of results suggests the possibility that the youth who participated in youth leadership programs were: (1) self-selected (2) highly motivated and (3) more developmentally adjusted than their peers.
Because these individuals began the programs possessing relatively high levels of developmental skills and assets, it is less likely that their adjustment scores would change over the course of the program.
We decided, thus, to explore the possibility that leadership activities would have the biggest impact on those youth who were less skilled or competent prior to beginning the program.To accomplish this objective, a second set of analyses was conducted on two groups of youth involved in both the leadership programs and comparison groups -namely, those characterized by relatively high levels of functioning at the start of the study as compared to those functioning at considerably lower levels.These split groups were derived by computing the grand mean among all pre-test measures used in the outcome evaluation and using this statistic to divide the samples into thirds.Youth whose grand mean scores fell within the top and bottom thirds of the sampling distribution were retained for further analysis.
This method produced two separate groups of youth in leadership programs and two separate groups of youth in the comparison groups.The high functioning youth within the leadership and control groups included individuals who reported the highest level of functioning on youth development measures prior to beginning the program.The low functioning youth within the leadership and control groups included those youth who reported the lowest level of functioning on the youth development measures on the pre-test.These four groups were contrasted in order to determine if the lower functioning youth who participated in leadership activities derived the most benefit compared to higher functioning youth engaged in leadership activities or those in the comparison groups.
The analysis was conducted using repeated measures analysis with pre-test and post-test scores as the within subjects factor and group membership (leadership, comparison) and functioning (high, low) as the two between subjects factors.This produced comparisons in the amount of change each of the four groups noted above reported from the beginning to the end of the program.Our expectations in doing these analyses were: (1) participants in leadership groups would report greater changes than participants in the comparison group and (2) lowfunctioning youth in the leadership groups would show the greatest level of change following participation in the program.Significant results are reported below and the subgroup means are summarized in Table 2. Social Self-Efficacy.A significant 3-way interaction between level of functioning (high, low), group (leadership, comparison), and time was found for reported levels of social self-efficacy (F(1,535) = 5.13; p < .02).Low functioning participants in leadership programs showed significantly more positive change than did higher functioning youth in the youth leadership programs.The positive rate of change in the lower functioning group of youth leadership participants was also significantly better than the rates of change in either of the two comparison subgroups (high and low functioning).Youth who began the youth leadership program at a lower level of functioning on general indicators of positive youth development reported the most change on social self-efficacy scores.
This finding supported the initial evaluation question by finding a positive change for a subgroup of youth engaged in youth leadership activities over and above that shown for youth in the comparison group.The results also supported the secondary evaluation question by finding greater change among a subgroup of youth who began the project at a lower level of functioning.
Self-Assertive Efficacy.Consistent with the findings for social self-efficacy, when high and low functioning participants were considered in analyzing pre-test and post-test changes, a significant 3-way interaction was found for reported levels of self-assertive efficacy (F(1,528) = 8.7; p < .001).Low functioning participants in leadership programs showed significantly more positive change than higher functioning youth in leadership programs.The positive rate of change in the lower functioning group of youth leadership participants was also significantly better than the rates of change in either of the two comparison subgroups (high and low functioning).Youth who began the program at a lower level of functioning on general indicators of youth development reported feeling more able to assert themselves when dealing with others.
Neighborhood Support.A positive result also was found for the neighborhood support measure.A significant two-way interactions indicated that participants in leadership programs overall showed significantly more positive change than did youth engaged in comparison group activities (F(1,532) = 5.5; p < .05).This finding mirrored the result reported earlier for the analysis of the entire sample.
Neighborhood Activities.A significant 3-way interaction was found for reported levels of neighborhood activities (F(1,535) = 3.97; p <.05).The positive rate of change in the lower functioning group of youth leadership participants was significantly better than the rates of change in the high functioning leadership group and the rates of change in either of the two comparison subgroups (high and low functioning).

Presence of Caring.
A significant 3-way interaction also was found for the presence of caring (F(1,528) = 8.2; p < .05).Low functioning participants in leadership programs showed significantly more positive change than youth in the higher functioning leadership group.As in previous analyses, the positive rate of change in the lower functioning group of youth leadership participants also was significantly better than the rates of change in either of the two comparison subgroups (high and low functioning).Youth who began the leadership program at a lower level of functioning on general indicators of positive youth development reported feeling that there were now more trustworthy people in their lives outside of home and school.They could depend upon these people for help, guidance, advice, and support.
In sum, the most significant gains among youth were among those who started the leadership program at a lower overall level of personal and social development as defined by their overall score based upon the population grand mean.Low functioning participants in leadership programs showed significantly more positive changes in social self-efficacy, self-assertive efficacy, neighborhood activities, and the presence of caring from adults in their lives than did higher functioning youth in leadership programs.The positive rates of change in the lower functioning group of youth leadership participants were also significantly better than the rates of change in either of the two comparison subgroups (high and low functioning).

Definitions of Leadership: Interview Results
All programs that participated in this evaluation were required to offer leadership training and opportunities to a subset of youth who were engaged in their programs.However, because of the variety of programs that participated in the project, it was important to examine how programs had defined leadership and, thus, which leadership components programs were hoping to instill in youth participants.At the end of the first year of the two-year project one staff member in each program was interviewed to determine how the program defined leadership.
Several prominent themes emerged from these staff interviews.One of the most consistent themes, reported by 67% of respondents, was that teaching communication skills was an important dimension of leadership training.Effective communication was described as necessary for leaders.Listening was the communication skill most often cited by programs.
Other desired communications skills were offering support and encouragement to others, providing input and feedback to the program, and an openness to discussing current issues in the community.
These skills were thought to be facilitated by the modeling of leadership by staff members.Half of the participating programs believed that the staff involved with the youth should model desired leadership behaviors.For instance, respondents referred to the staff as "adult mentors" and "role models." In addition, 67% of the contacts defined leadership in terms of one's actions and attitudes towards others.Leadership was considered the ability to influence and motivate one's peers.A youth advisory council staff member stated that leaders should be "accessible to other youth and interested adults."Another respondent characterized leadership in terms of making healthy choices for oneself and encouraging those choices in others.In addition, leadership was described as providing "direction and support" for one's peers.
Furthermore, 85% of these leadership programs reported that they had allowed youth opportunities to perform in real leadership roles.That is, youth were offered opportunities to plan and execute various activities.For instance, a respondent from a Combating Underage Drinking program reported, "Leadership is about displaying specific skills, promoting community service, reviewing grant proposals and making funding decisions." Youth Advisory Committee members generally thought that offering youth the opportunity to provide valuable input to their local government would, in turn, provide youth with hands-on experience in leadership both by observing local officials and by having their voices heard in these influential settings.One Youth Advisory Committee staff member reported that youth "attend monthly meetings to discuss current issues that affect [town's] youth.Youth's presence at these meetings offers leadership skills such as good listening and providing input to [town's] staff and officials." Participation in community activities was another related theme.Fifty percent of respondents specifically mentioned youth contributing to a specific cause or helping those who were less fortunate.Some of these causes were supporting groups such as SADD (Students Against Destroying Dreams), raising money to improve the lives of kids in Connecticut, and promoting other kinds of community service.
Less common definitions of leadership also were mentioned by a few contacts.For instance, it is interesting to note that while many of the definitions and themes previously mentioned seemed to allude to responsibility as part of leadership, only one respondent specifically used the word responsibility.Another contact mentioned a personal commitment as part of leadership in reference to their program.Additionally, one Combating Underage Drinking representative described what was referred to as "Situational Leadership."According to this respondent, necessary leadership skills vary across circumstances and thus self-knowledge and self-awareness is essential to accommodate diverse situations and maintain one's leadership role.
Overall, the responses of program staff were quite consistent with how leadership was defined in the literature reviewed earlier.Themes such as empowerment, active participation, partnership with staff and other adults who model leadership skills, and service in the community were evident.

Discussion
In general, youth who completed youth leadership training were more likely than those who did not to feel an improved sense of support from their local communities.They viewed their neighborhoods as offering more support, help, and protection, and they reported that people worked more closely together.This is an important finding because many researchers have pointed out that a connection to the community as opposed to a sense of detachment and alienation is an important protective factor and predictor of positive adjustment and pro-social behavior (Catalano, et al., 2002;Coie, 1996;Hawkins, Herrenkohl, Farrington, Brewer, Catalano, & Harachi, 1999).
Leadership training also appeared to offer additional benefits to males.Males who participated in leadership activities reported significant improvements in their social self-efficacy when compared to other subgroups of youth (females in leadership, and all youth in comparison groups).Social self-efficacy refers to a belief in one's capacities to organize and execute the actions needed to manage interpersonal and social situations (Bandura, 1997).According to Bandura (1997), individuals' assessments of their social self-efficacy are based upon the acquisition of specific skills or skill sets.It appears that leadership involvement may have enabled adolescent males to acquire these important skills.Males in youth leadership also reported significant improvement in the self-regulatory efficacy (self control) compared to males in the comparison group.Self-regulation has been found to be important factor in countering the kinds of externalizing behaviors founds to be associated with antisocial behavior (Coie, 1996;Hawkins et al, 1999;Lipsey & Derzon, 1999).
Another important finding was that youth who participated in the leadership programs appeared to be a uniquely talented group of individuals.They scored higher initially on a variety of youth outcome measures when compared to youth who participated in the comparison group.We are not able to comment on how youth were actually recruited into these leadership programs, but it is likely that it was a combination of youth being referred by adults who saw some leadership qualities in them and self-selection.In the latter case, youth who are intelligent, highly motivated, and interpersonally skilled are likely to choose to participate in leadership training programs and activities.
However, the data also suggest several other factors that youth programs may wish to consider when recruiting youth participants into leadership programs.Despite the generally high level of functioning of participants overall, there was a subgroup of youth engaged in leadership activities that was less socially and emotionally skilled.This subgroup was most likely to report positive changes over the evaluation period.They were most likely to report significant improvements on such outcomes as social self-efficacy, self-assertive efficacy, neighborhood activities, and the presence of caring adults in their lives.Most programs that target positive youth development are universal in nature.That is, they are open to all youth in a given community who express an interest.However, programs may want to focus their recruiting efforts not only on those who have obvious talents, but also on those who may appear less suited for leadership training because they may be the ones who can benefit the most from it.

Recommendations
Finally, the evaluation results point to several recommendations to be taken into account in future evaluations.These include: 2.More effectively assess the kinds of activities youth in the comparison groups were involved in.That is, were youth who completed the comparison group surveys participating in no other, qualitatively different, or similar youth development programs and activities than youth in the leadership programs?This is an important concern because other researchers have suggested that it is the engagement in challenging and stimulating activities that lead to the development of important life skills (Catalano et al., 2002, Eccles, Stone, & Hunt, 2003;Larson, 2000;Walker et al., 2005).Without knowing more specifically the kinds of activities youth in the comparison groups were engaged in, we cannot separate the effects of leadership training from the effects of being engaged in challenging and stimulating activities.If members of the comparison groups were engaged in stimulating activities, this might explain why these youth also reported positive changes on some of the outcome measures.
3. Assess program dosage.Evaluation researchers have often pointed to frequency and length of participation as a potentially important consideration in maximizing positive developmental changes for youth (Sabatelli, Anderson, & LaMotte, 2001;Scheirer, 1994).The present evaluation included an item on the post-test survey that asked youth to indicate how long they had been involved in the program.Although length of time in the program, as measured by this item, was not a significant factor in explaining who profited from participation in youth leadership training, future evaluations might benefit from a more extensive assessment of this variable.This last point highlights the importance of systematically collecting accurate attendance data.

1.
Focus carefully on how programs define youth leadership and collect additional process data regarding what activities are offered in different programs that offer youth leadership training.It was reassuring to find that a majority of programs in the present evaluation shared many of the same basic assumptions about what constituted leadership skills training.However, it remains unclear to what extent programs offered a uniform set of training procedures and activities to promote youth leadership.

Table 1
Mean scores on pre-test and post-test outcome measures for males and females in leadership and comparison groups

Table 2
Mean scores on pre-test and post-test outcome measures for high and low functioning youth in leadership and comparison groups.