Measuring Life Skills : Standardizing the Assessment of Youth Development Indicators

While the development of life skills (e.g., communication, problem solving, etc.) is a commonly targeted youth program outcome, the lack of standardized conceptualizations and instrumentation make it difficult to compare impacts across programs and develop validated best practices. In order to promote a more unified approach to life skill development, literature reviews were conducted for 10 life skill domains to identify common definitions and, if available, appropriate outcome measures. Data were then collected from an ethnically diverse sample (N = 758) of elementary, middle, and high school aged youth for the 10 identified instruments. Analyses were conducted to ascertain the psychometric qualities of each measure, the interrelationships among measures, and the measures’ relationships with gender, ethnicity, and school level. Results are discussed in terms of their relevance to life skill theory and measurement.


Introduction
Over the last 20 years the field of positive youth development (PYD) has experienced significant growth and increased researcher and practitioner attention.This movement resulted in part from a developmental systems theory perspective on the inherent plasticity of developmental trajectories and the ability to influence these trajectories, especially among youth, to a greater degree than previously supposed (Lerner, Almerigi, Theokas, & Lerner, 2005;Lerner, von Eye, Lerner, & Lewin-Bizan, 2009).This paradigm shift has expanded the focus of adolescent intervention efforts to address not only risk and protective factors but also facilitators of positive development (Damon, 2004).A broad range of youth programs have become the delivery vehicle for the provision of positive development facilitators.
A central issue facing positive youth development programs has been the identification, evaluation, and measurement of potentially effective program models.A seminal work in this area was completed in 2002 by the Social Development Research Group at the University of Washington and involved the review of more than 150 evaluations of PYD programs (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002).Deliverables from this effort included a clearer conceptualization of PYD in general and key program outcomes and elements.The authors also noted the need for a standardized tool kit of positive youth development outcome measures.Such a collection of standardized measures would enable increased comparability of outcomes across programs and improved identification of best practices.
Since Catalano et al.'s call, a variety of efforts have emerged to conceptualize and operationalize key facilitating factors of positive youth development.For example, the Search Institute (Benson, 2003) has promoted its list of 40 Developmental Assets and concept of thriving, America's Promise Alliance (2007) has issued its five promises, and the Five C's (Lerner et al., 2005;Roth & Brooks-Gunn, 2003) have also received attention.Amidst this work, measurement development efforts have produced specific PYD related measures (e.g., Oman et al., 2002;Oman, Vesely, Tolma, Aspy, & Marshall, 2010;Park & Peterson, 2006;Theokas et al., 2005).While each of these efforts have positively contributed to the creation of a richer body of standardized measures, there is still a need to provide practitioners with access to validated measures for other commonly targeted program outcomes.
For example, 4-H programming has a long standing focus on the development of life skills (Boleman, Cummings, & Briers, 2005;Boleman, Cummings, & Briers, 2004;Boyd, Herring, & Briers, 1992) but the lack of commonly identified, specific life skill domains and accompanying standardized measurement approaches hampers the ability to promote a unified life skill development strategy across all 4-H programs.Fortunately, the USDA has been funding recent efforts to promote the standardization of life skill programming and outcome measurement.As an initial step in this effort, an online survey of program directors of USDA Children, Youth, and Families At-Risk (CYFAR) funded programs across the country was conducted to identify the most salient life skill domains (Duerden & Witt, In Press).These individuals were given a list of 35 skills from the Targeting Life Skills Model (Hendricks, 1998) and asked to select areas most in need of measurement standardization.The 10 most commonly identified life skills were used to guide this study.For each of the 10 life skill areas a review of literature was conducted, available instruments assessed, and a target instrument was selected for further study.A subsequent field test was conducted utilizing the 10 selected instruments.

Life Skill Literature Review
Literature reviews were undertaken for the following life skill areas: communication, community volunteering, critical thinking, decision making, leadership, problem solving, responsible citizenship, self-esteem, self-responsibility, and teamwork.As noted, conceptualizations were selected or developed which were simple enough to afford practitioner accessibility but robust enough to provide a strong link to an associated measurement tool.A total of 170 references were reviewed across all life skill areas.The following sections provide an overview of each life skill.

Communication
The mechanics and components of human communication have been discussed and debated since the time of the ancient Greek philosophers (Bowman & Targowski, 1987) and multiple models and theories have been developed to describe the communications process.Although differences exist across models, most contain the same core elements of speaker, message, and listener (Lysaught, 1984).The selected definition for communication was drawn from the work of Mincemoyer, Perkins, & Munyua (2001) who define communication as "the dynamic process by which people exchange thoughts, ideas, and messages.Listening is the act of interpreting sounds and/or visual stimuli and using those interpretations to give them meaning."

Community Volunteering
While adults are more likely to volunteer in order to make a contribution to their community, youth volunteer to gain skills, explore career fields, or to participate with friends who also volunteer (Shannon, 2008).Community volunteering allows youth to explore new roles and identities, foster a sense of empowerment and self-efficacy, and advance a personal set of morals (Youniss, Mclellan, Su, & Yates, 1999).It may also advance academic learning through improved skills in decision making, abstract thought, and reasoning (Conrad & Hedin, 1989).For this study community volunteering was defined as an attempt to meet a community need through uncompensated giving of time and or talents to others.

Critical Thinking
Critical thinking has long been identified as one of the key goals of education (Bloom, Englehart, Furst, & Hill, 1956).It is considered a key skill necessary for lifelong learning (Terenzini, Springer, Pascarella, & Nora, 1995) and navigating an increasingly complex world (Facione, 2006).In an effort to produce a more parsimonious measure of critical thinking geared towards adolescents, Perkins and Mincemoyer (2002) developed a unique conceptualization of critical thinking based upon their own review of literature.Their work resulted in a conceptualization of critical thinking as "thinking that evaluates reasons and brings thought and actions in line with evaluations" (Mincemoyer, et al., 2001).

Decision Making
Individuals utilize the skill of decision making daily.When a child picks out his or her outfit for school, this involves decision making.When a teenager is considering whether or not to do drugs, this involves decision making.When a high school graduate is debating what college to attend, that involves decision making.Some choices will not be tough to make while others require some serious thought by adolescents.The problem arises, however, because "adolescents are presumed to be more susceptible to peer influence, have a tendency to focus more on immediate rather than long-term consequences, and to be less risk averse" (Scott, Reppucci, & Woolard, 1995, p. 222).Effective decision making requires the "the ability to control one's thoughts and behaviors in situations in which there is a relatively strong motivational component (e.g., rewards and losses tied to one's performance)" (Carlson, Zayas, & Guthormsen, 2009, p. 1076).For the purpose of this project, decision making was conceptualized as the ability to define a problem, choose between alternatives, identify the risk and consequences for each alternative, selection of an alternative, and finally evaluating the situation (Mincemoyer & Perkins, 2003).

Leadership
A plethora of definitions exist for leadership.Stogdill and Melvin (1974) state that "there are almost as many different definitions of leadership as there are people who have tried to define it" (p. 7).Bennis and Nanus (2003) add to this by noting that "decades of academic analysis have given us more than 850 definitions of leadership" (p.4).This diversity makes it difficult to identify a universal definition.For the purpose of this project, leadership was defined by the most common elements that recur throughout the literature.Accordingly, leadership was defined as the ability to interact with a group to exercise influence and achieve a common goal (Mills, 2009).

Problem Solving
Solving problems is the process of using reasoning and analysis to look beyond the surface of a problem to the underlying concepts that need to be part of the solution.Problem solving is a sub concept under the meta construct reasoning which can be defined as "the inferential steps that lead from a given state of affairs to a desired goal state" (Barbey & Barsalou, 2009, p. 35).
For the purposes of this study, social problem solving was defined as "the self-directed cognitive behavioral process by which a person attempts to identify or discover effective or adaptive ways of coping with problematic situations encountered in everyday living" (D'Zurilla & Maydeu-Olivares, 1995, p. 410).

Responsible Citizenship
The majority of the literature addressing responsible citizenship states that a responsible citizen is civically engaged, whether by community volunteering, participation in advocacy groups or recycling.The term responsible citizenship and civic engagement are often used interchangeably, especially when referring to youth who may not yet be of age to vote or participate in political processes.Responsible citizenship is best defined by Bobek, Zaff, Li and Lerner's (2009) multi-construct definition of civic engagement: 1. Sense of generalized reciprocity 2. Ability to be involved in civic society and democracy 3. Desire to make positive contributions to community 4. Participation in activities to better community (p.616)

Self-Esteem
Self-esteem is generally thought as one's own perception of worth.The term is often used synonymously with self-concept, self-image, and self-worth (Butler & Gasson, 2005).DeBord (2001) defined self-esteem as "the degree to which children feel accepted and valued by adults and peers who are important to them" (p. 1).Self-esteem is constructed when one compares oneself to other people such as family, friends, or characters seen through the media.An individual is consistently receiving confirmation from these sources to see if they have matched up to their own expectations.Therefore, based upon the reviewed literature, we defined selfesteem as the construct resulting from the degree of congruence between perceptions of their ideal and real selves, perceptions based upon comparisons with and feedback from others.

Self-Responsibility
The concept of adolescent self-responsibility has connections to a broad array of literature.Researchers have tied self-responsibility into discussions of altruism and personal norms (Schwartz & Howard, 1984), academic outcomes (Spencer, Dupree, & Swanson, 1996;Wang & Stiles, 1976), and values-based physical education interventions (Watson, Newton, & Kim, 2003) to mention just a few.One potential approach comes from a collaborative project between researchers at the University of Utah and the American Camp Association (2007a).In an effort to assess the impact of camp experiences on youth participants, this team created measures for seven constructs including responsibility (Ellis & Sibthorp, 2006).Ellis and Sibthorp defined responsibility as the "habit of owning and accepting consequences of personal actions" (p.38).Two sub-domains were identified: ownership (i.e., taking ownership for one's actions) and correction (i.e., willing to accept correction and consequences resulting from one's actions).This definition appears most applicable for PYD contexts.

Teamwork
Definitions of teamwork converge on three key areas: actions, people, and tasks.For example, teamwork can be defined as "the set of interrelated behaviors and actions that occur among team members while performing on a task" (Salas, Burke, & Cannon-Bowers, 2000, p. 344).Similarly, "teamwork is a set of interrelated thoughts, actions, and feelings of each team member that are needed to function as a team and that combine to facilitate coordinated, adaptive performance and task objectives resulting in value-added outcomes" (Salas, Sims, & Burke, 2005, p. 562).There seems to be a general consensus among researchers who have studied teamwork that it is a multidimensional construct comprised of specific knowledge, skills, and attitudes (Lembke & Wilson, 1998).Therefore we defined teamwork as the knowledge, skills, attitudes, or behaviors that increase the likelihood that an individual can be an effective member of a team.

Methods
The study was designed to identify and test measurement tools related to 10 life skill constructs identified by youth program administrators as needing additional measurement development.Measures were selected that linked to the chosen conceptualizations and that were applicable for use in a variety of youth program settings.The research team also recognized that for the instruments to have broad applicability they needed to perform equally well across diverse populations.

Sample
Data were drawn from participants in a school district sponsored after school program in a large urban setting.The IRB at the researchers' university and the participating school district approved the research project as part of a larger outcome study of the after-school program.All 4th grade and older youth who were signed up for the after school program were included in the study sample.Participation was voluntary and occurred during fall 2010.In total, 2089 surveys were distributed and 758 completed surveys were returned for a 37% response rate.The refusal rate for completing the survey is unknown, since some children were absent on the day the survey was administered, some children from the compiled list had dropped out of the program, and some children only attended on certain days of the week, which might not have included the survey administration day for a particular school.However, reports from the program site coordinators indicated that most children in attendance on the day the survey was administered completed the survey.
Out of the 758 collected surveys, 270 were missing some data.Chi-square and one-way ANOVA tests were conducted to test for differences between those with complete and missing data across all mean measure scores, grade level, gender and ethnicity.The only statistical difference between the complete and missing data subsamples occurred on the decision making scale (F = 5.78, p = .02),with the complete group having a higher mean score.Therefore, the decision was made to calculate mean scores for all cases with at least a 90% completion rate on each scale.The newly calculated means produced a final dataset with 608 complete cases.Within this sample, 46.4% were Hispanic, 44.6% were African American, and 4.9% were White, and 3.1% were variously classified as other.Females comprised 54.4% of the sample and 38.2% of the sample were in elementary school (grades 4-6), 38.5% were in middle school (grades 7-8), and 23.4% were in high school (grades 9-12).

Instruments
Communication.The 23-item Communications Scale from the Youth Life Skills Evaluation (Barkman & Machtmes, 2002) was chosen for the study due to the acceptable survey length and adolescent focus of the items.The scale has been used extensively with 4-H youth populations and has produced an average reliability of .79(Perkins & Mincemoyer, 2009).
Community volunteering.A primary goal of community volunteering is to create youth who are more considerate of other's needs (individual and community) and therefore more altruistic.The Self-Report Altruism Scale (Rushton, 1981) was selected for this study.The 20-item scale measures altruistic behavior with questions that ask how often various altruistic actions are performed.The survey asks the subject to respond to each question on a scale of never, once, more than once, often or very often.The measure has been widely used for adult subjects and has a reliability rating of .84(Rushton, 1981).Some modifications were made to the original survey to make items applicable for children and adolescents.Some items were removed that deals with tasks many youth are too young to perform, for example driving a car.See Table 1 for a full overview of all item adaptations.

Table 1
Comparison of Original and Adapted Altruism Scale (Rushton, 1981) Original Items Adapted Items I have helped push a stranger's car out of the snow.
---I have given directions to a stranger.I would give directions to someone I did not know.
I have made change for a stranger.
---I have given money to a charity.I would give money to a charity.I have given money to a stranger who needed it (or asked me for it).
--- I have allowed someone to go ahead of me in a lineup (at Xerox machine, in the supermarket).
I would allow someone I did not know to go in front of me in line.
I have given a stranger a lift in my car.
---I have pointed out a clerk's error (in a bank, at the supermarket) in undercharging me for an item.
I would point out a cashier's error in undercharging me for an item.
I have let a neighbor whom I didn't know too well borrow an item of some value to me (e.g., a dish, tools, etc.).
I would let a neighbor I did not know well borrow an item of value to me.
I have bought 'charity' Christmas cards deliberately because I knew it was a good cause.
---I have helped a classmate who I did not know that well with a homework assignment when my knowledge was greater than his or hers.
I would help a classmate who I did not know well with a homework assignment when my knowledge was greater than his or hers.
I have before being asked, voluntarily looked after a neighbor's pets or children without being paid for it.
I would look after a neighbor's pet or children without being paid.
I have offered to help a handicapped or elderly stranger across a street.
I would offer to help a disabled or elderly person across the street.
I have offered my seat on a bus or train to a stranger who was standing.
I would offer my seat on a train or bus to someone who was standing.
I have helped an acquaintance to move households.I would help someone in my neighborhood move houses.
Critical thinking.The 20-item Critical Thinking in Everyday Life Scale (CTEL; Perkins & Mincemoyer, 2002) was chosen for this study.The scale assesses the following critical thinking sub-skills: reasoning, enquiry, analysis/information processing, flexibility, and evaluation.Previous use of the scale has produced acceptable (α = .75)reliability levels (Perkins & Mincemoyer, 2009).
Decision making.The Making Decisions in Everyday Life Scale (Mincemoyer & Perkins, 2003) was chosen as the most appropriate measurement tool for this project.The scale was designed specifically for adolescents and features 20 statements actions does during the decision making process.Extensive use of the scale with 4-H populations has produced an average reliability level of .74(Perkins & Mincemoyer, 2009).
Leadership.The six-item Leadership Efficacy Scale (Chi, Jastrzab, & Melchior, 2006) was selected for this study.It covers the elements of leadership without touching on the technical items such as the leadership styles.Previous applications of the scale have produced somewhat weak reliability statistics (Chi, et al., 2006).Reliability scores were lower for girls than for boys, .59 and .68,respectively.For African Americans the alpha reliability was only .49.
Problem solving.For this study the 24-item Solving Problems Survey (Barkman & Machtmes, 2002) was selected.Result of pilot testing with 4-H youth participants showed that the survey has good internal consistency; α = .86(Perkins & Mincemoyer, 2009).
Responsible citizenship.The 10-item Civic Responsibility Survey level 2 (Furco, Muller, & Ammon, 1998) was selected for the study.Previous use of the measure produced adequate levels of internal consistency (α = .84).
Self-esteem.The 10-item Rosenberg Self-esteem Scale (Rosenberg, 1965) was selected for this study.
Self-Responsibility.The American Camp Association's six-item Self-Responsibility measure (2007a) was used in this study.As noted with their construct definition of self-responsibility (Ellis & Sibthorp, 2006), this measure of responsibility is both succinct and can be used to assess the development of self-responsibility within a variety of settings.These attributes coupled with strong psychometrics (α = .84)makes this measure a strong selection for PYD project evaluative purposes.
Teamwork.The American Camp Association's eight-item Teamwork scale (2007a) was selected for use in this study.The scale has a 9 th grade reading level and is intended for use with older campers (i.e., ages 10 and up).Previous application of the survey has produced excellent levels of internal consistency (α > .9)and between item correlations were each above .5(American Camping Association, 2007b).

Data Collection
The 10 life skill measures were divided into two different survey versions in order to reduce the overall number of items each respondent had to complete.Version A contained measures of communication, altruism, decision making, responsible citizenship, and teamwork.Version B contained measures of critical thinking, problem solving, leadership, self-responsibility, and selfesteem.All survey items are provided in Appendix A. Item wording remained in its original form for each instrument except for previously discussed modifications to the altruism measure (Rushton, 1981).
In cooperation with the after school program administrators, survey packets were sent to afterschool programs at nine elementary schools, four middle schools, and eight high schools.
Program staff at each location were provided with a protocol script which they were to read to the students before inviting them to complete the survey.

Analysis Strategy
All measures were scored using a five point Likert scale with either never to always or strong disagree to strongly agree response formats.Mean scores were calculated for all measures.Internal consistencies were computed for each measure.Pearson's correlations were conducted to assess the relationship between the measures on survey A and B respectively.T-tests were conducted to look at gender and ethnicity differences across the measures and one-way ANOVA's were used to test for school level differences.

Descriptive Statistics
Table 2 provides a complete overview of descriptive statistics for all measures and groups.
Mean scores for all measures ranged from a low of 3.16 for altruism and a high of 3.89 for selfresponsibility.Psychometric Analyses Internal consistency.Reliability analyses were run on each scale including corrected itemtotal correlations and Cronbach's alphas.Items number two and five on the communication scale, item two on the critical thinking scale, and item nine on the problem solving scale, all returned item-total correlations under .2and were thus dropped from further analyses.
Table 3 lists the reliability for each scale.Reliabilities ranged from .71 to .94.All scales produced adequate levels of internal consistency when compared across gender, school level, and ethnicity.The only marked performance difference occurred with the self-esteem scale which produced Cronbach's alphas of .70 for elementary students and .83for high school students.Steps were also taken to compare the alphas from this study's data with previously reported alpha's (Table 4).For the most part, alpha's for the measures were similar across studies although there are some marked differences for the critical thinking, decision making, leadership and self-esteem scales.The critical thinking and decision making reliabilities were .11and .12respectively higher than reported levels from previous 4-H samples.The leadership alpha from the current study was .14 higher than results published by Chi et al. (2006) and selfesteem scores were .10lower than reported levels from a sample of university students (Robins, Hendin, & Trzesniewski, 2001).Critical thinking and decision making differences in reliability may be due to the fact that the 4-H samples included third graders whereas this study's sample started with 4 th graders.The self-esteem differences may be attributable to age as the previously reported sample consisted of university students.The reasons for the discrepancy in leadership scale alphas are not as apparent as the two samples appear fairly comparable.Group differences.Results from t tests indicated the no significant gender differences (see Table 6).Due to small numbers in some of the ethnicity groups the analysis was delimited to African Americans and Hispanics.T test results revealed significant differences for altruism, decision making, leadership, and problem solving.African Americans scored higher on altruism and decision making and Hispanics reported higher self-responsibility scores (see Table 6).One-way ANOVA results with Bonferroni comparisons indicated a number of differences across school levels (Table 7).

Discussion
This study highlights both the importance of life skills as a targeted outcome domain for many youth development programs and the breadth of approaches and conceptualizations that have been applied to life skill constructs.Findings suggest that although little agreement exists regarding which life skill domains are the most critical for youth programs, some consensus exists, at least among CYFAR Program Directors, regarding the most frequently targeted life skills.
The identification of 10 key life skill domains provided the research team with a roadmap for the other elements of the study.Attempts were made to select life skill definitions from the existing literature that were most aligned with PYD programs and the adolescent they served.Although the original plan included the possibility of construct new life skill measures where needed, it became apparent that ample measures existed and that the real issue was selecting and testing those that appeared most applicable to PYD programs.Hopefully the results from this study will help promote increased standardization of life skill conceptualization and measurement across youth development related fields and programs thereby addressing at least in part Catalano et al.'s (2002) call.
In terms of the testing and validation of the selected measures, it appears that each of the measures performed adequately with a diverse sample of youth when assessed by ethnicity, of gender, and school level.The findings represent an important contribution to the literature for many of the selected measures, especially those that had previously been employed with older (e.g., college students) or primarily homogenous (e.g., 4-H youth) samples (see Table 4).
Having respondents complete multiple life skill measures on the same survey also allowed the study to address the measures' convergent validity.The fact that all measures employed in this study showed mild to strong positive correlations provides evidence of concurrent validity.The high degree of positive correlation between long and short forms of communication, critical thinking, decision making and problem solving supports the efficacy of both forms of these measures.
It was interesting to note that school level produced the most significant score differences across measures.There were no gender differences and only three measures with ethnicity differences.For most the part, elementary students scored higher than either middle school or high school respondents except for communication where high school students had higher scores and self-esteem were middle school students had higher scores.

Limitations and Future Research
Although this study contributes to the psychometric validation of the selected measures in terms of their reliability and concurrent validity, further work is needed to test their discriminant and predictive validity.This would require the employment of the measures in studies employing measures of theoretically opposed scales (e.g., deviant behavior) as well as experimental designs that included measures of constructs theoretically justified as being linked to and potentially influenced by selected life skills.A greater understanding of the validity of key life skills would greatly enhance the efficacy of these measures in both research and practice.

Implications for Practice
Although life skills receive frequent programmatic attention in the field of out-of-school time programming and other youth focused intervention contexts, the abstract nature of many life skills makes assessment difficult.Part of the issue arises due to the variety of definitions applied to life skills.For example, although leadership may be an intentionally targeted outcome of multiple programs, the way each organization defines leadership could be quite different.Lack of conceptual uniformity then fosters the development of multiple assessment tools to measure a single life skill.For example, as part of the review process for this study we assessed 15 different potential teamwork measures.The myriad of measurement options not only makes it difficult for practitioners to identify measures, it also renders it nearly impossible to make comparisons across programs and interventions that share common outcomes but use different measures to assess impact.
The findings from this study will hopefully create common ground for researchers and practitioners around standardized definitions and measurement tools for the 10 life skill domains discussed in this paper.Increased conceptualization and assessment uniformity will both help improve the breadth and applicability of impact evaluations as well as provide a means to more efficiently identify best practices.Imagine the insights that could be gained if programs targeting similar outcomes could all agree on a set of standard measures.Lack of measurement standardization is one of the weaknesses in evaluation of youth development programs but hopefully this study and similar efforts can help facilitate the development and adoption of standardized assessments for commonly targeted program indicators.

Conclusions
The sequence and breadth of this study make a strong contribution to the life skills literature.It also highlights the fact that the conceptualization and measurement of abstract life skill domains remains a field in need of further attention.While it is generally agreed that life skills remain key targeted outcome domains for many youth development programs, the true power of life skills focused programs will not be obtained without standardization of definitions and measurement approaches.Hopefully this work can contribute to helping achieve this goal and promote further development of life skills research and practice.
have donated goods or clothes to a charity.I would donate clothes or goods to a charity.I have done volunteer work for a charity.---I have donated blood.---I have helped carry a stranger's belongings (books, parcels, etc.).I would help carry belongings of someone I did not know.I have delayed an elevator and held the door open for a stranger.I would hold the door open for someone I did not know.

Table 2
Descriptive Statistics

Table 4
Comparison of Cronbach's Alphas with Previous Research

Table 6
Gender and Ethnicity Differences

Table 7
School Level Differences