Acquisition of Knowledge and Skills in an Online Graduate Degree in Youth Development Leadership

As the youth development field has grown, there has been an increased focus on building academic preparation programs in youth development that prepare leaders and staff in youth-serving organizations. Very few degree programs exist in preparing youth leaders. Even fewer, if any, evaluations have been conducted on the potential outcomes and benefits of graduate degree programs on alumni who serve in leadership positions in youth organizations. This study presents findings from a survey of graduates of a master’s level youth development leadership (YDL) degree program. The findings focus on graduates’ perceptions of the outcomes linked to attaining a degree from such a program and associated competencies and opportunities in the field of youth development. Questionnaires were distributed to YDL alumni (2007-2018) requesting their perspectives on professional benefits and opportunities associated with degree completion. Respondents reported a statistically significant increase (reaching medium to large effect sizes) in relevant competencies targeted in each course. Additionally, respondents noted opportunities in salary increases, promotions, and more challenging responsibilities because of completion of the YDL program; rated their level of application of learning to new professional skills applied to their professional positions highly; and identified products created in their professional positions directly related to their academic preparation. A discussion of the value of youth development degree programs is provided.


Introduction
The youth development field would benefit from academically trained professionals who could serve as leaders in building impactful youth-serving organization to promote youth well-being and preparation for a successful adulthood (Borden et al., 2020). For example, youth leaders who were trained in domains such as positive youth development theories, best practices, staff and volunteer teamwork, and funding streams via grant writing and community collaborations report delivering youth experiences and opportunities in communities that would bolster youth potential and life satisfaction (Garst et al., 2019). The field of youth development would grow more rapidly with the existence of academic and certificate programs that could train youth development professionals more effectively in organizations than what currently exists (Mahoney & Warner, 2014). There are, however, very few degree and certificate programs explicitly focused on preparing youth leaders. There are even fewer evaluations conducted on these degree and certificate programs that can be found in the empirical literature (Curry et al., 2013;Quinn et al., 2020;Stein et al., 2005). Therefore, the purpose of this article is to report the outcomes, including benefits to youth leaders, linked to participating in a graduate level youth development degree program from the perspective of alumni of the program. As the graduates were working professionals both during and after their time in the program, this study provides an opportunity to examine if and how the degree program influenced their professional goals and achievements and contributes to the scant literature on the effects of degree programs on youth-serving professionals.

Youth Development Work Force
Youth programs are intentional prosocial initiatives, structures, and activities that engage youth with their peers, schools, and community organizations. These programs build on youth strengths and assets, and fostering relationships and leadership abilities (Dymnicki et al., 2016).
Significant growth of the youth development field is dependent on the expansion of academic degree-granting programs (Borden et al., 2011;Quinn et al., 2020). Academically trained professionals who become youth development leaders could contribute significantly to the challenges that face the field of youth development. In fact, given uneven and unpredictable funding and volunteer support for youth development programs currently and in the foreseeable future (Arnold & Rennekamp, 2020;Ettekal & Agans, 2020), changing demographics across the United States that may impact the cultural-relevancy of programs (Simpkins et al., 2017), and the current and future strain on local and state budgets due to significant reductions in tax bases, youth program efficiency and effectiveness may increasingly rely on youth leader and staff competence to create strategic recruitment plans, identify funding Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 5 DOI 10.5195/jyd.2021.1098 Acquisition of Knowledge and Skills 233 sources, and develop best practice programs. There are at least three broad-based challenges for youth development organizations as proposed by Quinn et al. (2020): (a) poor attendance in out-of-school time (OST) settings, which represent a high percentage of youth program settings; (b) increasing need for developing partnerships to garner resources, community buyin, adequate youth participation, and consensus on community mission in creating a healthy youth environment; and (c) creating and implementing culturally responsive organized youth activities that traverse multiple youth settings. Youth settings that reflect diverse participation are increasingly expected to meet community, organizational, state, or federal standards (Simpkins et al., 2017).
The end goal of youth development workforce activities is to prepare professionals to deliver youth programs that foster healthy development, psychological well-being, and substantive preparation for a successful transition to adulthood to the young people whom they serve.
Youth leaders trained in youth development competencies enhance the probability that youth will experience such outcomes within and across communities, states, region, and the nation.
When developing a quality workforce of youth development leaders, it is critical to clearly define the competencies (i.e., the knowledge, skills, and perspectives) necessary to achieve the above noted outcomes. Core competency models identify what youth-serving professionals need to deliver high quality programs that contribute to positive outcomes in youth (Starr et al., 2009).
For example, the National Afterschool Association (2011) developed a framework detailing 10 core competencies for after-school and youth development professionals. These competencies include child and youth growth and development; learning environment and curriculum; child/youth observation and assessment; interactions with children and youth; youth engagement; cultural competency and responsiveness; family, school, and community relationships; safety and wellness; program planning and development; and professional development and leadership. Youth-serving organizations have increased their emphasis on knowledge and skills of staff to enhance the quality of youth programs that they sponsor (McNamara et al., 2020). For instance, Radhakrishna and Ewing (2011) explored the relationship between youth development leaders' competencies and youth outcomes. They found statistically significant relationships among seven adult leader competencies and youth skills. Smith and Soule (2016) discuss the importance of training in cultural competence to promote mentoring, and Simpkins and colleagues (2017) offer guidance on designing culturally responsive programming for participants in after-school programs. However, organizations often face challenges in providing professional development opportunities to the youth workforce. For example, youth leaders do not uniformly identify as "youth workers" and funding support often does not specify the qualifications needed by youth organizational staff to execute the program  (Borden et al., 2020;Borden et al., 2004). Therefore, the best approaches to enhancing staff knowledge and skills is uncertain. Helitzer et al. (2010) and Millar et al. (2001) urge the field to build and draw on logic models which require professional training and focus on identifying the unique work of youth development professionals to prepare and support them as either a driver or complement to professionals trained in one of the helping professions (Borden et al., 2020). Additionally, Borden and Perkins (2006) found in their investigation that the varying availability of educational systems in youth development to provide community professionals with the knowledge and skills essential was central to successfully accomplishing work tasks generating youth impact. Benefits and consequences of a formal educational degree-seeking program in youth development leadership have been reported (Garst et al., 2021). As these studies suggest, a number of challenges arise when providing educational opportunities to youth development leaders representing youth-serving organizations with varying purposes, goals, and structures.

Academic Degree Programs
Academic programs that represent youth development principles and practices have demonstrated a pathway to preparing a youth development work force (Curry et al., 2013;Garst et al., 2019;Quinn et al., 2020;Stein et al., 2005). Their utility in preparing a youth development work force is seen in the multiple approaches to delivering youth development content. For example, as far back as 1997 a youth development worker certificate program was launched requiring 12 hours of college credit in collaboration with Youth Net to provide staff with expertise to teach youth development principles and practices (Borden & Perkins, 2006). Shockley and Thompson (2012) review a 12-credit youth studies certificate supported by employers to deliver a relevant curriculum to prepare front-line workers.
Over time, interest in academic programs evolved into the development of online degree programs. Online programs have grown to reach working professionals who have a desire for professional development, but who are not geographically mobile or whose life circumstances preclude them from relocating to a college or university setting. Youth leaders who participate in such programs report benefits of clarifying their identity as youth development professionals. Robideau and Santl (2020) report the value of online learning for youth development professionals in a cohort learning structure including gaining familiarity with research, relevance of content to their work environments, and the value of being a part of a learning community.
The Great Plains consortium comprised of multiple human science colleges at 10 universities Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 5 DOI 10.5195/jyd.2021.1098 Acquisition of Knowledge and Skills 235 provides opportunities for students to learn youth development principles and apply their skills locally via practicum experiences to merge both national and local contexts to further grow the youth development field. A key step in developing these academic programs is to align these professional opportunities for youth leaders to existing competency models (Garst et al., 2007;Vance, 2010). Several university-based degree programs have developed to link these paradigms (e.g., Mahoney & Warner, 2014) including the graduate degree program that is the focus of the present study. Previous articles have provided an overview, detailed description, and student learning experiences and benefits of this graduate program (Garst et al., 2016;Garst et al., 2019;Quinn et al., 2020). However, these prior studies as well as prior research in the area of youth workforce development have not explored how graduates applied specific skills linked to courses to their work, nor has this prior research identified the opportunities and products linked to engaging in a youth development graduate degree program in a comprehensive way.
Thus, the purpose of this study was to examine student outcomes in a graduate level degree program in youth development leadership pertaining to both the impact of course completion as reported by graduates following degree completion as well as the relevance or utility of the degree program in its application over time in their chosen youth development positions. Data for this study were analyzed to assess (a) the relative benefits of each course offered in the curriculum pertaining to acquisition of knowledge related to theoretical and applied aspects of youth development leadership (e.g., foundational theories of youth development, youth program design and evaluation, staff and volunteer development, grant writing, and leadership principles and practices) and (b) the opportunities to apply the knowledge gained in the degree program to their work tasks and responsibilities.
The research questions posed in this study are as follows: 1. How much, if any, knowledge acquisition on the ten benchmarks (see National After School Association Core Competencies, 2011) of youth development was achieved from the perspective of graduates of a Master of Science program in Youth Development Leadership?
2. What skills learned in the degree program were applied to their professional youth work activities?
3. What new professional opportunities were offered or available to you as a result of obtaining this degree? 4. What deliverables/products were produced for respondents' youth organizations a result of capacities gained in the degree program?

Method Study Site
This 2019 study examined student outcomes generated from a graduate degree program in youth development at a southeastern university through an online questionnaire distributed to all student cohorts beginning in 2007 (i.e., the year of the program's inception).

Participants and Data Collection
Alumni of the YDL program were recruited through electronic requests: an initial email explaining the purpose of the study, and which included a link to the online questionnaire and a follow-up email 6 weeks following the initial email. Recruitment strategies incorporated the departmental email list as well as other directory sources like the university alumni relations list, YDL Facebook page contacts, and contacts from the program coordinator who remained in contact with students after they obtained their degrees. Of the 106 graduates who were recruited, 71 questionnaires were completed for a response rate of 66.9%. Of these 71 responses, 56 were completed with sufficient information to be included in the analyses. This study was approved by the Institutional Review Board of Clemson University.
The demographic data (see Table 1) provide an overview of respondent characteristics. Of the 56 respondents, 42 (75%) were female and 14 (25%) were male. The majority of respondents were White (76.8%). Respondents were grouped into four age ranges (25-34, 35-44, 45-54, and 55-68); the age distribution suggests that many respondents completed the degree several years prior to the survey.  Table 3.

Professional Opportunities After Degree Completion
Study participants were asked what opportunities became available to them as a result of completion the YDL program. Respondents could select more than one option from among statements that described improvements in their pay, position level, or responsibility level; leaving a position for one they deemed better or to start their own youth-serving organization; and no change in opportunities or position. Responses are summarized in Table 4 in the Results section.

Youth Development Products/Deliverables Resulting From YDL Degree Program
An open-ended question, (i.e., "Identify 3-5 products you have developed as a result of your experience in the YDL MS degree program.") was used to explore the frequency and range of products or deliverables that respondents perceived they had undertaken and completed as a

Data Analysis
Quantitative data were analyzed using Qualtrics. For the question, "Rate your level of knowledge of content pertaining to each course in the curriculum before enrolling in this course and after completing this course," mean differences were analyzed using paired t-tests for graduates/alumni (Garst et al., 2019;Lee et al., 2011). After the initial data analysis process was completed independently by one team member, another team member external to the data collection process completed an audit of the qualitative analysis by comparing the original data, constructed codes and categories, and interpretations (Shenton, 2004) to provide data validation.

Results
To test the hypothesis whether level of competencies before and after joining the YDL program differed, a set of dependent sample t-tests was performed on the 10 competencies. To account for the multiple comparisons conducted, we applied a Bonferroni correction to control for a family-wise error rate of α = .05. An adjusted alpha level of .005 (.05/10) was used to evaluate the significance of differences. As shown in Table 2, the null hypothesis of equal levels of competency means was rejected for all 10 areas. These results suggest that there was a significant enhancement of knowledge in youth development content areas as viewed by graduates. Because these courses are built upon benchmarks in the youth development field pertaining to qualifications either informal or as part of a credential in the field (NAA Core Competencies), results suggest that a graduate degree program offers substantial professional development in staff and leadership positions in youth-serving organizations. Also, to test how large the effect of the YDL M.S. degree was for each area Cohen's d was estimated and the results revealed a large effect (Cohen, 1992) for the impact of course content on the respondent's acquisition of knowledge (see Table 2). This finding suggests that the respondents viewed each course as valuable in advancing their knowledge on the topics covered in each course.  Table 3 shows the mean levels of demonstration of targeted YDL program skills in youth practice based on respondents' ratings. Overall, the completion of a graduate program in youth development increased graduates' competencies in areas targeted by the YDL degree program. Note. Participants rated the frequency of use of each skill from 1 (not at all) to 4 (a lot).
a The e-portfolio was added to the curriculum in 2014 Respondents were asked what opportunities became available as a result of obtaining their YDL degree. YDL graduates could select more than one option among seven choices. Approximately one fourth (25.4%) of respondents reported a salary or pay increase from their current employer (see Table 4). The next category of opportunities selected by graduates as a result of the degree was "I left my employer and obtained a higher-level position" (16.9%), followed by "I left my current employer and obtained a higher paying position" (14.1%). There were some respondents who reported "nothing has changed for me" (16.9%). Overall, the majority of graduates reported a salary increased and/or new professional opportunities as a result of the degree. Graduates were asked to provide information, through an open-ended question, on organizational products or deliverables that resulted from knowledge or skills gained in the YDL program (see Table 5). A total of 19 categories emerged from the data. Among these, four categories of products/deliverables were ranked highest: grant and fundraising proposals and materials, assessment and evaluation reports or measures, program curriculum (creating and/or updating), and employee or volunteer training plans or programs.  Entrepreneurship (development of a consulting business/community foundation) 3 Stakeholder engagement plans/programs 3

SWOT analysis 2
Interactive workshops (respond to job offer, interview prep) 2 Program logic model 2

Student intervention and graduation plans 2
Research artifacts (master's research project, academic journal article) 2 Board of directors' development plan 1 Policies and procedures manual 1 Development and facilitation of child/adolescent therapy activities 1

Discussion
Identifying effective ways to prepare and professionalize the youth development workforce is critical to enhancing the quality of youth programs and promoting the well-being of youth.

Results based on this sample of graduate of Master of Science degree in Youth Development
Leadership program suggests that a curriculum based on principles of youth development can deliver knowledge congruent with these principles that prepare professionals to lead youthserving organizations. Graduates reported a substantial gain in knowledge of theoretical principles that drive decision making, youth program implementation and evaluation principles, and business-related skill sets such as in leadership and staff development. Furthermore, the curriculum directly influenced awareness of the necessity of planning, building, and growing youth programs that address diverse youth populations that are culturally relevant to youth necessary to acknowledge changing demographics and societal expectations. Furthermore, the respondents viewed the curriculum as a vital source of concepts and ideas that prepare them to incorporate effective business practices to build and grow a youth-serving organization.
The study findings offer substantial evidence that a graduate program in youth development leadership provides multiple benefits to youth leaders. The findings indicate that graduates utilize YDL content in multiple ways in their work environments. Graduates reported that participating in the degree program was linked to considerable increases in their knowledge of core competencies of youth development; respondents reported increases that would be considered large effect sizes on nine of the 10 competencies. The only competency that would be considered a medium effect size (d = 0.64) was "interactions with children and youth"; this relatively smaller, but significant, effect may be related to the degree program almost exclusively enrolling students who have been engaged with youth to a substantial degree.
These results also point to an implication in interpreting these large effect sizes that many current youth leaders do not possess the skills and competencies that would benefit them and their organizations in serving youth effectively; however, additional work that includes the perspectives of multiple reporters such as students' colleagues and supervisors as well as the youth and families they serve, would help to clarify student competencies at the onset of their engagement in the degree program. This uncertainty also points to the need for the establishment of benchmarks for youth leader competencies.
Because the YDL program curriculum is built upon benchmarks established by the National Afterschool Association (NAA), it is reasonable to assume that graduates are well prepared to be successful in multiple ways in youth development out-of-school time working contexts. As such, benefits are derived by youth-serving organizations in drawing on the work contributions and effectiveness as youth leaders. Graduate programs in YDL may contribute greatly to a more competent work force, which in the end contributes significantly to the lives and well-being of youth. This study further illuminates how organizations may benefit from employees who graduate from a YDL graduate degree program. It is very likely that such opportunities for alumni result in strengthening their youth-serving organizations in multiple ways including major organizational functions such as fundraising, program evaluation, staff and volunteer development, and program development. Such job assignments will preserve and strengthen youth organizations to be more impactful with youth in their communities.
These results suggest that alumni of a graduate program in YDL develop competencies and skills that provide preparation and opportunity to contribute to strengthening their youth organizations. Since the courses are designed based on the core competencies in OST professional development (see NAA, 2011), results suggest that this degree program strengthens the OST work force in programs that are growing to meet the needs of youth in communities. Furthermore, such relevant organizational contributions may bolster and advance the field of youth development and provide youth environments that are increasingly beneficial to fostering positive youth outcomes, enhancing youth well-being, and preparing more youth for a successful adulthood. Consistent with prior research (Garst et al., 2016;Garst et al., 2019;Quinn et al., 2020), a great majority of the graduates (83.1%) reported diverse opportunities associated with the completion of the YDL Master of Science degree program including salary increases and promotions, and a few graduates even started their own youth-serving organizations. In any academic degree program in which a cohort enrolls at a given time, however, the power of a curriculum will have differential impact on students based on many variables including maturity, work experience, professional goals, life circumstances, and quality of academic preparation. It is likely that one or more students may have reported less impact than other cohort members based on events or circumstances subsequent to attaining the degree. Several reasons may explain why a few YDL graduates reported that minimal changes occurred for them or their professional life as a result of the degree. For instance, they may be working in a position defined in a way that does not accommodate the content or skills acquired in the YDL program.
The complexity of individual experiences within the YDL program is reflected in the somewhat counterintuitive findings when comparing the reported frequencies by which course-specific skills demonstrated by graduates and the products and deliverables these same graduates report creating. For example, writing grant proposals, conducting evaluations, and developing training programs were less frequently demonstrated by graduates in their day-to-day youth development work; however, these same products were the most frequently identified as "products" created through completion of the YDL degree program through the lens of the respondents' career to date. This inconsistency may make some sense as these products are more long-term or irregular tasks within the life of a youth-serving professionals as compared to considering the diversity of youth when planning a program or applying a principle of child and adolescent development to address issues faced; however, these products might also reflect the outcomes associated with being a youth development leader within one's organization. Future work should explore how youth-serving professionals define the competencies and activities that define a youth leader versus a youth development leader.

Limitations
A few study limitations are acknowledged. First, the program graduates in the current sample were enrolled over multiple years. While the results suggest substantial learning and application of youth development content to their work in youth environments, it remains unclear the extent of the cumulative impact over time. It may be that respondents who graduated many years prior to the survey have had more opportunities than more recent graduates to apply those skills. This study did not examine the comparison between those graduates who had been in youth-serving organizations longer versus a briefer time since graduation. Second, data derived in this study are solely based on self-report responses of graduates of a youth development program, and thus are subject to response bias. Therefore, corroborating data from representatives of the youth-serving organizations of which these respondents are affiliated, or youth themselves, is not available to determine the relative benefits of graduate skill development in youth development to the organization or youth themselves. Third, measures used, while constructed to answer questions of interest to examine the goals of the YDL master's degree program, may not have been central to the nature of the work assignments of respondents in the study.

Conclusion
This article supports the value of graduate academic degree programs for youth-serving professionals. Graduate degree programs of this kind may be one of the most vital opportunities available in the youth development field currently, as many youth-serving programs are illequipped with staff who have knowledge of youth development theories, fundamentals of best practice programs, youth program assessment skills, grant writing, or a grasp of leadership theories and strategies. The open-ended question in the survey requesting respondents identify any deliverables or products that they would ascribe to their degree program experience suggests that grant writing, youth programming and evaluation, and staff training were highlevel skills useful to the youth-serving organizations in which they were affiliated. The study provides empirical support documenting that a curriculum of courses for a graduate level program in youth development leadership strengthens the youth development work force by potentially enabling graduates to impact their organizations in practical, meaningful ways. As a result, we can surmise that youth are the ultimate beneficiaries of a graduate degree program in youth development leadership. Future research should explore the benefits to youth and families served by youth-serving professionals engaged in youth development graduate degree programs.