A Self-Assessment Approach to Understanding 4-H Professional Development Needs in the Northeast

In 2017, the National 4-H professional research, knowledge, and competencies (PRKC) taxonomy was updated reflecting the current needs of 4-H youth development professionals. Knowledge and skill gaps of 4-H professionals need to be identified to optimize resource allocation for professional development on the state and regional level. We developed a web-based self-assessment of the PRKC tool using Qualtrics, that evaluated the core competencies across the 6 PRKC domains: (a) youth development; (b) youth program development; (c) volunteerism; (d) equity, access, and opportunity; (e) partnerships; and (f) organizational systems. The responses from 188 Extension 4-H professionals (approximately a quarter of the 4-H Extension professionals in the Northeast region) were analyzed to (a) identify the knowledge and skills gaps in competencies within 6 domains, and (b) provide suggestions for professional development that would be of value to each state or region, based on the aggregate data. The results show respondents perceived their personal knowledge, skills, and competencies highest in the domains of access, equity, and opportunity and organizational systems. The domains with the lowest perceived personal effectiveness were youth program development and volunteerism. Not surprisingly, the aggregate results indicate that respondents with a greater number of years of experience in Extension reported higher competency in three domains (youth program development, volunteerism, and organizational systems). This tool can be utilized by Extension at any level to better understand the needs of the 4-H professional workforce. Results can aid the design of professional development opportunities to meet the knowledge and skill gaps identified among respondents. highest in AEO and organizational systems and lowest in youth program development and volunteerism. For state administrators, this study provides quantitative and qualitative information for maximizing the limited resources dedicated to professional development by implementing more intentional


Introduction
High quality 4-H programs are dependent on well-trained 4-H professionals. The knowledge, skills, experience, and expertise of the 4-H professional determines the quality of and methods in which programs are developed, managed, and evaluated (Borden, 2014). Determining professional development needs in an organization is always a challenging task. This is especially the case in Cooperative Extension when staff are geographically dispersed and have diverse job responsibilities. In addition, historically, 4-H professionals often had a disciplinary focus (e.g., animal science, home economics, or horticulture) and sometimes lacked foundational training in facilitating and teaching positive youth development and life skills (McDowell, 2001).
In 2004, a professional competency taxonomy was developed for 4-H youth professionals by the National Professional Development Task Force (Stone & Rennekemp, 2004). This taxonomy was updated in 2017 to reflect recent advancements in positive youth development competencies (National Institute of Food and Agriculture [NIFA], 2017a). The professional development competencies identified professional or specific skills or behaviors that are necessary for one to be an effective youth development professional, which were organized into six domains of research-based knowledge known as the professional research, knowledge, and competencies taxonomy (PRKC) that include (a) youth development; (b) youth program development; (c) volunteerism; (d) access, equity, and opportunity (AEO); (e) partnerships; and (f) organizational systems.
A self-assessment tool based on the PRKC was developed as part of the original taxonomy and was revised in 2017 (NIFA, 2017b). For this project, a web-based self-assessment tool based on the six domains of the PRKC was created using Qualtrics. This tool is intended to familiarize youth development professionals with the knowledge and competency areas and to help them identify areas in which they may wish to focus their own professional development.
A loud refrain for investment in professional development can be heard across the 4-H system. Borden et al. (2014) called the 4-H program to action, specifically highlighting the need for professional development of 4-H professionals that is focused on outcomes and program quality. The development and adoption of the 4-H Thriving Model has reinvigorated training efforts around foundational positive youth development principles (Arnold & Gagnon, 2020).
More recently, amid the pandemic, Arnold (2020) made an appeal for investment in positive youth development and specifically an investment in training and support of 4-H professionals as they turn to a whole-child approach. Fostering and sustaining developmental relationships with youth is a key feature of positive youth development and an aspect of youth development practice that should be emphasized right now as we face significant mental health challenges among youth and teens (Arnold & Rennekamp, 2020).

Project Framework
The objective of this project was to identify 4-H youth development professionals' strengths, and areas in need of improvement related to the PRKC framework. Utilizing the updated National 4-H PRKC self-assessment questions, we developed a web-based self-assessment tool with automated scoring that included additional open-ended questions. This web-based tool enhanced the functionality and useability of the paper-based tool. The project collected data across the Northeast region and utilized the 4-H PRKC in a novel way that can inform the investment of professional development resources at the state and regional level. This project has the potential to strengthen the Extension workforce throughout the region and positively improve the outcomes and impacts of programming reaching hundreds of thousands of youths engaged in 4-H programs throughout the Northeast.
The PRKC self-assessment tool was not intended to serve as a research instrument but rather as a way to assess staff development training needs for 4-H Extension professionals. Some states, such as Virginia (Garst et al., 2007) and California (Heck et al., 2009), used the original PRKC at the state level to inform professional development plans. The Professional Development Committee of the National Association of Extension 4-H Youth Development Professionals (NAE4-HYDP) completed a national assessment of 4-H professionals using the PRKC to determine perceived ability versus perceived importance for each competency item (M. Benge, personal communication, April 13, 2022). This project, which was completed in the fall of 2021, complements the findings of this regional study.

Methodology
In the summer of 2020, 4-H youth development professionals from the Northeast region were invited to participate in this project, which was reviewed by the Institutional Review Board (non-human subject determination).

Participants
Northeast state 4-H program leaders (NESPL) were provided an email template and asked to share it with all Extension professionals with a youth development role. With a completed response of n = 188, this project had an estimated reach of 25% of 4-H Extension professionals in the Northeast region.

Tool
This first phase of this project asked participants to complete a 10-minute PRKC selfassessment tool. The project team added additional demographic questions (Land Grant University (LGU), job role or title, number of years in youth development, number of years in Extension), open-ended feedback (suggestions for professional development topics, resources, ideas, and concerns) and put it in an online Qualtrics form. This was the first time the PRKC self-assessment tool had been implemented regionally to help inform professional development in the Northeast. The web-based PRKC self-assessment tool includes 10 to 20 questions in each of the six domain areas on which respondents are asked to rate their own proficiency using the 5-point Likert "Reflect Me" scale: untrue of me, somewhat untrue of me, neutral, somewhat true of me, true of me (Vagias, 2006). The possible total score range is from 11 to 55 for youth development, 17 to 85 for youth program development, 11 to 55 for volunteerism, 14 to 70 for AEO, 12 to 60 for partnerships, and 20 to 100 for organizational systems. Cronbach's alpha, a measure of reliability/internal consistency, was computed for each domain. The psychometric properties of each of the six domains were shown to be internally consistent, as evidenced by the following alpha coefficients: youth development (alpha = 0.828), youth program development (alpha = 0.915), volunteerism (alpha = 0.899), AEO (alpha = 0.892), partnerships (alpha = 0.898), and organizational systems (alpha = 0.865).

Procedure
When the self-assessment tool was completed, participants received a score in each domain, based on a possible total of 100%. Participants were invited to use their self-assessment data to complete a personal professional development plan. At the aggregate level, a detailed regional report was provided to state 4-H program leaders and Extension directors in the Northeast. In addition to the quantitative data summarized below, qualitative feedback from participants with professional development suggestions for their state was collected and coded.
Qualitative feedback was manually coded by the project team into major themes that emerged: volunteer recruitment, volunteer management/training/support, partnerships and collaborations, grant writing, fundraising, mental health and social-emotional well-being, program development, positive youth development (PYD), 4-H youth engagement, reach, programming, content area trainings, technology tools, virtual programming engagement, diversity, access, inclusion, social media/marketing, cultural competency, work/life, adult family engagement, and other. The topics identified by respondents cut across all domains and most competencies. You can find a copy of the self-assessment tool and a detailed report for this project: https://nj4h.rutgers.edu/4hprkc/.

Results
In all, 188 respondents representing twelve land grant institutions in the Northeast region completed the PRKC self-assessment tool. Over 80% of respondents identified as full-time faculty (FTF) or full-time staff (FTS) and the majority (over 71%) of respondents reported that more than 75% of their effort is focused on 4-H youth development programming versus other Extension program areas ( Figure 1). Respondents self-identified their role within the 4-H program but due to the variability of faculty and staff positions throughout the region grouping participants by role would not have been appropriate. Respondents reported a wide range of service years in both Cooperative Extension (from less than 1 year to 43 years) and in youth development prior to Cooperative Extension (from less than 1 year to 33 years). Table 1 includes demographics for those who completed the PRKC self-assessment.

Competency Scores by Domain
When reviewing the aggregate scores in each PRKC domain we found that respondents perceive their personal knowledge, skills, and competencies highest in the domains of access,  Table 2). To standardize total domain scores, we divided by total possible score in each domain to provide a percentage out of 100.

Correlations
Pearson chi-square tests revealed a statistically significant (alpha = .05) association between "competency" and "years in Cooperative Extension (CE)" for youth program development, volunteerism, organizational systems as well as overall competency score. To illustrate this relationship, years in Cooperative Extension and competency scores were transformed into categorical data for frequency analysis. Table 3 shows results for overall competency scores related to years in CE; the full report offers a breakdown by competency Extension the higher they scored in the domains of youth program development, volunteerism, and organizational systems. For the three above-mentioned competencies, comparing the marginal percentages (i.e., total row percentages) to the percentages for each "years in CE" category shows that respondents with higher competency scores are disproportionately distributed into higher "years in CE" categories. Conversely, a disproportionately lower number of respondents in lower "years in CE" categories scored high in those three domains. For example, of the 153 respondents whose competency score was 80% or higher for organizational systems, 90 (58%) indicated they had been in Cooperative Extension for 10 or more years while 63 (41%) indicated less than 10 years of experience in Cooperative Extension.
While not surprising, the correlation between competencies and years in Extension underscores the importance of employee retention. Interestingly, there were no statistically significant relationships found between prior years in youth development and competency scores. Note. Chi-square (9 df) = 17, p = .049

Competency Level Review
In this article, we focus on the areas where we found the greatest knowledge gaps. Looking at the two lowest-scoring domains, areas with the greatest opportunity for improvement, youth program development (competency score 80%) and volunteerism (competency score 80%) we can focus on the lowest-rated responses for individual skills and behaviors within those domains. Combining the two lowest response categories, untrue of me and somewhat untrue of me, we can see which skills and behaviors were rated lowest by respondents. For the youth program development domain, we found that respondents rated themselves lowest for the following two skills and behaviors: "understands what theories of action and change are and can apply those theories to youth program development" (20.8%) and "understands evaluation protocols for collecting and handling data and knows when to seek approval from the IRB process if appropriate" (25% For the volunteerism domain, we found that respondents rated themselves lowest for the following two skills and behaviors: "understands and implements multiple recruitment strategies based on varying volunteer roles and community demographics" (18.6%) and "develops and conducts impact assessments of volunteer efforts and communicates impact value to stakeholders" (24.5%). Most other skills and behaviors were rated considerably higher by respondents with the highest being "believes in the competence of volunteers and understand the role and value of volunteers in our organization" (97.3%) and "communicates the value of volunteerism both within and outside the organization" (90.4%). In review of the qualitative data participants suggested "volunteer recruitment -including identification of new volunteers, strategies, sustainability, etc." as valuable professional development topics when responding to the question "What kind of PD topics do you think would be valuable for your whole state?".
Further review of the qualitative data found 31 participants suggested "volunteer management/training/support -including communication, engagement, recognition, etc." as valuable professional development topics. Table 5 illustrates the distribution of responses for individual skills and behaviors within the domain Volunteerism.

Discussion
We have found benefits to utilizing the PRKC self-assessment in various ways. Youth development professionals can use the self-assessment tool to see their strengths and weaknesses and help guide them to addressing their professional development needs. This is especially helpful for new youth development professionals who are in the orientation and onboarding processes.
On a statewide level, program leaders can use state-level data to help drive professional development plans. The quantitative data generated by the PRKC self-assessment provides more detail for state leaders than the original PRKC tool. The Qualtrics format also allows for seamless implementation across a state. Moreover, state level data can inform the design of position descriptions and prioritize the recruitment of candidates for 4-H professional positions with specific skill sets. One northeast state used its state-level volunteerism data to advocate for additional staff support with that expertise. Having the data to back up the anecdotal need for additional volunteer management capacity was compelling and aided Extension leadership in deciding to support funding a position with a volunteerism focus. On a regional level, program leaders can work together to collaborate on professional development plans that meet the needs of many, also pooling resources among the group. The development or expansion of regional training programs may be a cost-efficient way to improve the preparedness of staff. Borden et al. (2014) called for the development of additional professional products specific to PRKC domains, specifically to assure quality 4-H program delivery. System-wide needs, like this example, are more likely to be effectively met though collaborative efforts at the regional or even national level.
On a national level, efforts to develop and adopt the 4-H Thriving Model will increase professional development opportunities around the positive youth development domain (Arnold & Gagnon, 2020). There is also the opportunity to create an online clearing house of resources that can be accessed by many, which will be very helpful to states that lack the staffing and/or resources to provide the professional development needed.
We have also found that those in their role as youth development professionals for longer periods of time rate themselves as more competent in several areas including youth program development, volunteerism, and organizational systems. This could help program leaders identify experienced professionals who can mentor professionals who are newer to their role and help them identify professional development opportunities.

Limitations of this study
Being the first of this type of study using the PRKC self-assessment tool, we note limitations related to research design and implementation. The following discusses limitations in the study including the self-reported data, assessment tool distribution, situational bias, and the use of the PRKC tool.

Self-Reported Data
There are several important considerations when asking participants to self-report including possible memory limitations (selective memory and telescoping), exaggerated responses, and social desirability bias. Social desirability bias is the need for respondents to select the socially desirable responses, those they perceive will make them look good (Donaldson & Grant-Vallone, 2002;Judd et al., 1991). Moreover, as a self-assessment of competency in youth development, the PRKC assessment tool may be limited in terms of accuracy. Self-ratings could be less accurate than an objective measure by others. Overall self-ratings tend to overestimate ability, and less competent individuals may be more inaccurate at self-report than more competent individuals (Kruger & Dunning, 1999).

Self-Assessment Tool Distribution
The competencies where they scored the lowest, but they also should consider the competencies where they perceive the most need specific to their roles. This study developed a way to make the PRKC self-assessment easier to use by creating a web-based tool with automated scoring.
While maintaining the integrity of the framework this study also collected demographic data and offered participants the opportunity to share additional feedback via open-ended questions. In various discussions while presenting this data, challenges with the tool emerged. Some indicated the tool was too intricate and others expressed a concern that the listed competencies were rather shallow, making it difficult to really gauge one's competencies in a particular domain.

Situational Bias
Finally, we anticipate that some of the overall results related to gaps in skills may look different depending on the timing that the tool was distributed. This self-assessment research project launched in the summer of 2020. This was in the heart of the COVID-19 pandemic lockdown, and a time of racial reckoning (Chang et al., 2020) which may have shifted individuals' perceived sufficiency or insufficiency of skills related to the PRKC. Reflection in self-reported competencies may be influenced by trends including experience in the field and current events.

Future Directions
In further discussion with the project team and Northeast state program leaders, next steps for how these findings can be used for individuals, state program leaders, and regional leaders were identified. For individuals, the digitized PRKC self-assessment tool improves accessibility and provides immediate self-scoring which allows respondents to quickly identify their needs  Astroth and Lindstrom (2008). After presenting the results of this study to the Northeast state 4-H program leaders the conversation spurred the development of a regional professional development program (Clover Academy) geared toward new 4-H professionals that began in 2021. Some of the results influenced the development of objectives set. These results support the need to invest in quality system-wide professional development training that all can access.
This also supports national work toward making these types of opportunities more accessible to all.
Professional development resources suggested in the open-ended questions in this tool will be shared with the Professional Development Working Group (chartered under the 4-H Program Leaders Working Group (PLWG)). These resources can be used by the Professional Development Working Group to support their web-based project of identifying professional development resources to support each PRKC domain. The web-based PRKC self-assessment tool that was created will also be shared nationwide.

Conclusion
By reviewing the PRKC, it is clear that youth development professionals need to draw on a wide variety of skill sets. Professional development practices, which vary significantly across youthserving organizations (Garst et al., 2014), have implications for improve program quality and effectiveness that leverages the limited resources available. Use of the self-assessment tool to measure ongoing professional growth and changes in professional development throughout one's career would provide additional feedback for professionals. The PRKC self-assessment is a simple and accessible tool that can provide valuable insights that could improve the quality and effectiveness of 4-H programs at the local, state, and regional levels by informing targeted professional development.