4-H Volunteer Attainment of Quality Positive Youth Development Practices

The University of Florida 4-H program evaluated volunteer perceptions of behavior adoption resulting from the completion of training focused on best practices that promote positive youth development. The evaluation effort was a quasi-experimental design that surveyed all enrolled volunteers electronically. The survey was designed to determine the types and frequency of training that 4-H volunteers participated in, their satisfaction with the training, the volunteer’s role of service, and outcomes of adopting best practices of program quality. A response rate of 27% (n = 340) provided a representative sample of the enrolled volunteers across demographic categories and geographical regions of the state. The results Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 15 Issue 4 DOI 10.5195/jyd.2020.882 Volunteer Positive Youth Development Practices 88 show that volunteers report practicing elements of positive youth development as categorized by the Essential Elements framework and that correlations between the type of training volunteers participate in and their implementation of quality practices are significant. Implications for volunteer trainings include the consideration of frequency, expanding opportunities beyond traditional delivery, and incorporating training elements that promote high-quality youth programming. This evaluation effort provides baseline data that informs future research for the Florida 4-H program.


Situation
As efforts to evaluate Positive Youth Development (PYD) programs increase, one emerging thought is to focus evaluative efforts on program quality (Arnold & Cater, 2011). A significant element that impacts the quality of youth programming is the adoption of best practices by volunteers who directly work with youth audiences. The focus on indicators of program quality allows evaluators to understand the elements of programming that promote the end goal of developing young people (Arnold, 2015). Measuring the degree to which volunteers are implementing PYD practices as related to the training that they receive will inform practitioners as they develop educational programs for their 4-H volunteers.
Volunteers in 4-H are on the front line when it comes to delivery of youth programming, particularly when programming is delivered through volunteer-led clubs. Club leader volunteers provide a particularly high frequency of direct instruction to youth participants. Training 4-H volunteers in the best practices of program quality and measuring the adoption of these best practices will allow youth development professionals to better understand the quality of the experience for youth members.
High quality PYD programs containing best practices such as those reflected by the 4-H Essential Elements (EE) (Kress, 2005) promote the attainment of positive youth outcomes. Hager and Brudney (2005) state that few volunteer programs actively evaluate the results of the programming, and instead evaluators address economic or program assessments. These types of measures do not tell if volunteers are gaining knowledge or implementing skills learned during 4-H programming. Evaluation data measuring actual behavior change and the implementation of PYD practices in 4-H clubs is lacking for the Florida 4-H program. Thus, a systematic approach to evaluate the implementation of PYD practices by volunteers in direct service to youth is needed. • Does the frequency of training make a difference in outcome attainment?

4-H Youth
• Are volunteers satisfied with the training they receive?
• Does the type of training (face-to-face, blended, or online) make a difference in outcome attainment?
• Do volunteers report implementation of PYD best practices (4-H EE) in their club?
The survey was designed to determine the types and frequency of training that 4-H volunteers participated in, their satisfaction with the training, the volunteer's role of service, and to measure outcomes of adopting best practices of program quality. The team utilized an index of questions that represented each of the four Essential Elements: belonging, mastery, independence and generosity. The questions were designed to determine if volunteer leaders were implementing these practices in their volunteer role as a club leader.
The researchers followed practices stated by Dillman et al. (2014) including engaging contact with multiple messages by using a pre-notice followed by three reminders. Reminders were sent through multiple means of communication (Dillman et al., 2014); utilizing email, SMS, and the 4HOnline enrollment system in order to increase the response rate. A total of 1,243 enrolled volunteers received the survey with 340 respondents resulting in a 27% response rate.
The survey platform Qualtrics was utilized to disseminate the survey to 4-H volunteers who were enrolled in the state 4HOnline database. Data analysis was performed using SPSS and primarily analyzed by a data specialist at the University of Florida Survey Research Center (FSRC) who provided a data analysis report. Statistical analysis to determine index means and correlations between types and degree of training (independent variable) to implementation of PYD practices (dependent variable) was provided by FSRC. An expert statistician provided the state 4-H leadership team with an in-depth understanding of the data in order to best understand programmatic implications.

Results
A total of 340 enrolled volunteers agreed to participate in the evaluation. Most respondents were between the ages of 31 and 50; 87% were female. The sample is geographically representative of the state population for adult respondents. Reported educational attainment level shows 13.44% have a high school degree or less, 45.94% some college or an associate degree, 21.88% a bachelor's degree, and 18.75% a masters or doctoral degree.
Half of volunteers (50.63%) reported service of 1 to 5 years (n = 162), with 21.56% (n = 69) reporting 6 to 10 years, 11.25% (n = 36) 11 to 15 years, 5.31% (n = 17) 16 to 20 years, 5.0% (n = 16) 21 to 25 years, and 6.25% (n = 20) 26 years or more. Respondents were asked how they volunteer and to select all that apply with the majority reporting they volunteered in a community club (n = 188) followed by: a project club (n = 92), after-school club (n = 44), inschool club (n = 10), SPIN (special project interest) club (n = 6), and other (n = 39). The survey also asked volunteers the type of training they received and the frequency of the training in which they participated. The most frequent types of training volunteers participated in were volunteer orientation (60.2%), and county training (54.8%). Volunteers also reported use of the statewide volunteer web-based training series (40.3%), and participation in the statewide Make a Difference Monday virtual training series (16.8%). More than half of the volunteers participated in annual trainings and 20% in monthly trainings (Table 1).

Essential Elements in Practice
The EE outcomes each consisted of 10 questions that asked volunteers about how frequently they practiced each item on a 4-point Likert scale (never, rarely, often, always). Volunteers reported a high frequency of applying the practices that make up each of the EEs (Tables 2   through 5). Results indicated that volunteers perceive themselves as implementing practices that promote belonging, mastery, independence, and generosity. Each construct was combined to create an index variable that could range from 10 to 40.

Impact of Training
The impact of training frequency on generosity shows that those who participate in training monthly (M = 36.09, SD = 4.04) have higher scores on the generosity index than those who participate in training once a year (M = 34.18, SD = 5.37). One-way analysis of variance also shows that the independence index is higher for those who are very satisfied (M = 35.74, SD = 4.32) than those who are somewhat satisfied (M = 33.97, SD = 4.30). In both cases, these differences approached significance (at p < .05).
Difference of means tests (Table 6) show that volunteers who participated in volunteer orientation training scored significantly higher on the independence index than those who did not and scored significantly higher on the mastery index than those who did not. Similarly, those who had volunteer orientation training scored higher than those who did not on the belonging index. Finally, Pearson and Spearman correlations were calculated and show the relationships between the total number of trainings attended, frequency, and satisfaction on the outcomes of the belonging, mastery, independence, and generosity indexes (Table 7). Index variables were created for each of the EEs with a possible score of 40 out of 40. For each index variable Cronbach's alpha was used to determine the internal consistency for the scale.
Scores for all four index scales are strong (Table 8), well beyond the .7 threshold, but both exploratory (EFA) and confirmatory (CFA) factor analyses were performed to assess scale construction. When all 40 measurement items are included in EFA, a five-factor solution is suggested. However, CFA conducted on both models suggested the five-factor model does not improve on the four-factor model in notable ways, and neither is statistically optimal. Future modeling should augment these results with input on validity from 4-H personnel with detailed knowledge of program development.

Implications
The findings of this evaluation are of practical benefit to Extension practitioners in the field of PYD. Extension agents should consider variables such as frequency of training, and opportunities for training outside of traditional face-to-face meetings. Extension Agents should also provide educational training that promotes a high-quality PYD context, and not simply hold informational meetings. Educational programs for volunteers should focus on increasing the knowledge of volunteers in the practices essential to a high-quality PYD program and the role they play in the ultimate attainment of youth outcomes. Increased education on skills that enhance both mastery and generosity should be considered based on the findings of this evaluation as well as foundational practices beyond EE such as 5Cs (Lerner and Lerner, 2013), the eight critical elements of quality PYD (Eccles and Gootman, 2002), and Developmental Relationships (Search Institute, 2014).
As a result of this program evaluation the Florida 4-H leadership team developed a 4-H club basics virtual academy, revised volunteer policies and practices, created a volunteer management handbook, and is training county faculty in the importance of high-quality volunteer training that focuses on the critical elements of PYD. Additional efforts have been made to provide county faculty with easy-to-use evaluation tools that measure knowledge gain and behavior practice at the county level among both volunteers and 4-H members.

Limitations
The measure on the types of training volunteers received was limited by not asking them about one-on-one training received through visits with the county agent, mentorship by other volunteers, and field visits to clubs by the county agent. A second limitation is the inability to compare the self-reported participation of the volunteer to the actual amount of training participated in by the volunteer, which is collected by the respective 4-H agent. This data is not available since surveys were collected anonymously.

Future Research
An additional level of rigor can be gained by adding a qualitative component to further understand the variables measured-for example, exploring why volunteers indicate less confidence in their practice of independence and generosity, as compared with belonging and mastery. Qualitative methods can be used to better ascertain the type of PYD training volunteers need and the outcomes they perceive are attained as a direct result of training.
Finally, collecting data to explain the youth perception of quality practices implemented in the 4-H setting and the influence of these practices on their 4-H experience can enhance the validity of the findings and further increase our understanding about the quality of 4-H programs.
Further research efforts to modify the survey instrument based on the factor analysis loadings will allow the team to strengthen construct indexes and shorten the instrument enabling more frequent data and mainstreamed data collection as programs evolve. Moreover, current findings point to further exploration of the association between self-reported volunteer characteristics and training participation, as well as how volunteer-level variables may predict their EE program practices. Last, there is a benefit in moving from variable-centered to person-centered approaches in data collection. For example, varying EE practice profiles (e.g., latent profile analysis) among 4-H volunteers can be linked with training participation and satisfaction to shed light on specific educational needs.