4-H STEM Curriculum Challenges: A Delphi Study

Science, technology, engineering and math (STEM) skills have become essential for today’s youth. STEMliterate individuals are a necessity to fill the ever-growing STEM employment pipeline. Employers are expecting applicants with STEM skills. 4-H programs across the nation have recognized the importance of educating STEM-minded youth. As STEM programs become more prevalent within 4-H, it is essential to address challenges Extension educators and 4-H volunteers face when teaching STEM curriculum. In this study the Delphi technique was utilized to collect the opinions of a geographically dispersed group of 4-H Extension educators and volunteers. Each group served on a separate panel and were asked the question, “What challenges do you face when teaching STEM curriculum?” After 3 rounds of study, both the educator and volunteer panels identified 2 challenges faced when teaching STEM curriculum. Through identification of these challenges, Oklahoma state 4-H staff can better address the professional development needs of educators and volunteers within the organization, and support the desired scientific literacy outcomes of 4-H youth.


Introduction Background
Scientific literacy is essential for youth success in the 21 st century (National Science Board, 2018). Employers are searching for scientifically literate young people to fill the STEM (Science, Technology, Engineering, and Math) pipeline (U.S. Department of Commerce, 2017). According to the National Science Board (2018), the number of jobs requiring extensive STEM abilities has increased by 34% in the last 10 years. However, little research exists on preparation of educators and volunteers to teach science curricula; it is imperative to identify these challenges to ultimately improve scientific literacy of youth (Smith & Schmitt-McQuitty, 2013).
The need for STEM education is being addressed by government agencies, private entities, school systems, and non-formal educational settings, such as 4-H (Cafarella et al., 2017; National Academy of Science, 1996;Shafer, 2015;Turnbull et al., 2013;The White House, 2017). These agencies and organizations recognize the importance of STEM education and are striving to create a scientifically literate society and provide youth with the knowledge and skills needed to succeed in 21 st century life (4-H National Headquarters, 2011;Rice et al., 2016).
Non-formal education organizations are providing additional avenues to expand this knowledge and peak science interest (Kahler & Valentine, 2011;Kisiel, 2006).

In 2006, an initiative was introduced by National 4-H Headquarters and the National 4-H
Council to increase and improve STEM programming within 4-H (Turnbull et al., 2013). By 2012, 1.33 million new youth had been reached with 4-H science programming due to the direct impact of the 4-H Science Initiative (Noyce Foundation, 2013). Through this initiative, pedagogical methods of science instruction were examined to determine their effectiveness in increasing youth engagement and knowledge (Turnbull et al., 2013).
While the implementation of STEM programming has been positive, retention of 4-H Extension educators to carry out the programming is an ongoing issue (Harder et al., 2014;Safrit & Owen, 2010;Vines et al., 2018). Safrit and Owen (2010) found training is an important aspect of job satisfaction and retention of 4-H Extension educators.. Additionally, 4-H programs rely on volunteers to ensure the success and impact of 4-H programs in promotion of positive youth development (Alexander & Freel, 2018;Pleskac, 2009;Sinasky & Bruce, 2007). Numerous researchers found volunteers are motivated for a variety of reasons and it is also important to address their motivation and retention as it relates to the 4-H program (Alexander & Freel, 2018;Cleveland & Thompson, 2007;Culp & Schwartz, 1999a;Terry et al., 2013;Worker, 2016). With an established need for both 4-H Extension educator and volunteer training in STEM topics, the awareness of instructional methods of STEM education can help guide professional development and increase student learning (Astroth, 2007;Haugen et al., 2016;Safrit & Owen, 2010).

Problem and Purpose
To address scientific literacy within the

Delphi Method
The Delphi technique was the methodology selected for this study. Within the realm of education, the Delphi method has been utilized for varying purposes including curriculum development, evaluation, and identification of program barriers (Martin & Frick, 1998). Delphi studies allow for the collection of opinions from numerous experts within a field, despite geographic separation (Hsu & Sandford, 2007;Mayfield et al., 2005;Sitlington & Coetzer, 2014). Structured, anonymous commentary is provided by the experts to gather consensus of opinion regarding a topic or issue (Brady, 2015). Questionnaires are sent to a panel of experts with information and results being presented to panel members between each round (Hanfin, 2004). Through this iteration process panelists can examine and clarify their ideas (Dalkey et al., 1972;Hsu & Sandford, 2007). Delphi methodology combines both quantitative and qualitative data to inform practice within an organization (Ludwig, 1997).
The study utilized a 3-round Delphi conference approach and included two panels of experts, so items meeting consensus after three rounds could be compared ( Ramsey, 2009;Young et al., 2019). Two populations were of interest this study: Oklahoma Cooperative Extension educators and Oklahoma 4-H volunteers. Purposive samples of each population were proportionally stratified based on geographic location, ensuring equal representation of the four Oklahoma Cooperative Extension districts: Northeast, Southeast, Northwest, and Southwest (Johnson & Christensen, 2014, Saucier et al., 2012.

Panel Selection
The first panel consisted of volunteers who met the following two characteristics: 2. Volunteer panelists must be tenured, meaning they have served 5 or more years in their role with 4-H (Culp & Schwartz, 1999b).
The second panel consisted of Oklahoma Cooperative Extension educators nominated by their respective 4-H district program specialists (DPSs) to serve on the second panel. The nominations followed the recommendation of Hsu and Sandford (2007) to receive nominations from well-known and respected leaders. The DPSs were asked to recommend educators they deemed "expert" in the area of 4-H youth development, with a minimum 50%-time 4-H appointment in the Oklahoma Cooperative Extension system.

Instrument Development and Data Collection
A personalized email (Dillman et al., 2014) was sent to seven volunteers and seven nominated educators from each of the four districts (a total of 28 for each panel), requesting their participation in the study. The email included an internet survey link to the first-round questionnaire. The first questionnaire was composed of demographic questions and the openended question, "What challenges do you face when teaching STEM curriculum?" Demographic questions included district association, years served in respective roles, gender, race/ethnicity, education attained, and whether or not formal or non-formal training had been received in science. Fourteen volunteers (50%) and 22 educators (79%) completed their respective firstround questionnaires.
Upon receiving first-round responses challenge statements were developed. To generate the challenge statements presented in Round 2, a thematic analysis was completed on responses to the open-ended question for both panels (Johnson & Christensen, 2014. Participant responses were examined for duplicate wording and ideas, along with compound statements that would need to be separated (Ramsey, 2009;Shinn et al., 2009). Inductive codes were utilized (Johnson & Christensen, 2014). Statements were divided into meaningful analytical units (Johnson & Christensen, 2014;Rubin et al., 2009) and hand-coded to identify concepts within the qualitative data set. To ensure intercoder reliability, an additional researcher coded the data independently (Johnson & Christensen, 2014). Results were shared and common themes developed into the challenge statements presented to each panel.
Panelists were asked to indicate their agreement with each challenge statement on a 6-point Likert-type scale: 1 (strongly disagree), 2 (disagree), 3 (slightly disagree), 4 (slightly agree), 5 (agree), and 6 (strongly agree). This response scale follows other agricultural education professionals who have used Delphi methodology for social sciences research (Ramsey, 2009;Shinn, et al., 2009;Siegfried, 2011). A comment box was provided with each challenge statement, allowing participants to clarify ideas or offer justification for responses (Hsu & Sandford, 2007;Ludwig, 1997). Any comments provided were used by researchers in data analysis. Eight volunteers (29%) and 13 educators (46%) completed Round 2. Ludwig (1997) suggests having a predetermined level of consensus for Delphi studies. For our study, an a-priori decision was made: If 75% of each panel ranked an item 5 or 6 then the statement would meet consensus and be removed from further study. Items receiving less than 75% agreement, but more than 51% agreement, were sent in a Round 3 questionnaire. Items receiving less than 51% agreement did not meet consensus and were removed from further examination (Ramsey, 2009;Shinn, et al., 2009;Siegfried, 2011).
Completion of the second-round questionnaire prompted an invitation to participate in the thirdround questionnaires. Challenge statements not reaching consensus in Round 2 but reached agreement of 51-74% were included. Panelists were asked to rank their agreement with these challenge statements for a final time. Anonymous feedback from Round 2 was included for participants to examine (Fletcher & Marchildon, 2014;Ludwig, 1997). A comment box was also provided with each challenge statement, allowing participants to clarify ideas or offer justification for responses (Hsu & Sandford, 2007;Ludwig, 1997). Any comments provided were used by researchers in data analysis. Seven volunteers (25%) and 13 educators (46%) completed Round 3.

Round 1
Round 1 of this study sought to understand the perceived STEM curriculum challenges within

Round 2
Panelists who completed Round 1 were sent questionnaires to complete Round 2 of the study.
Through the questionnaire, each participant was asked to indicate their level of agreement with the challenge statements identified in the first round. The results are displayed in Table 1 and Table 2.
For our study: If 75% of our expert panelists rated an item 5 or 6 on the 6-point Likert-type scale, then the statement would meet consensus and be removed from further study. Items receiving less than 75% agreement, but more than 51% agreement, were sent to the next round. Items receiving less than 51% agreement did not meet consensus and were removed from further examination (Ramsey, 2009;Shinn, et al., 2009;Siegfried, 2011). This technique provided a predetermined a priori decision-making tool (Ludwig, 1997).

Round 3
Challenge statements reaching 51%-74% agreement in Round 2 were sent back to the panelists in the third round. This round sought to identify consensus on the remaining items. No challenge statements met consensus in this round (See Table 3 and Table 4).

Panel Findings Comparison
After three rounds of questionnaires, two challenge statements met consensus within each panel. Comparison of perceived challenges is presented in Table 5. Both panels reached consensus on the statement "competing with other activities for youths' time." The educator panel reached consensus on the statement "feeling adequately trained," and the volunteer panel reached consensus on the statement "funding for supplies/equipment." The comparison of challenge statements not meeting consensus after three rounds are presented in Table 6.

Implications for STEM Curriculum
The purpose of this study was to discover the challenges faced when teaching STEM curriculum by selected tenured 4-H volunteers and Cooperative Extension 4-H Educators. Between the two panels, three challenges were identified, which impact teaching of STEM curriculum:

Funding for Supplies/Equipment
Funding for Oklahoma Extension has decreased drastically in the past 7 years (27%) according to Trapp (2017). This has led to changes in staffing structure, which will continue to affect Extension within the state. Volunteers may not be able to offset personal expenditures for supplies and other items used for their 4-H activities. Without additional funding, STEM programming could be impaired. If club leaders struggle to provide the materials necessary to implement STEM curriculum, and counties or state staff are not in a financial position to assist, STEM programming will be set aside for more affordable and easily accessed curriculum.
During times of financial stress, it is imperative organizations demonstrate their impact to stakeholders (O'Neill, 1998;Workman & Scheer, 2012). As a public program, Extension relies on outside sources to provide funding, and must prove to university administrators, taxpayers, and legislators it is worth the investment (O'Neill, 1998). With diminishing funding, documentation of 4-H's impact in the counties and state is vital to receiving recognition and funding for the sake of positive youth development (Workman & Scheer, 2012). Historic data and solitary success stories, while worthwhile, are not enough to convince stakeholders in modern society to provide funding. If at all possible, dollar amounts and other economic impact data should be collected (O'Neill, 1998).

Feeling Adequately Trained
Training is essential for educators to increase scientific knowledge. Finding time to attend training and having sufficient funds to do so have been reported as obstacles in receiving science training (Riley & Butler, 2012;Turnbull et al., 2013). However, many educators lack competence and confidence in science subject matter; therefore, it is crucial to provide quality training (Haugen et al., 2016;Turnbull et al., 2013). Though lack of confidence in STEM subject matter did not reach consensus with the educator panel, it is still worth considering in the context of feeling adequately trained. Without proper training, educators may not teach STEM curriculum due to lack of confidence and understanding.
In Oklahoma, 4-H state staff provide a minimum of five science in-service training sessions per year. Findings from this study indicate state Extension staff should look at ways to address the needs of county 4-H educators more accurately (Sinasky & Bruce, 2007). Numerous educators commented on feeling confident in teaching the STEM workshops covered in professional development trainings, but they did not know how to proceed with science curriculum on their own. Specific content should be covered within these professional development sessions; however, additional time should be spent on pedagogical practices and learning strategies that can aid in youth engagement and educator understanding (Haugen et al., 2016).

Competing With Other Activities for Youths' Time
Competition with other activities is not a new phenomenon to 4-H member retention (Albright & Ferrari, 2010;Astroth, 1985;Harder et al., 2005;Meeks-Baney & Jones, 2013). This is recognized as a challenge by both the volunteer and educator panels. As youth get older, time conflicts can arise with jobs and other out-of-school organizations such as organized sports (Albright & Ferrari, 2010;Harder et al., 2005). Meeks-Baney and Jones (2013) contend youth want to participate in organizations meeting their needs for affiliation (relationships) and achievement (recognition). Youth also want to hold meaningful roles and utilize time wisely, and if needs are not met, youth will leave a program (Albright & Ferrari, 2010).
With the knowledge that youth leave 4-H due to other activities' meeting intrinsic needs, time constraints, and job requirements, educators and volunteers should support youth within these areas. By working closely with other organizations vying for youths' time and developing strategies to assist each other, 4-H members can experience affiliation and success within more than one organization and increase satisfaction and retention (Albright & Ferrari, 2010;Meeks-Baney & Jones, 2013). Club structure could also increase in flexibility to allow youth participation in other activities (Harder et al., 2005). To address the interests of participating youth, and allow for ownership of project choice, group leaders could conduct a needs assessment of their club membership (Harder et al., 2005). By allowing group input, recognizing its value, and putting results into practice, youth will feel empowered, which can encourage Organizations could work together to train volunteers and instructors. Partnering with other youth-serving organizations for STEM training may be an appropriate solution. Each organization must recognize its limitations and seek to partner with other organizations to capitalize on specialty and training strengths. Cooperation and communication among competing organizations could strengthen and enhance youth development efforts.
The same enhanced communication and cooperation among youth-serving organizations could allow youth to participate in more activities. We often talk to youth about choices and time limitations. We could switch this conversation to include more youth-serving organizations to adjust scheduling conflicts and cooperation between organizations with an improved focus on youth needs and development. Communication among competing organizations to focus on youth development, youth wants, and needs would be beneficial for all.
Youth development professionals need to consider these challenges in advance of planning and scheduling STEM programming. Advanced planning and communication will enhance the programmatic success and the educational outcomes of STEM education.

Recommendations for Research
Further research should be conducted to examine the STEM training needs of both educators and volunteers within Oklahoma 4-H and around the country. This research should evaluate current practices and address preferred methods of adult education. Research should also be done on effective evaluation techniques to demonstrate impact with a statewide Extension system. To address youth motivators towards STEM education, research should be conducted to determine preferred subjects and methods of dissemination. Additionally, further research should be done on STEM curriculum challenges of 4-H volunteers. The volunteer panel of the current study did not result in a reliable sample size. However, the qualitative comments provided by this panel were rich in information and ideas for future practice. These ideas should be used to develop a pilot program to improve volunteer training, which could be modeled in other states. These ideas should also be examined further to determine if pursuit of changes in practice is wise.