4-H STEM Curriculum Challenges: A Delphi study


  • Hannah Branscum Oklahoma State University, 4-H Youth Development
  • Jeff Sallee Oklahoma State University, 4-H Youth Development
  • Kevin Allen Oklahoma State University, Natural Resources, Ecology, and Management
  • Marshall A. Baker Collegiate Edu-Nation
  • Samantha Blackwell Oklahoma State University, Agricultural Education, Communications, and Leadership




STEM, curriculum, Delphi, challenges


Science, technology, engineering and math (STEM) skills have become essential for today’s youth. STEM-literate individuals are a necessity to fill the ever-growing STEM employment pipeline. Employers are expecting applicants with STEM skills. 4-H programs across the nation have recognized the importance of educating STEM-minded youth. As STEM programs become more prevalent within 4-H, it is essential to address challenges Extension educators and 4-H volunteers face when teaching STEM curriculum. In this study the Delphi technique was utilized to collect the opinions of a geographically dispersed group of 4-H Extension educators and volunteers. Each group served on a separate panel and were asked the question, “What challenges do you face when teaching STEM curriculum?” After three rounds of study, both the educator and volunteer panels identified 2 challenges faced when teaching STEM curriculum. Through identification of these challenges, Oklahoma state 4-H staff can better address the professional development needs of educators and volunteers within the organization, and support the desired scientific literacy outcomes of 4-H youth.

Author Biography

Jeff Sallee, Oklahoma State University, 4-H Youth Development

Dr. Jeff Sallee serves as an associate professor specializing in 4-H STEM programming for the Oklahoma 4-H youth development program, Oklahoma State University, Agricultural Education, Communication and Leadership department. Since 1991, Sallee has been dedicated to Extension and 4-H programs in Arkansas, Georgia and Oklahoma. Sallee focuses on science, technology, engineering, and math (STEM) and has been successful at implementing service learning by applying youth voice and inquiry based educational techniques to integrate STEM elements into the Oklahoma 4-H program. Sallee has proved to be successful at incorporating STEM curriculum and programming into Oklahoma counties to develop educational projects and demonstrations, including robotics, water conservation, geospatial science, and digital media.



Albright, M., & Ferrari, T. (2010). “Push” and “pull” a qualitative study of factors that contribute to older youth leaving the 4-H program. Journal of Youth Development, 5(3). https://jyd.pitt.edu/ojs/jyd/article/view/209

Alexander, J., & Freel, A. (2018). From the top down: How a statewide listening tour and survey improves 4-H volunteerism. Unpublished manuscript, Arkansas 4-H, University of Arkansas.

Astroth, K. (1985). The challenge of retaining 4-H members. Journal of Extension, 23(3). https://www.joe.org/joe/1985fall/sa4.php

Astroth, K. (2007). Making the best better: 4-H staffing patterns and trends in the largest professional network in the nation. Journal of Youth Development, 2(2). https://jyd.pitt.edu/ojs/jyd/article/view/343

Brady, S. (2015). The Delphi method [uncorrected proof]. http://www.academia.edu/15618774/The_Qualitative_Delphi_Method

Buriak, P., & Shinn, G. (1989). Mission, initiatives, and obstacles to research in agricultural education: A national Delphi using external decision-makers. Journal of Agricultural Education, 30(4), 14-23. https://doi.org/10.5032/jae.1989.04014

Cafarella, J., McCullock, A., & Bell, P. (2017). Why do we need to teach science in elementary school? (Practice Brief No. 43). http://stemteachingtools.org/assets/landscapes/STEM-Teaching-Tool-43-Elementary-Science.pdf

Cleveland, L., & Thompson, J. (2007). Extension education and volunteer service: Assessing motivation and action. Journal of Extension, 45(1). https://www.joe.org/joe/2007february/rb1.php

Culp, K., & Schwartz, V. J. (1999a). Motivating adult volunteers. Journal of Extension, 37(1). https://www.joe.org/joe/1999february/rb5.php

Culp, K., & Schwartz, V. J. (1999b). Recognizing tenured 4-H adult volunteers. Journal of Agricultural Education, 40(2), 38-45. https://doi.org/10.5032/jae.1999.02038

Dalkey, N. C., Rourke, D. L., Lewis, R., & Snyder, D. (1972). Studies in the quality of life. Lexington Books.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.), John Wiley & Sons.

Fletcher, A., & Marchildon, G. (2014). Using the Delphi method qualitative, participatory action research in health leadership. International Journal of Qualitative Methods, 13, 1-18. https://doi.org/10.1177/160940691401300101

-H National Headquarters. (2011). Mission mandates [Fact Sheet]. https://nifa.usda.gov/sites/default/files/resource/4-H%20Mission%20Mandates.pdf

Hanafin, S. (2004). Review on literature on the Delphi technique. Department of Children and Youth Affairs, Ireland. https://www.dcya.gov.ie/documents/publications/Delphi_Technique_A_Literature_Review.pdf

Harder, A., Gouldthorpe, J., & Goodwin, J. (2014). Why work for Extension? An examination of job satisfaction and motivation in a statewide employee retention study. Journal of Extension, 52(3). https://www.joe.org/joe/2014june/a5.php

Harder, A., Lamm, A., Lamm, D., Rose, H., & Rask,G. (2005). An in-depth look at 4-H enrollment and retention. Journal of Extension, 43(5). https://www.joe.org/joe/2005october/rb4.php

Haugen, H., Stevenson, A., & Meyer, R. L. (2016). Participant comfort with and application of inquiry-based learning: Results from 4-H volunteer training. Journal of Extension, 54(1). https://www.joe.org/joe/2016february/a5.php

Hsu, C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12(10). http://pareonline.net/pdf/v12n10.pdf

Johnson, B., & Christensen, L. (2014). Educational research (5th ed.). Sage Publications.

Kahler, J. & Valentine, N. (2011). Stemming the gap. AfterSchool Today. http://techdirections.com

Kisiel, J. (2006). Urban teens exploring museums: Science experiences beyond the classroom. The American Biology Teacher, 68(7), 396-401. https://doi.org/10.2307/4452027

Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of Extension, 35(5). https://www.joe.org/joe/1997october/tt2.php

Martin, A. G., & Frick, M. J. (1998). The Delphi technique: An informal history of its use in agricultural education research since 1984. Journal of Agricultural Education, 39(1), 73-79. https://doi.org/10.5032/jae.1998.01073

Mayfield, C. A., Wingenbach, G. J., & Chalmers, D. R. (2005). Assessing stakeholder needs: Delphi meets the internet. Journal of Extension, 43(3). https://www.joe.org/joe/2005june/iw1.php

Meeks-Baney, C., & Jones, K. (2013). Whatever it takes: A comparison of youth enrollment trends in the 4-H livestock and non-livestock programs. Journal of Extension, 51(3). https://www.joe.org/joe/2013june/rb2.php

National 4-H Council. (2018). What is 4-H? https://4-h.org/about/what-is-4-h/

National Academy of Sciences (1996). National Science Education Standards. National Academy Press.

National Science Board. (2018). Science and engineering indicators (Policy Companion Brief No. NSB-2018-7). https://www.nsf.gov/nsb/sei/companion-brief/NSB-2018-7.pdf

Noyce Foundation. (2013). 4-H science impact: From inception to impact. https://4-h.org/wp-content/uploads/2016/02/4-H-Science-Initiative-from-Inception-to-Impact-Comprehensive-Report.pdf

Oklahoma 4-H. (2017). Oklahoma 4-H Volunteer Board Constitution. http://4h.okstate.edu/leadership/copy_of_state-volunteer-board/ConstitutionandBylaws_2014.docx/

O’Neill, B. (1998). Money talks: Documenting the economic impact of extension personal finance programs. Journal of Extension, 36(5). https://www.joe.org/joe/1998october/a2.php

Pleskac, S. (2009). Wisconsin 4-H volunteers documenting their contributions and volunteer development. http://www.uwex.edu/ces/4h/resources/mgt/documents/WIVolunteerContributionReport11409final.pdf

Ramsey, J. W. (2009). Identifying entry-level skills expected by agricultural industry experts and determining teachers’ perceptions on whether they are being learned through students’ participation in the supervised agricultural experience component of the secondary agricultural education program: A two-panel Delphi study (Doctoral dissertation). Available from ProQuest Dissertations and theses database. (UMI No. 3390968)

Rice, J. E., Rugg, B., & Davis, S. (2016). Minnesota 4-H science and agriculture challenge: Infusing agricultural science and engineering concepts into 4-H youth development. Journal of Extension, 54(3). https://www.joe.org/joe/2016june/iw4.php

Riley, D., & Butler, A. (2012). Priming the pipeline lessons from promising 4-H science programs. https://4-h.org/wp-content/uploads/2016

Rubin, A., & Babbie, E. R. (2009). Research methods for social work (7th ed.). https://www.researchgate.net/publication/237131811_Research_Methods_for_Social_Work

Safrit, D., & Owen, M. (2010). A conceptual model for retaining county Extension program professionals. Journal of Extension, 48(2). https://www.joe.org/joe/2010april/a2.php

Saucier, R., McKim, B., & Tummons, J. (2012). A delphi approach to the preparation of early career agricultural educators in the curriculum area of agricultural mechanics: Fully qualified and highly motivated or status quo? Journal of Agricultural Education, 53(1), 136-149. https://doi.org/10.5032/jae.2012.01136

Shafer, L. (2015, November 13). Why science? Usable Knowledge. https://www.gse.harvard.edu/news/uk/15/11/why-science

Shinn, G. C., Wingenbach, G. J., Lindner, J. R., Briers, G. E., & Baker, M. (2009). Redefining agricultural and extension education as a field of study: Consensus of fifteen engaged international scholars. Journal of International Agricultural and Extension Education, 16(1), 73-88. https://doi.org/10.5191/jiaee.2009.16106

Siegfried, H. (2011). Benefits of the Oklahoma Operation: Military Kids program- a Delphi study (Master’s thesis). Available from ProQuest Dissertations and Theses database. (UMI No.1534745)

Sinasky, M., & Bruce, J. (2007). Volunteers’ perceptions of the volunteer management practices of county extension 4-H educators. Journal of Extension, 45(3). https://www.joe.org/joe/2007june/tt5.php

Sitlington, H., & Coetzer, A. (2014). Using the Delphi technique to support curriculum development. Education + Training, 57(3). https://doi.org/10.1108/ET-02-2014-0010

Smith, M. H., & Schmitt-McQuitty, L. (2013). More effective professional development can help 4-H volunteers address need for youth scientific literacy. California Agriculture, 67(1), 47-53. https://doi.org/10.3733/ca.v067n01p47

Terry, B., Pracht, D., Fogarty, K., Pehlke, T., & Barnett, L.J. (2013). The similarities between volunteer behavior and consumer behavior: A study of volunteer retention. Journal of Extension, 51(6). https://www.joe.org/joe/2013december/rb2.php

Trapp, J. (2017). Oklahoma Extension: Today and tomorrow [PowerPoint slides]. Oklahoma Cooperative Extension Service.

Turnbull, B. J., Mielke, M., & Butler, A. (2013). The 4-H science initiative: Summary observations from an evaluation. http://3t61of1t6u3x3af7ir2y91ib.wpengine.netdna-cdn.com/wpcontent/uploads/2016/02/4-H-Science-Initiative-PSA-Summary-Report.pdf

U.S. Department of Commerce. (2017, March). STEM jobs: 2017 update (Issue Brief No. 02-17) http://www.esa.doc.gov/sites/default/files/stem-jobs-2017-update.pdf

Vines, K., Cletzer, A., Westfal-Rudd, D., Lambur, M., Hunnings, J., & Vines, N. (2018). Identifying needs and implementing organizational change to improve retention of early career agents. Journal of Extension, 56 (1). https://www.joe.org/joe/2018february/a2.php

The White House. (2017). President Trump signs memorandum for STEM education funding. https://www.whitehouse.gov/articles/president-trump-signsmemorandum-stem-education-funding/

Worker, S. (2016). Volunteer educators’ influence on youth participation and learning in 4-H STEM learning by design programs (Doctoral dissertation). Available from ProQuest Dissertations and theses database. (UMI No. 10165795).

Workman, J. & Scheer, S. (2012). Evidence of impact: Examination of evaluation studies published in the journal of extension. Journal of Extension, 50(2). https://www.joe.org/joe/2012april/a1.php

Young, K., Sallee, J., Ramsey, J. & Cartmell, D. (2019). Using the Delphi technique to determine challenges facing the Oklahoma’s 4-H program. Journal of Youth Development 14(3) https://doi.org/10.5195/jyd.2019.740






Research & Evaluation Studies