Group Mapping in a 4-H Camp Mixed-Methods Evaluation


  • Vikram Koundinya University of California, Davis and University of California Cooperative Extension
  • Jenna Klink University of Wisconsin, Madison, Division of Extension
  • Joanna Skluzacek University of Wisconsin, Madison, Division of Extension
  • Claire Barrett University of Wisconsin, Madison
  • Cristina Chiarella University of California, Davis



4-H camp, evaluation, mixed methods, systems thinking, complex systems


A week-long residential 4-H Agri-science youth camp had the goal of increasing understanding of agriculture as a system, however evaluating such a complex construct is difficult by adopting any one data-collection method. This paper presents the methods used in the evaluation of the camp and how these methods enhanced the overall program evaluation. The camp introduced 28 students, who were about 12 years of age from rural and urban counties of Wisconsin to the different aspects of the agricultural system, and promoted understanding of the connections among the various system components. The study utilized a mixed-methods triangulation design with group mapping (qualitative and quantitative), participant observation (qualitative), student reflections in the form of group presentations (qualitative), and retrospective post-then-pre surveys (quantitative) to measure and assess the anticipated outcomes. The qualitative assessment of the maps from Days 1 to 5 showed a significant improvement in students’ understandings of the interconnections of farm systems, which was validated by quantitative assessment of maps. Participant observation revealed that the camp provided students with a rich experiential learning experience and identified areas of improvement such as coordination of efforts between the different camp instructors. Student reflections and the end-of-camp survey indicated that the students increased their knowledge and skill levels as a result of attending the camp. The mixed-methods design used in this study serves as an example for designing better quality youth and Extension education program evaluations, as well as programs where the goal is to increase participants’ understandings of complex systems.

Author Biography

Vikram Koundinya, University of California, Davis and University of California Cooperative Extension

Evaluation Specialist

University of California (UC)-Daivs and UC Cooperative Extension


Arnold, M. E., & Cater, M. (2011). From then to now: Emerging directions for youth program evaluation. Journal of Youth Development, 6(3). 83-94.

Bell, A. (2007). Designing and testing questionnaires for children. Journal of Research in Nursing, 12(5), 461-469.

Bennet, T. (2018). Including the youth perspective: The development of the CPQA Camper Survey. Journal of Youth Development, 13(1-2), 266-285.

Besterfield-Sacre, M., Gerchak, J., Lyons, M., Shuman, L. J, & Wolfe, H. (2004). Scoring concept maps: An integrated rubric for assessing engineering education. Journal of Engineering Education, 93, 105-115.

Brady, B., & O’Regan C. (2009). Meeting the challenge of doing an RCT evaluation of youth mentoring in Ireland: A journey in mixed methods. Journal of Mixed Methods Research, 3(3), 265-280.

Brown, J. S. (2006). Rethinking concept mapping for youth participatory evaluation in the context of youth development programs. Journal of Youth Development, 1(2).

Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6, 97-113.

Carlson, S. P., Heimlich, J. E., Storksdieck, M. & Meyer, N. (2009). Best practices for field days assessment tool and observation protocol.

Creswell, J. W. & Plano Clark, V. L. (2006). Designing and conducting mixed methods research. Sage.

Dale, E. (1969). Audiovisual methods in teaching (3rd ed.). Dryden Press.

DeWalt, K. M. & DeWalt, B. R. (2011). Participant observation: A guide for fieldworkers (2nd ed.). Rowman & Littlefield.

Diem, K. G. (2002). Using research methods to evaluate your extension program. Journal of Extension, 40(6).

Edwards, H. C., Culp III, K., & Jordan, J. W. (2019). Using an innovative multiple-methods approach to evaluate extension conferences. Journal of Extension, 57(2).

Emery, M., Higgins, L., Chazdon, S., & Hansen, D. (2015). Using ripple effect mapping to evaluate program impact: Choosing or combining the methods that work best for you. Journal of Extension, 53(2).

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). John Wiley & Sons.

Garton, M. S., Miltenberger, M., & Pruett, B. (2007). Does 4-H camp influence life skill and leadership development? Journal of Extension, 45(4).

Henderson, K., Powell, G., & Scanlin, M. (2005). Observing outcomes in youth development: An analysis of mixed methods. The Journal of Park and Recreation Administration, 3(4), 58-77.

Jayaratne, K. S. U. (2001). Agricultural extension educators’ perceptions regarding the teaching and learning processes as related to sustainable agriculture: Implications for agricultural extension education (Unpublished doctoral dissertation). Iowa State University.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Johnson, R. B., Onwuegbuize, A. J., & Turner, L. A. (2007). Towards a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133.

Johnson, V., Ronan, K., Johnston, D., & Peace, R. (2014). Evaluations of disaster education programs for children: A methodological review. International Journal of Disaster Risk Reduction, 9, 107-123.

Jones, K., Gwynn, E., & Teeter, A. (2019). Quantitative or qualitative: Selecting the right methodological approach for credible evidence. Journal of Human Sciences and Extension, 7(2), 61-87.

Jorgensen, D. (2015). Participant observation. In R. Scott & S. Kosslyn (Eds.), Emerging trends in the social and behavioral sciences (pp. 1-15). John Wiley & Sons.

Klatt, J., & Taylor-Powell, E. (2005). Using the retrospective-post-then-pre design. Quick Tips #27. University of Wisconsin-Extension.

Klink, J., Koundinya, V., Kies, K., Robinson, C., Rao, A., Berezowitz, C., Widhalm, M, & Prokopy, L. (2017). Enhancing interdisciplinary climate change work through comprehensive evaluation. Climate Risk Management, 15, 109-125.

Koundinya, V., & Martin, R. A. (2011). Teaching methods and tools used in food safety extension education programs in the North Central Region of the United States. International Journal of Agricultural Management & Development, 1, 157-167.

Kwaw-Mensah, D. (2008). Perceptions of agricultural extension educators regarding livestock waste management education in the North Central Region (Unpublished doctoral dissertation). Iowa State University.

Lewis, K. M., Bird, M., Wilkins, T., Borba, J., Nathaniel, K., & Schoenfelder, E. (2018). Developing a common evaluation tool for camps. Journal of Youth Development, 13(1-2), 306-315.

Mertens, D., & Hesse-Beber, S. (2013). Mixed methods and credibility of evidence in evaluation. New Directions for Evaluation, 138, 5-13.

Morgan, D. (2007). Paradigms lost and pragmatism regained. Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.

Nathaniel, K., & Kinsey, S. B. (2013). Contributions of youth engagement to the development of social capital through community mapping. Journal of Extension, 51(1).

National 4-H History Preservation Program. (2019). 4-H motto, creed and pledge.

Patton, M. Q. (2015). Evaluation in the field: The need for site visit standards. American Journal of Evaluation, 36(4), 444-460.

Rallis, S. F. (2015). When and how qualitative methods provide credible and actionable evidence: Reasoning with rigor, probity, and transparency. In S. I. Donaldson, C. A. Christie, & M. M. Mark (Eds.), Credible and actionable evidence: The foundation for rigorous and influential evaluations (2nd ed., pp. 137–156). Sage.

Reeves, K. T., Donaldson, J. L., Naylor, M., & LeBleu, L. (2017). Youths perceive some improvement in substance abuse prevention knowledge, skills, and assets from participation in 4-H Health Rocks! Journal of Extension, 55(4).

Rennekamp, R. A., & Engle, M. (2008). A case study in organizational change: Evaluation in Cooperative Extension. In M. T. Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizational system: Lessons from Cooperative Extension [Special issue], New Directions for Evaluation, 2008(120), 15-26. Jossey-Bass.

Roberts, G. (2006). A philosophical examination of experiential learning theory for agricultural educators. Journal of Agricultural Education, 47(1), 17-29.

Sami, A. (2016). Integrating quantitative and qualitative data in mixed methods research--Challenges and benefits. Journal of Education and Learning, 5(3).

Sheldon, J., & Hopkins, L. (2008). Supporting Success. Why and how to improve quality in after-school programs. The James Irvine Foundation.

Stair, K. S., Blackburn, J. J., Bunch, J. C., Blanchard, L., Cater, M., & Fox, J. (2016). Perceptions and educational strategies of Louisiana agricultural education teachers when working with students with special needs. Journal of Youth Development, 11(1).

Stair, K. S., Moore, G. E., Wilson, B., Croom, B., & Jayaratne, K. S. U. (2010). Identifying confidence levels and instructional strategies of high school agricultural education teachers when working with students with special needs. Journal of Agricultural Education, 51(2), 90-101.

Wells, M. S., & Arthur-Banning, S. G. (2009). Mapping out your success: Using mind maps to evaluate youth development programs. Journal of Youth Development, 4(2).

Whittington, A., & Garst, B. A. (2018). The role of camp in shaping college readiness and building a pathway to the future for camp alumni. Journal of Youth Development, 13(1-2), 105-125.

Yohalem, N., Granger, R., & Pittman, K. (2009). The quest for quality: Recent developments and future directions for the out-of-school-time field. New Directions for Youth Development, 121, 129-140.






Research & Evaluation Studies