Measuring STEM Learning in After-School Summer Programs: Review of the Literature

Erica Mallett Moore, Ari Hock, Bronwyn Bevan, Katie Headrick Taylor


After-school and summer programs provide important opportunities for youth to learn STEM practices and form STEM-related identities. However, there has been limited coordination across these programs to measure effectiveness toward learning outcomes. To better understand the constructs that are used to evaluate these programs, we searched key terms related to out-of-school time STEM learning on several education research databases. Our search uncovered 36 different tools. Across these tools, we discovered 76 measures, which were then grouped into 10 constructs based on similar themes. Constructs included: attitude toward science, career awareness and career interest, curiosity, engagement, home/school environment, interest, motivation, nature of science, self-efficacy, and STEM practices. Each construct is defined and clarified with examples from the tools. The review also considers tensions between attempts to standardize measures for evaluating program success and the need to account for equitable STEM learning pathways and adaptability across diverse communities.


out-of-school time; program evaluation; STEM learning; literature review

Full Text:



Afterschool Alliance. (2012). Know your funders: A guide to STEM funding for afterschool.

Allen, P. J., Chang, R., Gorrall, B. K., Waggenspack, L., Fukuda, E., Little, T. D., & Noam, G. G. (2019). From quality to outcomes: A national study of afterschool STEM programming. International Journal of STEM Education, 6(1).

Azevedo, F. S. (2015). Sustaining interest-based participation in science. In K. A. Renninger, M. Nieswandt, & S. Hidi (Eds.) Interest in mathematics and science learning (pp. 281-296). American Educational Research Association.

Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Ed.), Encyclopedia of human behavior (Vol. 4, pp. 71-81). Academic Press. (Reprinted in H. Friedman [Ed.], Encyclopedia of mental health. Academic Press, 1998).

Bang, M., & Vossoughi, S. (2016). Participatory design research and educational justice: Studying learning and relations within social change making. Cognition and Instruction, 34(3), 173-193.

Banks, J. A. (2006). Race, culture, and education: The selected works of James A. Banks. Routledge.

Barron, B., & Bell, P. (2015). Learning in informal and formal environments. In L. Corno & E. Anderman (Eds.), Handbook of educational psychology (3rd ed., ch. 24). Lawrence Erlbaum.

Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49(4), 193-224.

Bell, J., Besley, J., Cannady, M., Crowley, K., Grack Nelson, A., Philips, T., Riedinger, K., & Storksdieck, M. (2019). The role of engagement in STEM learning and science communication: Reflections on interviews from the field. Center for Advancement of Informal Science Education.

Bevan, B. (2016). STEM learning ecologies: Relevant, responsive, and connected. Connected Science Learning, 1(1). NSTA Press.

Bevan, B., Calabrese Barton A., & Garibay, C. (2018). Broadening perspectives on broadening participation in STEM: critical perspectives on the role of science engagement. Center for Advancement of Informal Science Education.

Blackley, S., & Howell, J. (2015). A STEM narrative: 15 years in the making. Australian Journal of Teacher Education, 40(7).

Blosser, P. E. (1984). Attitude research in science education. Information Bulletin, No. 1. ERIC Clearinghouse of Science, Mathematics and Environmental Education.

Briggs, L., Trautmann, N., & Phillips, T. (2019). Exploring challenges and lessons learned in cross-cultural environmental education research. Evaluation and Program Planning. 73,156-162.

Calabrese Barton, A., & Tan, E. (2010). We be burnin'! Agency, identity, and science learning. The Journal of the Learning Sciences, 19(2), 187-229.

Chalkiadaki, A. (2018). A systematic literature review of 21st century skills and competencies in primary education. International Journal of Instruction, 11(3), 1-16.

Chen, S. (2006). Views on science and education (VOSE) questionnaire. Asia-Pacific Forum on Science Learning and Teaching, 7(2).

Collins, P. H. (2018). Confronting color-blind STEM talent development: Toward a contextual model for black student STEM identity. Journal of Advanced Academics, 29(2), 143-168.

Collins, P. H., & Bilge, S. (2016). Intersectionality. Polity Press.

DiSalvo, C. (2012). Adversarial design (Design thinking, design theory). MIT Press.

Engeström, Y. (1999). Expansive visibilization of work: An activity-theoretical perspective. Computer Supported Cooperative Work (CSCW), 8(1), 63-93.

Falk, H. J., Dierking, L. D., Babendure, J., Pattison, S., Meyer, D., Verbeke, M., Coe, M., Palmquist, S., Storksdieck, M., & Canzoneri, N. (2017). The learning SySTEM. STEM Ready America.

Flaskerud, J. H. (1988). Is the Likert scale format culturally biased?. Nursing Research, 37(3), 185–186.

Fu, A. C., Kannan, A., & Shavelson, R. J. (2019). Direct and unobtrusive measures of informal STEM education outcomes. New Directions for Evaluation, 2019(161), 35-57.

Garibay, C., & Teasdale, R. M. (2019). Equity and evaluation in informal STEM education. In A. C. Fu, A. Kannan, & R. J. Shavelson (Eds.), Evaluation in informal science, technology, engineering, and mathematics education. New Directions for Evaluation, 161, 87-106.

Germann P. J. (1988). Development of the attitude toward science in school assessment and its use to investigate the relationship between science achievement and attitude toward science in school. Journal of Research in Science Teaching, 25(8), 689-703.

Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159-1176.

Grack Nelson, A., Goeke, M., Auster, R., Peterman, K., Lussenhop, A. (2019). Shared measures for evaluating common outcomes of informal STEM education experiences. New Directions for Evaluation, 161, 59-86.

Greenberg, M. T., Domitrovich, C. E., Weissberg, R. P., & Durlak, J. A. (2017). Social and emotional learning as a public health approach to education. The Future of Children, 27(1), 13-32.

Halim, L., Abd Rahman, N., Zamri, R., & Mohtar, L. (2018). The roles of parents in cultivating children's interest towards science learning and careers. Kasetsart Journal of Social Sciences, 39(2), 190-196.

Harty, H., & Beall, D. (1984). Toward the development of a children’s science curiosity measure. Journal of Research in Science Teaching, 21(4), 425-436.

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.

Hughes, R. (2015). An investigation into the longitudinal identity trajectories of women in science, technology, engineering, and mathematics. Journal of Women and Minorities in Science and Engineering, 21(3), 181-213.

Jenkins, H., Itō, M., & boyd, d. (2016). Participatory culture in a networked era: A conversation on youth, learning, commerce, and politics. Polity Press.

Joshi, A., Kale, S., Chandel, S., & Pal, D. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396-403.

Khishfe, R., & Abd-El-Khalick, F. (2002). Influence of explicit and reflective versus implicit inquiry-oriented instruction on sixth graders’ views of nature of science. Journal of Research in Science Teaching, 39(7), 551-578.

Kpolovie, P. J., Joe, A. I., & Okoto, T. (2014). Academic achievement prediction: Role of interest in learning and attitude towards school. International Journal of Humanities Social Sciences and Education, 1(11), 73-100.

Lee, J. W., Jones, P. S., Mineyama, Y., & Zhang, X. E. (2002). Cultural differences in responses to a Likert scale. Research in Nursing & Health, 25(4), 295-306.

Merriam-Webster. (n.d.). Framework. In dictionary. Retrieved June 25, 2020, from

Merriam-Webster. (n.d.). Instrument. In dictionary. Retrieved June 25, 2020, from

Merriam-Webster. (n.d.). Mathematics. In dictionary. Retrieved June 25, 2020, from

Merriam-Webster. (n.d.). Measure. In dictionary. Retrieved June 25, 2020, from

Nasir, N. S. (2012). Racialized identities: Race and achievement among African American youth. Stanford University Press.

Nasir, N., Rosebery, A. S., Warren, B., & Lee, C. D. (2006). Learning as a cultural process: Achieving equity through diversity. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (489-504). Cambridge University Press.

National Academies of Sciences, Engineering, and Medicine. (2018). How people learn II: Learners, contexts, and cultures. National Academies Press.

National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. The National Academies Press.

National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press.

National Research Council. (2015). Identifying and supporting productive STEM programs in out-of-school settings. Committee on successful out-of-school STEM learning. Board on Science Education, Division of Behavioral and Social Sciences and Education. The National Academies Press.

Noam, G. G., & Shah, A. (2014). Informal science and youth development: Creating convergence in out-of-school time. Teachers College Record, 116(13), 199-218.

Ogden, J., & Lo, J. (2012). How meaningful are data from Likert scales? An evaluation of how ratings are made and the role of the response shift in the socially disadvantaged. Journal of Health Psychology, 17(3), 350-361.

Organisation for Economic Co-operation and Development. (2006). OECD program for international student assessment 2006.

Philip, T. M., & Azevedo, F. S. (2017). Everyday science learning and equity: Mapping the contested terrain. Science Education, 101(4), 526-532.

Saa Meroe, A. (2014). Democracy, meritocracy and the uses of education. The Journal of Negro Education, 83(4), 485.

Schiefele, U., Krapp, A., & Winteler, A. (1992). Interest as a predictor of academic achievement: A meta-analysis of research. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.),The role of interest in learning and development (183-212). Erlbaum.

Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26(3-4), 207-231.

Science Learning Activation Lab. (2016a). Scientific sensemaking. (Measures Technical Brief version 3.2).

Science Learning Activation Lab. (2016b). Engagement in science learning activities. (Measures Technical Brief version 3.2).

Sneider, C., & Noam, G. G. (2019). The common instrument suite: A means for assessing student attitudes in STEM classrooms and out-of-school environments. Connected Science Learning, 1(11).

Subedi, B. P. (2016). Using Likert type data in social science research: Confusion, issues and challenges. International Journal of Contemporary Applied Sciences, 3(2), 36-49.

Vakil, S. (2018). Ethics, identity, and political vision: Toward a justice-centered approach to equity in computer science education. Harvard Educational Review, 88(1), 26-52.

Yerdelen-Damar, S., & Peşman, H. (2013). Relations of gender and socioeconomic status to physics through metacognition and self-efficacy, The Journal of Educational Research,106(4), 280-289.


Copyright (c) 2022 Erica Mallett Moore, Ari Hock, Bronwyn Bevan, Katie Headrick Taylor

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.