Fostering Curiosity, Inquiry, and Scientific Thinking in Elementary School Students: Impact of the NE STEM 4U Intervention

In this qualitative study, we assessed the impact of the NE STEM 4U intervention on elementary school youth in terms of excitement, curiosity, and STEM concepts. The NE STEM 4U intervention incorporates a problem-based learning theoretical framework in an after-school time, weekly or twice-weekly intervention. We assessed student performance over two academic years of participation in the intervention using the Dimensions of Success observational tool. Ultimately, we link this mentor-led program with increases in curiosity, inquiry, and STEM concept gains. Taken together, these findings support after-school interventions in STEM areas as key encouraging excitement in youth and motivation to pursue a career in a STEM content area.


A Critical Time for STEM Education in After-school Time
There is a great need to fill the science, technology, engineering, and mathematics (STEM) pipeline in the United States. While youth in the United States continue to rank behind youth in other countries in STEM performance, namely, 24 th in science and 39 th in mathematic competencies out of 65 countries (Desilver, 2017), the United States also fails to meet the needs of current employers in terms of skilled laborers in STEM areas. Currently, the state of Nebraska reflects these trends. To address these shortcomings, the Omaha, Nebraska Chamber of Commerce seeks to establish the city of Omaha as a "tech hub" by 2020 with an additional 4,000 jobs (Greater Omaha Chamber of Commerce and AIM/Careerlink, 2015). Moreover, Omaha was recently designated as one of the national "STEM Ecosystem" sites by the STEM Funders Network to build community efforts for bringing youth into the STEM pipeline (Teaching Institute for Excellence in STEM, 2016). Such STEM Ecosystem sites coalesce community efforts for that purpose. Synergistically, Nebraskans are committed to providing high-quality training in STEM areas, and feeding the pipeline for long-term economic stability.
While these goals of the state and nation are laudable, we continue to face challenges both locally and nationally.
Challenges for building a robust STEM community and workforce include: (a) how to maintain the curiosity by youth to pursue STEM areas of study, (b) how to structure a STEM pipeline that takes youth into careers, and (c) how to retain a talented STEM workforce in Nebraska. In this study, we investigated the first of these challenges through an intervention developed and implemented in the Omaha metropolitan area. Specifically, we created an after-school program called NE STEM 4U, in partnership with several local stakeholders including Omaha Public Schools, Collective for Youth, Beyond School Bells, and local funding agencies, (Cutucache, Luhr, Nelson, Grandgenett, & Tapprich, 2016) where we deliver problem-based learning lessons in STEM to elementary and middle school students. Problem-based learning (PBL) is a learnercentered method of teaching that engages students in hands-on activities that require them to solve problems, conjecture, experiment, explore, create, and communicate. The goal of PBL is to engage students in the learning process, enhance their critical thinking skills, and boost their long-term retention of material (Chng 2011;Nicholl & Lou 2012;Salinitri, O'Connell, Garwood, Lehr, & Abdallah, 2012;Wiznia, Korom, Marzuk, Safdieh, & Grafstein, 2012;). It has been shown to help students at all levels to more effectively learn complex concepts, particularly in science and mathematics (Avanzato, 2000;Cusick, 2001).
The value of PBL is widely recognized, and the use of PBL exercises has increased over the past decade. In the biological sciences, however, this teaching approach has been restricted mostly to graduate-level and professional-level education (Husain, 2011;Imans & Wilkerson 2011;Klegeris & Hurren, 2011;Lira & Lopes, 2011;Okubo et al., 2012). Such an active learning strategy adds to students' abilities, including, for example, the ability to apply the process of science and use quantitative reasoning. Through this program, we have reached out to Omaha youth to enhance their awareness and curiosity about STEM areas.
It is well-documented that youth choose whether or not to pursue careers in STEM fields by eighth grade and choose whether they enjoy STEM content areas by as early as fourth grade (Maltese & Tai 2010; President's Council of Advisors on Science and Technology, 2010; Tai, Liu, Maltese, & Fan 2006). The NE STEM 4U program was designed to target youth who are forming their opinions about STEM areas and STEM careers. Our data (and that of others: Schnittka, Evans, Won, & Drape, 2015;Cutucache, Luhr, Nelson, Grandgenett, & Tapprich, 2016) show that youth participating in after-school programs like NE STEM 4U demonstrate increased content knowledge, as well as improved problem-solving skills. Youth who participate in out-ofschool time (OST) STEM activities are also more interested in STEM disciplines in college (Dabney et al., 2011). Such interests coupled with learning gains will contribute to career competitiveness for youth that choose to pursue a career in a STEM area. NE STEM 4U provides a model that can be replicated by other metropolitan universities to increase STEM awareness and curiosity in youth nationwide.
After-school STEM experiences have broad positive impacts on youth development. For example, after-school programs that build upon the school day improve academic performance, attendance rates, social skills, and STEM interest (Carpenter, 2015;Constan & Spicer, 2015;Lauer et al., 2006;Miller, 2003;Tyler-Wood, Ellison, Lim, & Periathiruvadi, 2012). Learning gains that result from exposure to after-school STEM at the elementary level also promote youth preparedness for the transition to middle school STEM curricula (Moreno, Tharp, Vogt, Newell, & Burnett, 2016). Relatedly, after-school programs at various academic levels show potential long-term effects on perceptions of science careers, STEM technical skills, and understanding of such skills in STEM careers (Duran, Höft, Lawson, Medjahed, & Orady, 2014;Tyler-Wood et al., 2012;). Importantly, the relationship between quantitative ability and STEM interest is relatively stable by middle school, develops concurrently thereafter through the undergraduate years, and is positively related to the likelihood of an individual to earn a degree in a STEM major (Le & Robbins, 2016). Students who participate in expanded learning opportunities are also 20% less likely to drop out of school and 30% less likely to participate in criminal activities than peers who did not participate (Moeller & Reitzes, 2011;Vandell, Reisner, & Pierce, 2007). While studies among middle and high school students often demonstrate significant positive effects of out-of-school time (OST) STEM programs on youth interest in STEM, data about how similar programs impact elementary students' interest in STEM are lacking (Young, Ortiz, & Young, 2017). Existing after-school programs that provide such opportunities to youth can utilize various pedagogical methods, such as inquiry-based learning (IBL), project-based learning, and peer-led team learning (Alexander, 2000). We incorporated problem-based learning to elementary youth in this intervention (Cutucache, Luhr, Nelson, Grandgenett, & Tapprich, 2016).

Participants
Since it was founded in 2012 and first implemented in 2013, the NE STEM 4U program has reached between seven and 11 schools at the middle and elementary level, per semester. At the time of this writing the program has served approximately 8,000 youth. We work with citywide agencies, social workers, a state agency, and the school district administration to identify schools to participate in the program. Selection of schools is based upon low performance on Nebraska standardized assessments as well as high free and reduced lunch rates (Omaha Public Schools, 2014). NE STEM 4U was initially focused on middle schools. However, in an effort to impact a broader range of age groups as they form their opinions about STEM, the program recently expanded to include a small subset of four elementary schools. This study focused on evaluating the program in these four elementary schools, with 114 participants (57.9% males, 42.1% female) in grades K-6, from spring 2015 to spring 2016. All data were collected and analyzed under the IRB-approved protocol. Program participation among all schools is primarily composed of students in low income categories. These elementary schools meet the same selection criteria as middle schools. Moreover, each of the four elementary schools is part of the Nebraska 21 st Century Community Learning Centers (21 st CCLC), a program that supports expanded learning opportunities in schools with more than 40% low-income students (eligible for free/reduced lunch). The overall student body at the four participating sites is largely composed of minority groups where more than 88.8% of students qualify for free or reduced lunch (Table A1).
The NE STEM 4U program serves students across K-16 by providing STEM learning opportunities for K-12 students, and teaching and mentorship experience for the undergraduate and graduate fellows who develop and deliver STEM learning lessons during each site visit. One high school mentor has participated as well, and that student subsequently attended University of Nebraska, Omaha and continued to work in the program. Mentors include both males and females of various ethnicities. Undergraduate and graduate mentors initially complete a facultysupervised orientation, training, and shadowing, and subsequently commit to providing a minimum of 3 hours per week of mentorship, lesson development, and teaching with the program. For each site visit included in this study, lessons were led by two to three mentors, except in one case where only one mentor was available.

Activity title
Exploring Gases

Description
This lesson includes a chemical reaction with baking soda and vinegar, as well as a physical reaction with Diet Coke ® and Mentos™.

Objectives
To demonstrate the difference between a chemical reaction and physical reaction.

Materials
Pop bottles, vinegar, baking soda, water, balloons, paper for funnels, diet cola, club soda, mint Mentos™, a golf ball, and a clear cup.

Space requirements
Classroom and outdoor area.

Activity
The group will take observations of a chemical reaction by mixing baking soda with vinegar in a bottle topped with a balloon. They will observe the formation of carbon dioxide as it fills the balloon up. Next, they will observe a physical reaction by mixing Mentos™ with Diet Coke ® . The students will compare their observations of the reactions and discuss the similarities and differences.
Number of staff 1-3

hour
Ages/grade span K-8 grades

Skills addressed
Scientific skills: taking observations, mixing chemicals together safely

Extensions (if applicable)
Work cooperatively in a group, communicate effectively, set goals

Documentation & reflections
The benefits of this lesson include performing scientific skills such as making Consequently, youth voice is then included in our data analysis using direct quotes from the participants. The authors used these comments as qualitative evidence of student gains resulting from the NE STEM 4U intervention.

Dimensions of Success Observation Tool
The Consequently, youth voice is then included in our data analysis using direct quotes from the participants. The authors used these comments as qualitative evidence of student gains resulting from the NE STEM 4U intervention.

Results of Classroom Observations using the Dimensions of Success Tool
A total of nine observations of NE STEM 4U lessons were conducted among four elementary schools from spring 2015 to spring 2016. Observation days and times were randomized in order to avoid bias from any particular lesson, instructor, or preparation time by instructor. Average DoS scores among observations were greater than or equal to 3 in all but one category, Relevance, which received an average score of 2.7. Scores were highest (greater than 3.5), on average, in five major categories, including Space Utilization, Relationships, Materials, Organization, and Youth Voice (Table 1). Further, observer comments on DoS evaluation sheets provided qualitative evidence of positive impacts of the NE STEM 4U program on elementary school students' excitement, curiosity/inquiry, and understanding of STEM concepts.

Average Range
Features of the learning environment Organization 3.6 3-4  Students had retained much of the information from previous activities, their favorite being the electrical current with the apples and bolts.
Under "Participation," observers often noted that students were actively engaging/participating in the activity, following directions, and completing the tasks. Additional observations indicated that students engaged in the hands-on activities and active exploration of STEM content, and that materials were "appealing and kept their attention." In two cases where the activity scored a 4 in the Relationship category, the evaluator noted that, "The students responded in a positive manner," and, "were listening and following directions while having the freedom to move around," regarding the interactions of the students with mentors.
Student curiosity/inquiry. In addition to increasing youth interest in science, the NE STEM 4U program seeks to encourage students to think critically and develop questions in preparation for or in response to the STEM topics to which they are exposed. DoS evaluations with scores of 3 or greater in the "Inquiry" category noted student inquiry in the form of making observations, asking questions, performing experiments, modeling peer sharing/feedback, and drawing conclusions, as evidenced by the examples listed below:  During a Mentos™ and cola activity, students asked questions (e.g, "What if we added the whole package?"), carried out investigations, justified their hypothesis and shared their findings with each other.
 Making observations as they each tried to manipulate the arm model; asking questions as they were building the model; developing a model as they drew their design on the board; and sharing their findings with peers. At the conclusion of the class, [the mentor] asked each student to share what they learned. …One student commented, "Our model could be used for other parts of our body like our legs. They both have muscles to help us move.
 As each student tried an experiment, the friends would give advice to help them be successful in the challenge.
Furthermore, an external assessment of our program (The PEAR Institute & Institute for Measurement, Methodology, Analysis & Policy, 2016) demonstrated that 78% of students reported that they had a more positive attitude about STEM because of their after-school experience, 73% had a STEM identity, and 80% indicated that their STEM career knowledge increased because of their after-school experience program. Additional specific highlights of our programming and impact on Nebraska's economy and expanding learning opportunities across Nebraska can be found in Cole, Sigmon, and Wishart (2017).
When any dimension received a rating of a 2 or less (n = 6 across all reports), evaluator comments noted weak evidence to support fulfillment of that category and/or strategies tending toward less authentic, cookbook, and/or staff-lead activities. For example:  In the Inquiry category, a score of 2 (n = 2) yielded comments such as: "Students engaged in scientific practice, but in a more inauthentic manner. The activities were more of a cookbook nature that limited the student reasoning and thinking. They conducted the balloon/bottle experiment and made an observation, but it seemed the activities were [more] staff-led than students conducting authentic research practices"; and "The activity with building a new person was more of a step-by-step activity where staff was leading and students were following."  In the Reflection category, scores of 2 (n = 2) yielded comments such as: "There was weak evidence that students were engaged in reflection," and, "[The mentors] would ask students questions to engage them in conversation, however, majority of students were paraphrasing or repeating what staff had just said."  In the Relevance category, a score of 1 (n = 2) yielded comments such as: "There was not any evidence that the activities were made relevant beyond the activity itself." Student understanding of STEM concepts. A main goal and critical part of the mission of NE STEM 4U is to increase understanding in STEM topics among socioeconomically disadvantaged students. Thus, we reviewed DoS evaluations for potential evidence suggesting that program activities promoted these learning gains. All DoS evaluations showed a score of 3 or higher in the category of STEM Content Learning. Further, examples taken from a broader range of categories, including Purposeful Activities, Reflection, Engagement with STEM, and Relevance also suggested increased student understanding of concepts related to the activity (Table 2). related it to a bicycle (2 wheels), students were able to conclude, "So the muscle has two parts?" Students also applied what they learned when drawing an arm model on the board.
Mentor: "What would happen if we put a small piece of paper in the hole?" Student: "It will make more friction. Maybe even so hot it will make a fire." When asked where the nucleus of the cell was, one student answered, "It can't be outside the cell because it acts as its brain and powers the cell." able to apply what they learned, for example, Mentor: "What happens when you rub your hands together?" Students: "They get hot." Mentor: "What do we call that?" Students: "Friction."

Reflection
[The mentor] asked, "What happens when you cover your mouth?" "What then happens if you don't get any air?" Students were able to apply the knowledge of air being important to the lungs; expand, contracting when they take in air.
Mentor: "Think about your tiny little hole in your microscope. What could you see?" Student: "When I looked at things, they got bigger." When discussing the lysosomes, (blue shark candy) she said "think about what a shark might do if it lived in your cell?" One student mentioned, "It would eat it up." [The mentor] "Exactly, so the job of the shark (lysosome) is to eat the bacteria in the cell." Following the balloon/bottle experiment, [the mentor] asked students, "Whose balloon is the largest?
What made one balloon blow up more than someone else's?" One student replied, "He added more baking soda?" She then asked, "What else could have caused his to get bigger?" Another student answered, "The water level, it's stronger." As they left, [the mentor] asked them to say one thing they learned today. Majority of students were able to list something related to the activities. Mentor: "What did it create?" [(same) Student]: "We made a gas that blew up the balloon." [The mentor], talking about muscles, asked students if they knew why it was called biceps. When he related it to a bicycle (2 wheels), students were able to conclude, "So the muscle has two parts?" Students also applied what they learned when drawing an arm model on the board.
Students were able to become actively engaged with their minds as they tried each of the experiments. This was evident in the heat conversion activity, when [the mentor] asked, "We talked about how friction causes heat. How do we know?" Students answered, "It gets hot." He followed up with, "How do we measure that heat?" They answered, "A thermometer."

Relevance
Mentor: "What can we use a microscope for?" Student: "To look at germs that make us sick."  Table 2. One of the major limitations we identified, as previously mentioned in the methods section, is that the instructor can influence student learning outcomes and interest, even when learning objectives, lesson plan, prior training, and delivery of information are all normalized. Similarly, students may identify with a mentor of a particular age, race, or gender more than another, thus influencing their engagement in the program. We cannot account for these preferences, nor any differences in instructor charisma or other personality traits, but these are partly mitigated by the fact that mentoring is done in teams of mentors.

Discussion
These results emphasize that NE STEM 4U is a highly effective STEM learning model for elementary school students, which maximizes the learning environment with smooth delivery of STEM learning activities and use of appropriate materials to enhance engagement and understanding of STEM concepts. Further, undergraduate and graduate mentors build rapport with students via one-on-one interactions, thus encouraging them to share their perspectives and make meaningful scientific decisions. Importantly, this intervention has led to improved curiosity, inquiry, and scientific thinking among youth in elementary schools. Sustained partnerships and high quality after-school programs help to build and maintain the STEM pipeline. The NE STEM 4U program serves as a model for other metropolitan universities to recruit youth into the STEM pipeline by raising STEM awareness and curiosity during a highly influential time in their development.