Pairing Educational Robotics with Geospatial Technologies in Informal Learning Environments

Educational robotics, when paired with geospatial technologies and taught in an informal educational environment, can be an innovative strategy to teach youth about science, technology, engineering, and mathematic (STEM) concepts. However, little is known about the true effects on conceptual knowledge and associated attitudes. Therefore, this study was conducted to examine the shortterm effects of a series of five-day summer robotics/geospatial camps held in Nebraska. The study was conducted at six diverse locations and consisted of a five-day 4-H camp experience. The study examined the experiences of 147 youth between the ages of 10 and 15. A pretestposttest quasi-experimental design was used in the study. Instrumentation consisted of a 37-question multiple-choice assessment targeting various STEM topics and a 38-question attitude questionnaire assessing STEM interests and attitudes. Results suggest that the 4-H robotics and geospatial summer camp program is a promising approach for supporting STEM-related learning and enhancing attitudes towards STEM.


Introduction
The Nebraska 4-H program, with grant funding from the National Science Foundation, is developing a program to increase science, technology, engineering and mathematics (STEM) achievement and interest using robotics and geospatial technologies.The widespread availability of education robotic platforms such as the LEGO NXT Mindstrom kit, handheld Global Positioning System (GPS) receivers, and geographical information systems (GIS) software like GoogleEarth and ArcMap make it possible for youth (ages 10 to 15) to explore the integration of these technologies.The curriculum was developed by Nebraska 4-H and faculty from University of Nebraska's Biological Systems Engineering Department, in cooperation with Carnegie Mellon University's Robotics Academy which involves 40 hours of instruction.The 40-hour summer camp activities include the building and programming of robots, exploring handheld GPS receivers while collecting georeferenced information and navigating outdoors, and the development of GIS maps.The camp activities were led by project staff and in some cases faculty from the University of Nebraska.The content and context for the activities were delivered in a short introductory lecture format followed by hands-on activities.Two formal STEM-related assessments were also administered, one related to conceptual learning and one related to attitudes.The participating youth also shared their general perceptions of the activities using a short feedback survey.
Research in the use of educational robotics in an informal learning environment implies that robotics can increase academic achievement in specific STEM concept areas closely aligned with formal education topics and coursework (Barker & Ansorge, 2007;Barker, Nugent, Grandgenett, & Hampton, 2008;Nourbakhsh et al. 2005).Similarly, past research has indicated that GIS can be used to teach project-based science, environmental education and geography concepts to middle school students (McWilliams & Rooney, 1997).Research also suggests that the use of GIS helps in the development of analytical skills and problem solving (Wanner & Kerski, 1999).Moreover, there is growing interest in examining students' attitudes towards learning, with recognition that affect surrounds cognition and can moderate learners' conceptual change (Alsop & Watts, 2003;Koballa & Glynn, 2007).Measuring a student's attitude is not a trivial matter; and much of the robotics literature looking at attitudes relies heavily on subjective secondary observation.For example, Rogers and Portsmore (2004) reported that using robotics as an outreach activity in elementary schools increased confidence and interest in mathematics and science.This conclusion was based on teacher perceptions; the researchers did not directly gather any data from individual youth participants.
There are several instruments that have been developed to assess youth attitudes within science-related contexts.The most widely used is arguably the Scientific Attitude Inventory (SAI) Moore and Sutman (1970).The SAI (I) consisted of a 60 item, four-point Likert-type scale with a series of 12 statements of attitude called "position statements."These twelve position statements assess six scientific attitudes -three based on intellectual attitudes and three based on emotional attitudes.The six attitudes include: 1) laws of science, 2) scientific explanation, 3) manner of scientific observation, 4) value of scientific activities, 5) usefulness of science to society and 6) student career aspirations.Each scientific attitude has a positive and a negative scale to create the 12 position statements used as potentially measurable constructs.Reliability was measured through the use of the Winer test-retest method using the pre and posttest scores of the control group resulting in a test-retest reliability coefficient of .934(Moore & Sutman, 1970).Based on the lack of significance using the SAI and other instruments when piloting the project, we elected to develop our own instrument based on specific constructs that originated from our 4-H robotics and GPS/GIS program.Our instrument measures eight scales including: task values in science, mathematics, robotics, and GPS/GIS, problem solving/critical thinking, cooperative learning/teamwork, self efficacy in robotics, and self efficacy in GIS/GPS.

Purpose and Methodology
The purpose of this study was to investigate the short-term impacts of informal summer programs centered on robotics and geospatial technologies in a) promoting STEM learning for youth ages 10-15 and b) positively impacting their attitudes towards STEM.From June to July 2008, 147 youth (112 males and 35 females) in six different camps participated in the summer program.Seventy-five percent of the youth participants self-identified as themselves as Caucasian, 12% as African American, 12% as Hispanic and 1% Asian.The overall mean age for the camps was 12.28 years.

Instrumentation
The instrumentation used for the study consisted of two parts.To measure STEM learning, the project staff developed a 37-item, paper-and-pencil, multiple-choice assessment, covering a variety of topics including computer programming, mathematics, geospatial concepts and engineering/robotics.The assessment was based on a previous 24-item robotics assessment instrument that demonstrated a Cronbach's alpha reliability coefficient of 0.86 (Barker & Ansorge, 2007).Two experts from Carnegie Mellon University's Robotics Academy and two engineers from the University of Nebraska-Lincoln Department of Biological Systems Engineering reviewed and validated the assessment instrument's content.The overall Cronbach's alpha reliability coefficient of 0.80 was reported for this instrument.
The attitude instrument was also developed by the project staff and was modeled after the Motivated Strategies for Learning Questionnaire (Pintrich, et al., 1991).The questionnaire focuses on the following eight constructs: task value for science, mathematics, robotics and GPS/GIS, problem solving/critical thinking, teamwork cooperative learning/teamwork, selfefficacy in robotics and self-efficacy in GPS/GIS.The task value for science included questions like "It is important to me to learn how to conduct a scientific investigation."The mathematics task value construct included questions like "It is important for me to learn how to make accurate measurements to help solve mathematical problems."The robotics construct asked questions like "It is important for me to learn about robotics."The GPS/GIS construct included questions like "It is important for me to learn about GIS."In addition, problem solving skills (i.e."I try new methods to solve a problem when one does not work") and teamwork constructs (i.e."I like being part of a team that is trying to solve a problem") were also explored.Finally the instrument examined self-efficacy in robotics (i.e."I am confident that I can program a LEGO robot to follow a black line using a light sensor") and GPS/GIS concepts (i.e."I am certain that I can log locations of a series of waypoints within a GPS unit").The overall Cronbach's alpha reliability coefficient of 0.94 was reported for this administration of the post attitudinal instrument.

Data Collection
The pretest was administered on the first day of the camp prior to the start of program activities.The posttest was administered on the morning of the last day of camp.Administration of the pretest-posttest assessment was conducted in the same manner for each camp.There was no review or targeted considerations of any items included in the assessment instruments between the pre-and posttests.

Data Analysis Procedures
The study used a pretest-posttest quasi-experimental design, with the same assessment acting as both a pretest and posttest in each summer campsite.The learning assessment used a total score for the number of items correct.The primary analysis was a repeated measures t-test for the combined groups by location.The attitudinal instrument used a five-point Likert-type scale with five equaling "strongly agree" and one equaling "strongly disagree."A total score was calculated by summing all 38 items and comparing them using a repeated measures t-test for the entire group and by location.

Results
Learning: On average, youth scored significantly better on the posttest (M = 20.12,SD = 5.60), than the pretest (M = 15.63,SD = 4.52), t(136) = 13.71,p < .001.The results of the repeat measures t-test support the conclusion that the robotics and GPS/GIS summer program supports SET learning as measured by the content test.Results of the pre and posttest sore by site are displayed in Table 1.In addition, a one-way analysis of variance was conducted to determine if there was a significant difference in posttest scores by location.Because the assumption of homogeneity of variance was violated the Brown-Forsythe F-ratio is reported.Results indicate a significant difference in posttest scores by location F(5,59.56)= 32, p < .001.Post hoc comparisons using the Dunnett C test indicated significant differences between the North Omaha camp (M=10.67,SD = 2.83) and the other five camps.These results indicate that the North Omaha location did not perform as well on the posttest as the other campsites.
Overall, these results suggest that the 4-H robotics and the geospatial summer camp program is a promising approach for supporting STEM-related learning.To get a better understanding of how individual sites scored on the test; the mean scores are separated by location.See Figure 1.The sites are listed in chronological order with the Omaha North camp first and ending with the Grand Island camp six weeks later.

Discussions and Conclusions
The significant increase in student scores on the learning assessment provides evidence for the use of robotics and GIS/GPS technologies as a means to promote short-term STEM learning.
With one exception (Omaha North), all the sites produced short-term knowledge gains from pre to posttest.One plausible explanation for the lack of improvement at the Omaha North site is that chronologically it was the first camp run by the project staff.Therefore, activities and presentation methods were still relatively new, and were still being tested and refined.This can be supported by the apparent increase of the mean paired difference between post and pretest for camps that occurred later in the summer.Another difference with the Omaha North site is that it had a lower mean score on the pretest (M=10.80,SD = 3.22) compared to other sites.The lower pretest score may indicate that this particular group of youth did not have as much initial experience and therefore, prior knowledge of robotics and geospatial concepts as other groups, perhaps suggesting that at least a minimal level of initial understanding of these topics is needed for students to be fully successful with this level of activities.
Documenting the positive impacts of robotics and GPS/GIS activities on student's attitudes has been a struggle in past research (Nugent, Barker, & Grandgenett, 2008).Prior to this study the project team piloted two other existing attitude instruments (Scientific Attitude Inventory, Moore & Foy, 1997;Pell & Jarvis, 2001) with nonsignificant pre to post comparisons.Past results suggest that youth have a difficult time in making the connection between STEM concepts and Robotics and GPS/GIS activities.When robotics and GPS/GIS are embedded into a natural experiential learning environment, as opposed to the more traditional direct instruction STEM learning environment, students may become excited about robotic and GPS/GIS, but not recognize that STEM learning is actually being integrated into the activities.
Results have led to curricular revisions, including specific instruction on how specific robotics activities relate to science, engineering, math and technology and the creation of a new attitudinal measurement tool.Additional research is needed to more fully examine each of the eight attitudinal constructs and to examine various trends and the potential interactions of these constructs with various participant demographics.
Overall, results suggest that educational robotics paired with geospatial technologies can positively impact youth conceptual STEM knowledge and associated attitudes.The middle school years are a critical time to foster improved STEM perceptions and attitudes and to lay the groundwork for more advanced math and science courses in high school.The use of robotics and geospatial technologies in an informal educational environment appears to be an effective way to support youth STEM cognitive and attitudinal development.

Figure 1 Posttest
Figure 1 Posttest Means by Location

Figure 2 Attitudinal
Figure 2Attitudinal Posttest Means by Location

Table 1
Descriptive Statistics Pre and Posttest

Table 2
Descriptive Statistics Pre and PosttestIn addition, a one-way analysis of variance was conducted to determine if there was a significant difference in posttest scores by location.Results indicate no significant difference in posttest scores by location F(5,131) =0.80, p =0.55.Overall, these results suggest that the 4-H robotics and the geospatial summer camp program is a promising approach for supporting STEM-related learning.To get a better understanding of how individual sites scored on the test; the mean scores are separated by location.See Figure2.