4-H Science : Evaluating Across Sites to Critically Examine Training of Adult Facilitators

As 4-H Youth Development focuses on developing and delivering high quality STEM learning experiences, the issues related to the preparation of the adults who facilitate learning with youth are important to address. This paper outlines a five-state pilot project funded by the 3M Foundation to test a model for training adult facilitators. The findings from this study raise questions about how non-formal educational programs involve and mobilize adult facilitators to work with youth in STEM-related learning when the emphasis is not only on engaging young people, but also on deepening their thinking and learning about engineering phenomena, in this case wind energy. Evidence from the process evaluation illustrates the extent to which three train-the-trainer applications incorporated the original educational design, surfacing questions about how to design high quality, yet practical, training applications within 4-H.


Introduction
As evidence mounts about the potential power of out-of-school educational experiences to increase young people's engagement and competence with science, technology, engineering, and mathematics (STEM) content (Bell, 2009), it is becoming critical to understand effective training approaches for preparing the adults who work with youth in these settings.This paper examines the application of a train-the-trainer approach for preparing volunteer adult facilitators to work with youth in 4-H program settings with design challenges to understand wind energy.The project is used as an example of a larger issue facing youth development as a field, and to make the case for the importance of critically examining how adults are prepared to work with youth in the STEM or 4-H Science arena.
The call to involve one million new young people in 4-H science programs is driven, in part, by trends seen in both the engineering and life science fields related to declining workforces, and the need to replace and increase the current workforce with younger workers.Nationally, the U.S. is investing to build the skills and interests of young people for those fields with an array of strategies, some of which are delivered beyond the traditional classroom in order to reach youth audiences who are not drawn to engineering or science in school.In addition, non-formal education reaches young people who have a high interest in a field and who seek a variety of learning experiences in a particular domain, both in school and outside of school.
Non-formal education is defined as "any organized educational activity outside the established formal system -whether operating separately or as an important feature of some broader activity -that is intended to serve identifiable learning clienteles and learning objectives" (Coombs, Prosser, & Ahmed, 1973).The key term in this definition is "intended," which distinguishes non-formal from informal education, (defined as everyday learning that happens across experiences), by the intentionality of its learning objectives.However, there are neither widely accepted nor clear distinctions between the characteristics of formal, non-formal, and informal education.Most noteworthy is the notion that non-formal education inhabits the "middle" of a conceptual continuum with a matriculating, standardized system of formal education on one end, and the lifelong learning from everyday experience that characterizes informal education at the other end.
This study tested the use of inquiry-based learning methods in 4-H clubs and programming based on the curriculum Power of the wind (Sebestik, 2008) a national 4-H curriculum released in 2008 and designed for youth ages 11-13 years.The curriculum is designed for application in a variety or blend of formal, non-formal, or informal settings.Learning objectives for the curriculum include increasing knowledge about wind energy concepts and increasing understanding of the engineering principles related to wind energy power (Sebestik, 2008).Minnesota 4-H Youth Development extension educators trained adult "lead trainers" in learning methods based on inquiry-based process skills related to questioning and investigating.Lead trainers applied the training to their respective state's programmatic needs and, in turn, trained adult facilitators in their states to work with youth.
The replication of training models through train-the-trainer approaches can be a challenging goal when applied in community-based youth programs that tend to be highly heterogeneous.The train-the-trainer approach, however, has a magnifier effect through training more adult facilitators who are prepared to involve more youth in the project.For these reasons, the model was designed to be applied within each state by the lead trainers with adjustments to fit the training and program to the organization and its sites.The training of adult facilitators was a common thread across the sites, as was the use of the written curriculum.This evaluation study examined the extent to which the trainings provided by lead trainers incorporated the inquiry-based learning methods by comparing three variations of trainings based on similar content and methods.

Study Questions
What similarities and differences occur across sites in the trainings of adult facilitators of nonformal education engineering programs?1.How do lead trainers apply the original training model designed for the pilot? 2. How were inquiry and challenge design methods, as defined and integrated into the lead trainer training, applied by lead trainers in the trainings for adult facilitators?

Training Approach
The framework for the study is the pedagogy and structure used to initially train the lead trainers from each state.The lead trainer training was designed by Katie Clarke at the University of Minnesota using learning methods adapted from the Exploratorium Institute for Inquiry© and applying them to the material from the Power of the wind Facilitator's Guide and Youth Guide (Sebestik, 2008).The original training was offered over a fifteen hour period and combined presentation/discussion with extended time dedicated to hands-on experiences in participant teams.The training built on lessons in the curriculum guides, but adapted them to demonstrate a range of instructional approaches that deliberately varied the amount of facilitator and learner control.Segments of the training were also dedicated to reflection on learning and applying the methods with youth.The use of questioning and investigating skills were emphasized throughout all aspects of the training.
The training was designed to build adults' skills to question and investigate through design challenges and then to apply these methods in programs to engage youth in the wind energy content.Specifically, the model for the project and the training emphasized instructional strategies to: • Encourage reflection about learning, • Foster cooperation and teamwork, • Enable youth to use inquiry-based strategies to form questions, plan and conduct investigations to reach higher order thinking skills, • Promote the belief that everyone can learn science.
The differences expected for the youth participants were directly connected to the extent that the adult facilitators are prepared to facilitate reflection, cooperation, teamwork, and inquiry based learning.

Application of the Training Model
An application model (as opposed to an expectation for direct transfer of training or strict fidelity of implementation) "brings into focus how principles, ideas, or theories interface with what constitutes the practical in different contexts" and involves negotiation between concepts and context (Ottoson, 1997, p.91).Application of ideas involves filtering, with the anticipation that ideas will look differently as they come into practical contact."Assessment based on application must deal with the translation and mutual adaptation of ideas and context over time.We are no longer looking for mirror images; we are looking for the essence of an idea" (p.92).The essence, or consistent core, in the Power of the wind study was the use of the curriculum lessons through methods that would increase adult facilitators' effective application of questioning and investigation skills in the context of 4-H programs.The engineering education field is well-positioned to partner with large youth organizations to incorporate the questioning and investigation methods.An analysis of the successes and setbacks of "new" applications of non-formal engineering learning provides critical information about how these partnerships can benefit youth.
Through this application lens, the study is not focused on tracking the replication of the original model.Rather, the study is weighted toward understanding the filtering, translation, and adaptation of the model by lead trainers.This study examines the presence of the essential elements of the training designed to prepare adults to generate questions, investigate phenomena related to wind energy, share findings, reflect on what was learned through conducting design challenges and work in teams.

Overview of Supporting Literature
Non-formal STEM education for youth Non-formal STEM learning is critical for both, increasing the science literacy of citizens and increasing the numbers of young people who consider and pursue STEM careers.Landmark research supports the critical role of engagement in afterschool STEM educational opportunities as a predictor of interest in STEM fields (as opposed to high grades in formal science courses but low interest) (Tai, 2006).Furthermore, educators and providers in the field are beginning to clarify the need for adequate STEM education dosage (Afterschool Alliance, 2011).
The amount of time that learners spend in formal learning settings peaks at 18.5% of total hours during the year when youth are in grades 1-12, and the formal learning time decreases as learners move into undergraduate/ postsecondary settings, graduate school, work settings, and retirement.The remainder of the learning "clock" is comprised of non-formal and informal learning."In popular, political, and academic discourse, learning is all too frequently equated with schooling.This widespread conflation privileges the consideration of formal academic outcomes while obscuring the central role of a broad range of everyday capacities and social outcomes now recognized to be associated with a more holistic view of youth development and preparation for life.While academic achievement is essential for the success of most individuals, education efforts writ large -including those associated with formal schooling, after-school programs, community youth programs, and information learning opportunitiesneed to help youth develop across many dimensions" (Banks, et al, 2007, p 18).
If non-formal learning is so important, then what constitutes a quality non-formal learning experience in STEM?The non-formal education field is striving to achieve a balance in the design of learning activities that 1) support in-school science learning, 2) highly engage youth, and 3) are "easy to implement" (Means,et al,p.8).However, it is not enough to provide a series of fun activities for young people outside of school and to expect rich, meaningful nonformal learning."Something" is not necessarily better than "nothing" when it comes to nonformal learning programs for youth.Young people who participate in no programs have better outcomes than youth who participate in low quality non-formal learning programs (Belle, 1999).

Inquiry
The scientific method can be practiced by learners through the scientific inquiry process.The National Science Education Standards (1996) define inquiry this way: "Inquiry is a set of interrelated processes by which scientists and students pose questions about the natural world and investigate phenomena."Inquiry consists of a set of skills or processes, including but not limited to questioning, observing, predicting, experimenting, explaining, comparing models, inferring/concluding, reasoning logically, and communicating ideas to others.The National Science Education Standards refer to scientific inquiry-based teaching as the use of methods that promote "the activities of students in which they develop knowledge and understandings of scientific ideas, as well as an understanding of how scientists study the natural world."Simply put, it is a pragmatic pedagogy used to help learners to understand and practice how scientists process their work (NRC, 2000).
In a review of research and evaluation studies of inquiry-based science instruction in formal, informal and non-formal settings (Minner, et al., 2010), an approach was defined as inquirybased if "the instruction was about life, earth or physical science, if it engaged students with scientific phenomena, instructed them in some part of the investigation cycle and used pedagogical practices that emphasized student responsibility for learning or active thinking" (Minner,479).Based on the degree to which learners are engaged in independently asking and answering questions, science education researchers have categorized scientific inquiry instruction by the extent to which the learner chooses their own questions (Colburn, 2000;Windschilt, 2002).

Status of Evaluation of Non-formal STEM Programs
The rigorous goals to 1) advance learning for young people in the content areas of STEM, 2) to build overall science literacy, and 3) to prepare young people for careers in STEM imply that programs must excel at both engaging youth and increasing youth understanding of content and learning processes.
However, the body of research and evaluation related to non-formal STEM programs is weak.
Existing research has bridged across formal and informal/non-formal learning settings, yet it has mainly occurred within the domain of science education."This time (informal/non-formal learning) is unaccounted for in science education research due to the difficulty of evaluating such heterogeneous subjects, learning environments, activities and everyday learning situations" (Gerber, et al, 2001,p.537).In a research synthesis of studies on inquiry-based science instruction conducted from 1984-2002, only six studies out of 138 quantitative and qualitative studies with a range of methodological rigor were conducted in "informal" education settings (non-formal settings were not specified in the study) (Minner, et al., 2010).
Evaluations of non-formal STEM-related programs primarily have been conducted at the individual program level, and have utlilized self-report pre/post-surveys to determine effectiveness of instruction and the learning experience.There is an urgent need to build a body of research-based knowledge that identifies the strategies and practices that make nonformal STEM educational programs both engaging and effective in advancing learning with youth.
Recently, several large youth development organizations and researchers have begun working to further the study of non-formal STEM-related programs.Harvard Family Research Project's review of STEM Out-of-School time programs for girls spotlights six programs; five of which use non-experimental evaluation designs and one of which used a quasi-experimental design (Chun, & Harris, 2011).The review concludes that the limited body of existing research on out-ofschool time programs focused on STEM-related education coexists with a lack of consensus on metrics.
This area of study is at a crossroads as the increasing investment and attention in non-formal STEM-related programs necessitates more rigorous study of program quality.In a newly NSFfunded study of publicly funded afterschool programs in California, SRI International staff has further identified challenges in studying a system of afterschool programs that are typically organized at the community level and that lack the network and geographical scope of formal education settings (Means, et al, 2011).These structural differences make representative sampling in a large study challenging and resource-intensive.

Adult Volunteers as Non-formal STEM Facilitators
In addition to the need to build the body of evidence for the effectiveness of non-formal STEM programs, the scope of understanding about the preparation of the adults who deliver non-formal programs deserves further scrutiny.The high degree of heterogeneity of adults competes with the heterogeneity of the youth participants themselves.An added complexity in many non-formal STEM programs is that programs for youth are often delivered solely, or in partnership with paid staff, by volunteers.As youth-serving organizations build their capacity to provide non-formal STEM programs, a deeper understanding of when to engage and how to prepare adult volunteers to most effectively deliver STEM programs is needed.
Cooperative Extension services in land grant universities, with statewide infrastructures that reach into communities with publicly-funded program areas ranging from community development to youth development, engage large numbers of adult volunteers in the delivery of youth programs.Extension relies on the diffusion of new methods (Ottosmon, 1997) to bridge new learning from research-based land grant universities to the community and to reach more youth with these methods.4-H Youth Development, with a national goal to reach 1 million new young people through STEM-related programs by 2013, relies heavily on volunteers to facilitate youth learning.This study of application seeks to identify effective training models for volunteers with attention to levels of investments in staff expertise and time.
Why is the examination of the training model of adults important now, in the early stages of piloting an approach within a large youth organization?In studying a developing area of practice, a balance must be sought to determine how much and when to invest resources to study program effectiveness.On the one hand, it is important to invest enough resources in the practical aspects of the program (training, materials, staff time, volunteer time) so that it can be successful.On the other hand, it is vital that these investments are wisely made and based on evidence related to the effectiveness of the practice.However, it is not feasible to launch a rigorous study when the model is not yet formulated nor is it justified to insert measures for constructs that are not clearly defined.This study strives to achieve a balance by both, allowing effective practices to emerge and by seeking evidence across the heterogeneity of the sites.

Implementation
Six land-grant University Extension organizations in the U.S. formed a partnership to pilot a non-formal learning program based on wind energy using the Power of the wind curriculum (Sebestik, 2008).The project period, originally intended to span from January 2009-December 2009, was expanded to June 2010.An advisory team of wind energy industry professionals, university faculty, and others formed to guide the project in the early months of the project period.
Lead trainers from each state participated in a two-day training model which was used to base trainings on in each state for adult facilitators who worked directly with youth.Since STEM non-formal learning is new to most 4-H adults facilitators, a blend of existing volunteers and newly-recruited volunteers were included in the project in order to learn more about how to best prepare adults to deliver these types of programs.
In youth development programs that are primarily facilitated by adult volunteers, the success of the learning experiences for youth is largely dependent on the success of the training of adults who bring widely ranging backgrounds to their work with youth.For the purpose of building youth understanding of the engineering process behind wind energy, inquiry-based learning methods were a foundation on which adults were engaged in learning how to ask questions and how to conduct investigations.The original training demonstrated and trained adults to use methods in which the learner directs questions and investigations.Inquiry-based process skills helped to meet a design challenge (e.g."design a better pinwheel") with questions followed by a process of planning, designing, investigating, reporting/communicating learning about phenomena (in this case, wind energy).The designers emphasized with partners that inquirybased learning is not the only, or the best way, to learn about wind energy but was a useful starting place for using the Power of the wind curriculum with youth in 4-H settings.
The inquiry-based learning method is in contrast to preparing adults to provide wind energy "activities" step-by-step from a lesson plan that may give youth experience with materials but may not contribute to a quality learning experience.Engineering instructional models applied in formal classroom settings use engineering design challenges to integrate science inquiry, thus moving a "build it and go" activity to an opportunity to both, think and act like an engineer and a scientist.A 15 hour face-to-face training was provided to lead trainers from across the sites.The objectives, as outlined in the training script, included that participants would: • Learn about their connection to wind energy, • Learn about the set-up and content in the Power of the wind curriculum guide, • Experience different approaches to teaching and learning that vary the amount of learner control, • Identify appropriate applications for different teaching/facilitation styles, • Identify teaching/facilitation methods that demonstrate varying amounts of learner control, • Explore phenomena and generate both investigable and non-investigable questions, • Learn to change non-investigable questions into investigable ones, • Design, implement, and share investigative findings, • Practice and understand the importance of reflection in an investigative process, • View differences in approaches to activities and have an opportunity to adapt a learning experience.
In addition, the grant for the project supported the development and provision of the initial train-the-trainer model to orient and train a group of lead trainers in each state who, in turn, trained adult facilitators in their state to work with youth.Materials (e.g.curriculum guides, wind turbine kits) were provided through the grant to support the implementation of the project in each state.Adult facilitators, with the help of lead trainers, recruited young people to participate in the project.

Methods
In the first phase of the study, a descriptive, process evaluation was conducted to document how the project was implemented in each of the states.Since the training and pilot model were untested and the approaches were relatively novel for the organization and lead trainers, the program evaluation focused on gathering evidence on the extent to which the training approaches effectively prepared adult facilitators.Surveys, conference call meetings, observation forms and program-related documentation were designed to assess the skills and confidence of the lead trainers, and of adult facilitators, and to finally assess the program effectiveness with youth.
The process evaluation methods measured the "surface" of the project, and the study uses methods to examine the content of trainings across three sites.To deepen the understanding of how wind energy content and inquiry processes were applied across the sites, the audio from three video recordings of a common training session across sites was transcribed, coded, and analyzed to measure the extent to which the questioning and investigation methods presented at the initial training were present in the trainings across sites.Specifically, adult facilitators' training related to five essential elements: 1) working with middle school age youth, 2) forming questions, 3) planning/ conducting investigations, 4) the use of the three instructional approaches, and 5) embedding inquiry into the curriculum, which were examined through visual and auditory evidence presented on the recordings of the trainings.
The goal of the study of the recordings was not to rate the quality or the skills of the lead trainers, but to better understand the various contexts for training, the adult facilitator audience, and the application of the methods through the lead trainers.The application of the original training was studied by analyzing three training segments based on the Power of the wind lesson "How can we design a better pinwheel?"(Sebestick, 2008, pp. 16-17).The NVivo 9 data management system was used to facilitate the analysis.

Results
The process evaluation results reported here focus on describing implementation of the project and the self-reported comfort levels with the essential elements of the training model by adults facilitators across the five states in the project.Then, the results from the study of adults who participated in one of three of the training applications are presented.

Process Evaluation
Over the course of the eighteen month project, a variety of project types emerged, ranging from one-time educational events to more intensive day camps.Fifteen lead trainers trained 137 adult facilitators, and 3,353 youth of all grade levels were involved in a range of related programming from one-time events to extended programs in club or group settings.Three hundred fifty-six youth participated in club or group Power of the wind programming.

Description of Lead Trainers
Eleven of the 15 lead trainers, five male and six female, were involved over the 18 month project.Most lead trainers had prior experience working with Cooperative Extension or 4-H.Of the 11 lead trainers, 10 were Extension employees and one was an adult volunteer.Eight lead trainers had eight or more years of experience in Extension.Only one lead trainer was new to Extension or 4-H.Although most lead trainers had prior experience with Extension and/or 4-H, only two had experience working in wind energy, either professionally or personally.One lead trainer reported they had between 1-4 years of experience and the other had between 5-8 years of experience connected to the wind energy industry.
Lead trainers became involved in the project for a variety of reasons.Most noted the importance of youth learning about wind energy, as well as renewable energy in general.Some also felt wind energy was a relevant topic for their audience because of wind farm development in their state.When asked if they would continue their involvement with project activities after the completion of the grant, all reported that they had plans to continue working with youth in non-formal learning related to wind energy in their state.
Reasons lead trainers were involved in the project (n=10): • Wind energy in my state is a great topic because of public awareness and economic impact.Great chance to promote in my state.• We have significant wind farm development in the area that I serve.
• Concern for the future of wind power in my state.
• Because there are many new wind farms being constructed in our area.It was an opportunity for me to learn more about wind power.• Our nation desperately needs to become more reliant on clean, renewable sources of energy.Teaching about the Power of wind is one step in opening up a "world of possibilities" to our youth.• An interest in renewable energy and grant funding.
• I feel it is important for youth to understand wind energy, so that they can be knowledgeable about it for future use because it is a great resource.• To gain knowledge to educate youth in the wind industry.
• Encourage creative problem solving and engineering skills in youth, learn more about wind power and the future of energy production, support green energy.• Teaching about and preparing youth to face environmental issues.
While the application of the content of wind energy was a strong fit for most lead trainers, the learning methods related to questioning and investigating were new to most.In fact, one lead trainer communicated to a member of the planning team that the use of questioning and investigation methods within the grant project was unexpected.This individual expected more training about the wind energy industry and equipment itself, and less focus on how to facilitate learning about it.This focus in the project on learning methods set up a tension early in the project about how to best prepare adults.

Lead Trainer Self-Assessment of Readiness
Lead trainers were surveyed immediately after completing the two day training to assess their comfort with the new learning method.Using a Likert scale to rate their "comfort", lead trainers overall reported being comfortable applying the inquiry training with adult facilitators, and helping to guide youth through the inquiry process (See Figure 1).A majority of the individuals at the training had an education background, while other lead trainers did not.This is indicated by the comment from one individual: "Not being a professional educator I am just getting up to speed on some of the vocabulary."Additionally, some individuals needed more practice to increase their sense of preparedness -"Doing it once with adults will increase comfort level."

Figure 1
Lead Trainer Self-Assessments (n=11) The lead trainer role was designed to implement the program and evaluation during the project, and to recruit and train adult facilitators to work directly with youth.Lead trainers gathered evaluation data through videotaping trainings, administering post-training surveys, and providing feedback.After the training, some of the lead trainers observed adult facilitators as they worked with youth, using an observation tool designed for the project.The tool served as both a data collection instrument and a structured guide for lead trainers to provide feedback to adult facilitators.Lead trainers were also involved in monthly project webinars, they designed mini-grant projects, and led some of their own youth activities.

Application of Training with Adult Facilitators
Lead trainers from each state applied the model in their organizational and community settings by training groups of adult facilitators.Overall, 137 adult facilitators were trained over the course of the project.Most adult facilitators across the states reported that they were prepared to implement the essential elements of the model, with less than one-tenth of all adults (4% -11%) reporting that they were uncomfortable with any one of these skills.The adult facilitators' (n=107) reports are documented in Figure 2.

Adult Facilitators' Assessment of Project with Youth
At the end of the 18 month project, adult facilitators were asked to rate project effectiveness in engaging youth in wind energy and inquiry-based learning.Over the 18 month period, 3,353 young people of all grade levels were involved in some aspect of the project, and 356 youth participated in a club or group program based on the project model across five states.Thirty adult facilitators provided ratings on project effectiveness with youth.Most (90%) agreed that Power of the wind was effective in making youth more curious about wind energy.From the adults' perspective, the youth were most effective in being able to explain wind energy concepts to another person.Over three quarters of respondents agreed that youth they worked with were more confident asking questions, planning an investigation and carrying out an investigation after their involvement.Nearly one quarter of adult facilitators surveyed disagreed that youth seemed more confident in planning an investigation as a result of their involvement in the project.

Figure 3
Youth Abilities in 4-H Science from Adult Perspective (n=30)

3%
(1) 13% ( 4) Because of Power of the wind, the youth I worked with seem more confident planning an investigation.

Three Trainings: Compare and Contrast
The usefulness of a process evaluation can be increased when design applications are examined more closely to understand differences.A key program design application in this project involved the partner's trainings, modeled after the initial training to fit the state organization's needs.Design choices for non-formal engineering programs have implications for program effectiveness and the ability of the program to promote youth learning, particularly with the new-to-4-H territory of training adults to work with engineering content and to use inquirybased methods.While the survey method used indicates that adults reported adequate comfort with the applying the essential elements, and suggest that youth had opportunities to learn in the project, the results were mixed and based on self-assessments of participants.The evaluation needs a deeper level of analysis to reveal needed improvements to the project training model.
Figure 4 compares and contrasts three applications of the training with adult facilitators that occurred in three of the partner states.These trainings were selected by lead trainers to videotape in order to archive promising practices that emerged through the applications.The recordings are also useful for narrative content analysis, to understand exactly how the trainings were structured, and how adult facilitators responded and interacted with the training models.

Figure 4
Three Training Applications

Adult Facilitators in Three Trainings
The self-reported comfort levels of the twenty-eight adult facilitators who participated in one of the three trainings are shown in Figure 5.The self-assessments of readiness to use the essential elements of the training model, while not an exact match, are reasonably similar to the overall results shown in Figure 2 for all 107 adults trained.

Figure 5
Adult Facilitators in Three Trainings Self-Assessments (n=28) How would you rate your comfort level doing the following…..?

Comfortable
Working with middle school age youth to learn about wind energy.

7% (2)
4% (1) 50% ( 14) 39% ( 11) Using the three approaches of teaching when working with youth.0% 4% (1) 46% ( 13) 50% ( 14) Guiding youth as they form questions that can be investigated.0% 7% (2) 43% ( 12) 50% ( 14) Guiding youth as they design an investigation to answer a question.0% 7% (2) 44% ( 12) 49% ( 14) Finding ways to embed inquiry into the Power of the wind curriculum.0% 18% (5) 39% ( 11) 43% ( 12) Adult facilitators' reports of not being ready or comfortable to work with youth using the essential elements of the training model are important to address.In addition to the implications for the quality of the learning experience for youth, these reports indicate opportunities for improvement and for building understanding about how to best prepare and involve adults as facilitators of non-formal learning of engineering content and processes.Figure 6 illustrates the distribution of "somewhat uncomfortable" and "uncomfortable" ratings across each of the three trainings.Despite the small and varying sample sizes of the three trainings, these ratings help to pinpoint patterns in self-assessments.

Figure 6
Adult Facilitators "Not Comfortable" with Essential Elements The highest numbers of adult facilitators who reported not being comfortable were in the essential elements of 1) working with middle school age youth to learn about wind energy (3 respondents), and 2) embedding inquiry into the curriculum (5 respondents).

Narrative from Three Trainings
A lesson from the Power of the wind curriculum (How can we build a better pinwheel?)was used across the three trainings and was selected for analysis.The essential elements (Figure 6) of the training model were used as a coding scheme.The author took a deductive approach to coding narrative in the tape that most closely represented one, or none, of the essential elements.Videotape audio was transcribed verbatim, then coded.The coding of narrative was verified with two external reviewers.In addition, the videotaped trainings were analyzed for the total minutes spent with adult facilitators/trainees working in teams, physically conducting their investigations, and sharing or reflection with the large group.NVivo 9.0 was used to manage and graph the coding for the narrative.
The results from the narrative analysis of the five essential elements of training are compared in

Figure 7
Narrative Use in Adult Facilitator Trainings Given that each application varied widely in the structure and length, Figure 7

Implications
In this current study, a new training model was introduced across five organizations within the context of the 4-H Youth Development program with a goal to prepare adult facilitators to work with youth to explore wind energy concepts using inquiry-based methods.It is assumed that the adults who stepped forward to work with youth were all considered novices in the sense that they were new to wind energy as a content area, were new to facilitating inquiry-based learning, or were new to working with youth in non-formal settings.Some adults were most likely novices in all three areas.Since this assumption has important implications for this study, it is important that future similar studies investigate the baseline skill level and experience of adult facilitators to more reliably represent the adult audience as learners and facilitators.
The first phase of the study produced self-report survey responses that were difficult to interpret and subsequently transfer to a deeper understanding about what adult facilitators truly learned.Some adult facilitators felt prepared to work with youth using design challenges to explore wind energy, but some did not feel prepared.This study was based on the belief that the effort spent preparing adults to facilitate non-formal engineering education is as important as the hoped for benefits for youth.In fact, the benefits for youth cannot be expected until the essential elements for the adequate preparation of adults are clear.In the study, partner sites applied and tested models with the anticipation that a strong, yet practical, model could emerge based on the foundations of the initial training.The analysis of training tapes from the pilot suggests that it is important and revealing to study the applications of trainings in a large, multi-site pilot project in order to document implementation and to discover new ways of delivering the training that were not originally thought of.Applications, in which the essence of the idea is carried forward, can result in innovations and improvements to an original design.
The limitations of this study relate to its exploratory and descriptive nature, which did not adequately capture the relationships that may exist between training of adults and the effectiveness of their work with youth.These findings cannot be generalized beyond, or within, this project.The analysis of three of the total twenty trainings for adult and youth facilitators throughout the project period does not offer a representative picture of all of the trainings that were applied.Furthermore, the data were collected in such a way that linking responses from youth participants to the adults they worked with (and the adults' specific training application) was not possible, making it impossible to draw linkages to the trainings for adult facilitators and the adult facilitators' work with youth.

Conclusion
The findings from this study raise questions about how non-formal educational programs involve and mobilize adult facilitators to work with youth in STEM-related learning when the emphasis is not only on engaging young people, but also on deepening their thinking and learning about phenomena.In this case, evidence suggests that three training applications studied more deeply incorporated the essential elements in varying degrees and with varying success.
The three adult audiences in the study were distinctly different from each other and it is likely that the audience played a large factor in how the trainings were applied.Adults were given instruction and experience in forming investigable questions, planning investigations, conducting investigations, working in teams, and reflecting on their results.Through a content analysis, it is possible to more deeply understand how a train-the-trainer model is applied in the community.An intensive two-day training of AmeriCorps members and a one-day training of science or engineering-related professionals offered examples of two different applications that were able to go more deeply and go more broadly across the essential elements of the training model.
Looking beyond the training of adults, future studies of non-formal learning programs in engineering are needed so that educational designers have credible evidence about the protocols and skills needed by adults that connect directly to youth engagement and learning in non-formal settings.Investments can be tailored to the most effective adult recruitment strategies, training design, training dosage, and resources that lead to engaging youth in nonformal engineering learning.

Figure 7 .
Figure 7.Most of the instruction and practice across the a better pinwheel segments related to planning or conducting investigations.Little or no instruction or practice was allocated toward methods for working with middle school age youth or instruction for embedding inquiry into the curriculum.Training 1 had the most narrative in the areas of forming questions and planning/conducting investigations.Training 2 had the highest or the only narrative in the areas of working with youth, the three approaches, and embedding inquiry into the curriculum.Training 3 had the highest percentage of "other" narrative.
compares the percentages of time and narrative related to Designing a better pinwheel.The total time allocated to the lesson across the three trainings ranged from 20-55 minutes in length."Designing a better pinwheel" instruction and experience was 4% of the overall 15 contact hours in Training 1, 12% of the overall 7.5 contact hours in Training 2, and 13% of the overall 2.5 contact hours in Training 3. Therefore, while the same lesson was used and analyzed for similarities and differences, the lesson occurred in the context of a whole training in which either a great deal of additional content and skill building was delivered or relatively little additional training was provided.With those important differences, the narrative text training tape was a useful indicator of the depth of the attention to, and exploration of the essential elements of the model.While each of the three trainings' lessons incorporated some or all of the essential elements emphasized in the initial training for lead trainers, Training 2 provided the most comprehensive integration of the elements.
More evidence was needed to understand what aspects of the model were not working to prepare adult facilitators.Deeper analysis of a purposeful sampling of tapes of the trainings for adult facilitators revealed that the application of the training model ranged widely in instruction and experience.The core strategies from the initial training were not incorporated consistently across the three training examples as lead trainers worked to make the model fit in their organizations.For example, the range in the length of trainings across the three examples indicates major adjustments with one training designed as a 2.5 hour training and another designed as a 15 hour training.Designing a training "product" that is attractive to and meets the needs of adult facilitators needs further study.The challenges involved in extensive training of adult facilitators, many of whom are volunteers, are steep.Even when resources allow stipends for time spent in training, these stipends typically cannot cover the costs of time away from work or home.
"Learning is not just the process of constructing products through hands-on activities; learning includes the precursor activities of reflecting on what you already know and generating learning goals for what you 'need to know' "(Brophy, Klein, Portsmore, Rogers,  2008, p. 376).While adult facilitators can learn (and be novices) alongside youth, the training that prepares them for this partnership in learning is an important reference model for how learning can occur.The project was designed to first give adults this experience in inquiry learning through the training.
Training 1, with 15 total contact hours, stands out from the other two as an application of the initial training that ensured that adult facilitators had relatively more instruction and experience in teamwork, forming questions, planning/conducting investigations, and sharing or reflecting on the results.While data were not collected to provide evidence of the differential effectiveness of the trainings to prepare adults to work with youth, the application of the training model through Trainings 1 & 2 show the strongest evidence from the post-training assessments and the videotapes that the goals outlined for the project were integrated into the application.The adult audience for Training 1, an AmeriCorps group of trainees, were fulfilling mandated training requirements for which they received stipend payments.The application in Training 2 shows evidence that relatively more time was dedicated to instruction on the three approaches and embedding inquiry into the curriculum, and that some time was spent in each of the other essential element areas.However, this application did not contain any practice conducting investigations in teams.This training was delivered to an audience of volunteers with strong science or engineering-related professional backgrounds.These volunteers did not receive a stipend for the day-long training.