Involving Youth in Community Emergency Preparedness: Impacts of a Multistate Initiative

The National Preparedness Guidelines (2007) state, “as uniformed responders account for less than 1% of the total U.S. population, it is clear that citizens must be better prepared, trained, and practiced on how best to take care of themselves and assist others in those first crucial hours during and after a catastrophic incident.” This is increasingly more evident due to recent disasters such as hurricane Katrina. The Alert, Evacuate and Shelter (AES) program identified and trained youth/adult teams to use geospatial technology to map shelter locations and evacuation routes. Training began with team building activities to strengthen and build youth/adult preparedness partnerships. Program evaluations revealed a major shift in thinking about the positive potential level of involvement of youth in emergencies. Survey results immediately following trainings revealed statistically significant increases in participant knowledge gain regarding emergency preparedness. Follow-up evaluations indicate the success of this project in meeting community preparedness goals.


Introduction
Disaster situations can, and do, affect thousands of individuals every year across the United States.Preparedness levels determine the degree of individual and community response and recovery.Though some people feel it is impossible to be prepared for unexpected events, the truth is that taking preparedness actions helps people deal with disasters much more effectively when they do occur (FEMA, 2009).Just as individuals and families must prepare for the unexpected, government and community agencies work on a larger scale to develop infrastructure strategies that help keep all residents safe.One component of community safety is the creation of maps specifying evacuation routes, shelter sites and emergency response equipment locations.With the onset of geospatial technology and its recent incorporation into the field of emergency management, map creation is becoming a reality.
Often our adult/government resources are pushed to the limit when planning for disasters.
Youth have important roles they can play to help ensure that planning is optimized and information resources are available to all.The National Research Council's 2006 report, Learning to Think Spatially, recommends that spatial thinking be recognized as a fundamental part of the K-12 education due to its importance as a problem-solving tool in many different disciplines.By understanding the relationship between people, movement and locations we gain insight into the concept of geography, and how important location is when preparing and responding to a disaster.Working alongside emergency management personnel, geospatial technology experts and county extension personnel, youth can assist agencies in educating communities about disasters.The concept of involving youth and adults as partners in community readiness networks became known as the Alert, Evacuation and Shelter (AES) program.
The AES program evaluation focused on three program objectives: 1. Increase knowledge and use of geospatial technology for emergency preparedness, 2. Promote and enhance youth and adult partnerships in emergency preparedness, 3. Increase awareness and participation in personal, family and community emergency preparedness activities.

The Program
In an effort to improve community preparedness, safety and available resources, the Alert, Evacuate and Shelter (AES) program identified and trained youth/adult teams to use geospatial technology to enhance local government and community agency emergency preparedness efforts.Teams were comprised of youth, extension personnel, adult volunteers, emergency management staff and geospatial technology experts and were recruited from the 11 southeastern states and the District of Columbia.These locations were prioritized based upon both recent and historical devastation by hurricanes and the urgent need to address emergency preparedness in these locales.Through the use of GPS (Global Positioning System) and GIS (Geographic Information System) geospatial mapping, teams learned how to work with community emergency personnel to help evaluate emergency resources and address evacuation and shelter mapping concerns.
While geography and the technology associated with it are important skills for emergency preparedness, one unique characteristic of this project was the involvement of teens in partnership with adults (The Innovation Center, 2005).Involving youth in emergency preparedness not only adds additional people available for planning and response, research indicates youth need the geographic literacy skills taught in the program (Backler, et al, 1986).Through the use of geospatial technology, youth and adult teams learned how to observe relationships, acquire information and map geographic representations of what they learned.In addition, based upon what they learned, teams worked with community agencies to map shelter locations and evacuations routes, further enhancing their knowledge of geographic relationships.

Training Overview
A multi-state group of educators recognized for their expertise in various components of session topics, e.g.youth/adult partnerships, community education, emergency management procedures, geospatial technology, were identified as trainers for this project.Trainers designed a three-day program model to introduce youth/adult teams to the field of emergency preparedness and to enhance geospatial knowledge and youth development.Youth experienced in using geospatial technology served as co-trainers teaching computer mapping applications and shared their prior experience making emergency evacuation route and shelter site maps.Each of the five regional trainings (Virginia, Maryland, Texas, Georgia, Louisiana) began with team building activities, designed to enhance the importance and value of each team member with a focus on building the youth/adult partnerships in the county teams (Zeldin, et al, 2008).Exercises supported the importance of youth/adult partnerships.Forty-six counties were represented at the trainings with 174 individuals completing (45% youth, 55% adult).
A modified tabletop exercise (TTX) allowed both youth and adults to practice leadership, decision-making, and mentoring roles.The TTX is a scenario-based exercise in which participants "practice" their response techniques and strategies in planning for a real disaster (FEMA, 2008).The TTX was designed to give participants insight into the process first responders go through when planning response to a disaster.Guest speakers reinforced the necessity of community preparedness and shared actual disaster response stories.Trainers reinforced the benefits of youth/adult partnerships and how they can impact community preparedness strategies by incorporating geospatial technology.
Participants learned many aspects of geospatial technology, including how to collect GPS coordinates, download points into computer mapping programs, incorporate digital pictures into maps, collect data for maps, format data for incorporation into maps, determine what datasets are needed, and incorporate selected databases into the completed map.In addition, youth learned how to conduct emergency resource inventories in order to assist their emergency responders in identifying gaps in needed services.
During the 3-day training, county teams were also given the opportunity to complete grant applications for mapping software to support their team technology efforts Tele-communication activities were initiated to support participants in this program implementation phase.Trainers facilitated teleconferences allowing for an exchange of ideas and an opening of network opportunities.Participants discussed strategies for incorporating their skills and talents into the county emergency preparedness planning system.In addition, materials, resources and training information were posted online at www.crn4h.org.This web site received over 26,000 hits in 2008 and over 1400 hits in the first two weeks of 2009.The purpose of the web site was to provide support for the teams that participated in the 3-day workshops.While others obviously viewed the materials, the program evaluation was concentrated on workshop participants.

Program Evaluation Methods
A multi-method design was used to measure program impacts immediately following the training, six months after training and one-year after training.The logic model was the guiding principal behind this design approach of measuring short, medium and long-term outcomes (Arnold, 2002).The Dillman (2007) tailored design method was also used in designing the evaluation format.
The first instrument was designed to measure knowledge gain of participants immediately following the AES training and used a retrospective pre-post survey design for this initial measure.The retrospective pre-post survey allows participants to rate their knowledge at the end of the program on the post and to think back to how much they knew before the program on the pre.Both the pre-survey and the post-survey are completed at the end of the program and helps to alleviate the potential of respondents over-and/or under-assessing their perceived learning, a potential constraint of the traditional pre-test post-test method.This method was chosen to help address the problem of "response shift bias" (Colosi and Duncan, 2006).
Approximately six months after the training, evaluators completed a telephone survey to ascertain the level of project implementation as a result of the training.Finally, the one-year retrospective follow-up survey was completed to measure long term impacts of the training (Davis, 2003).The retrospective pre-post survey was used in this final long term measure to help eliminate problems with tracking program participants, often a problem when conducting long term evaluations (Raidl, et al, 2004).
1. Immediately following training retrospective pre-post survey: Immediately following each of the five training sessions, a retrospective survey was administered to youth and adults.Surveys, collected on site, were voluntary and anonymous.Respondents were asked to rate 19 topics using a 5-point Likert-type scale with a 6th point "don't know." In addition, respondents were asked to select if they were participating as youth, or in one of the adult roles.The survey administered immediately following the training not only evaluated participant knowledge gain, but was immediately reviewed to help trainers improve subsequent trainings.Of the 174 registered participants, 84% returned a completed survey.
2. Six-Month follow-up telephone survey: A telephone survey administered to team leaders mid-way through the program asked a series of 15 open-ended questions, focusing on team activities and community engagement as a result of the training.One of the purposes of this mid-term qualitative evaluation was to determine which of the program's expectations were being acted on thereby allowing the connection of the program processes to participants' achievement of program goals and objectives.
Qualitative methods are well suited to the explanation of the program's theory in action (Weiss, 1998).Of the 46 teams who participated in the initial trainings, 50% responded to the telephone survey.Interviews were transcribed and qualitative data analysis was completed by reviewing the themes from the interviews.Representative quotes are included in the findings to help explain team progression.
3. One-Year follow-up retrospective pre-post survey: Long-term impacts were measured using a mail-out survey method, again using a retrospective pre-post design (Raidl, et al, 2004).The surveys were mailed to youth and adult participants one-year after completion of training.While separate instruments were used for youth and adults, identical topics were covered.Respondents were asked 13 demographic questions and 21 questions using a 5 point Likert-type scale with a 6th point "don't know."Response rate from the initial 174 site training participants for this follow-up survey was 25%.Cronbach's coefficient alpha was used to estimate reliability of the Likert-type scale survey items for the quantitative measures.The Cronbach score was high (r=.847;r=.918; r=.835) indicating a high level of survey reliability for each of the three scales used in the survey (Santos, 1999).
A Wilcoxon non-parametric statistical query was used for the quantitative data analysis for both the training survey and the follow-up survey.All evaluation instruments were approved through the University of Nevada Institutional Review Board to ensure that correct investigative protocols were maintained throughout the entire process to protect subjects' rights.

Evaluation Findings
1. Immediately following training survey: Survey results immediately following trainings revealed statistically significant increases in participant knowledge gain, based on comparison of mean pre-test and post-test scores, for all survey questions.Table 1 below shows the ranked mean scores for each of the teaching topics included in the survey (1=low rating and 5=high rating on a Likert scale).
The rankings shown in Table 1 indicate which topics had the greatest average score improvement comparing pre-to post-scores for the 19 topics surveyed.In the ranking of topics below, "how to use GIS (geospatial mapping) software to create maps" showed the biggest increase in knowledge gain.In general, the technology associated topics are ranked in the top four positions for biggest increases in knowledge gain.The "role of a teen CERT in a community disaster" was ranked in fifth place for knowledge gain.Those topics listed toward the bottom of Table 1 include "the importance of an alert system" and "the importance of personal and family disaster preparedness".While participants increased their knowledge about these lowest ranking topics, their knowledge was already high when they began the program; thus the smaller differences between pre and post.Descriptive statistics software (SPSS 14.0 Software, 2006) was used to analyze survey results (84% response rate).2) Six-Month Follow-up Telephone Survey: The telephone survey conducted mid-project provided examples of community engagement related to program goals and objectives as well as providing a report on team activities.In general, the telephone survey responses indicated that the knowledge gained during the AES training was being used to implement community projects.Examples of projects described during the telephone interviews included the following."We are locating fire hydrants, fill pumps and main valves using the GPS units.We want to provide emergency management, water and sewer, fire department and anyone else who would use it, a map." (note: municipal water and sewer availability after recent major hurricanes was disrupted for several weeks in some areas making locations for infrastructure an important issue).Other teams were working to build community support and relationships with their emergency managers and agency officials.Said one interviewee: "BRACE is a Hurricane expo where 3,000 people attend.At their planning meetings, information about us was brought up which allowed us to make contact with the county GIS person.We gave him a pamphlet and he took it to his bosses who gave him permission to do whatever is needed to help us.The county commissioner has given full support and he and the EOC chief officer have written letters for grant support." 3) One-Year follow-up retrospective pre-post survey: Results of the final evaluation survey are shown below and are reported in three different tables: the Opinion Scale, the Level of Involvement Scale, and Level of Knowledge Scale.A Likert-type scale (1-low and 5=high) was used for each of the questionnaire items shown in Tables 2, 3, and 4. (SPSS 16.0 Software, 2007 was used for analysis).
Opinion scale: Each topic listed in Table 2 had statistically significant changes associated with it.The rankings shown in Table 2 indicate which topics had the greatest average score change comparing pre-to post-scores for the 9 topics surveyed.In this opinion section of the final survey, the highest ranked statement was "I would recommend this project to others."This ranking is a comparison of the mean pre-test score to the mean post-test score.This ranking may indicate that participants were somewhat neutral (mean score of 3.45) about the project before participation, but gave the project an almost perfect score one year later (4.66).
The lowest ranked items in the opinion scale were the items about youth/adult partnerships.The difference between the pre and the post was smaller in comparison than the highest ranked items.These smaller differences can be attributed to the high scores on the pre-survey (mean of 4.21 on the 5-point scale).It appears that program participants understood the benefits of youth/adult partnerships prior to participating in the program.Level of Involvement scale: Each topic in the one-year follow-up survey listed in Table 3 had statistically significant changes associated with it.The rankings shown in Table 3 indicate which topics had the greatest average score change comparing pre-to post-scores for the seven topics surveyed."Youth are capable of assisting in emergencies" was the highest ranked item followed by "Youth are capable of providing educational training about emergency management."In contrast, the lowest ranked Level of Involvement item was "youth are capable of distributing emergency supplies."Participants rated this item high on the pre, thus the small difference in response between pre and post.
These findings may demonstrate a change in thinking about the level of involvement of youth in emergencies.This finding seems to demonstrate a general agreement that teen involvement in traditional adult directed jobs like distributors of supplies was an appropriate goal before the program was implemented.Data reveal that after program implementation the program goal of youth involvement in assisting with emergencies was achieved.Level of Knowledge scale: Level of Knowledge is the final category of this one-year follow-up survey.Each of the items in this category indicates significant improvements in the technical aspects of the training.This parallels with the results of studies conducted earlier in the program.Further discussions of these geographic literacy responses are provided following Table 4, and are shown as a graphic representation in Figure 1.

Additional Findings of One-Year Follow-up Study Data
Based upon project priorities and the richness of the data provided by project participants, additional analysis was completed on the one-year follow-up study data and is shown in Tables 2, 3 and 4 (c) Objective 3: Increase awareness and participation in personal, family and community emergency preparedness activities.Data from the survey questions related to the level of knowledge about CERT to enhance team involvement in community emergency preparedness measures are further analyzed.
For the purposes of preparing the figures that follow, the 5-point Likert scale was collapsed into three categories and reported as a percentage of the total response: (1) disagree (strongly disagree plus disagree), (2) neutral (no change) and (3) agree (strongly agree plus agree).This approach to reporting the data was made to simplify the narrative explanation.Descriptive statistics were used for this analysis.2).These survey results completed at the end of the training program contrast with attitudes reported in the one-year follow-up.
The findings on the one-year follow-up pre-survey now indicate that some of the adults were indeed skeptical as shown in Figure 2. The post survey reveals a much more positive attitude for both youth and adults on this topic of working together to benefit the community.Figure 2 shows the percentage of participants who disagreed, were neutral or agreed to the statement "youth and adults are capable of working together to benefit the community" both before and after the program.A review of figure 2 reveals that about 10% of the adults disagreed with the statement at the beginning, compared to 0% of the youth.However, after the program a significant change was noticed in adult responses.Figure 2 below shows the long-term change in adult opinion, significant at p<.05.After the program, a few of the adults were still neutral in their opinion, none disagreed with the statement and an overwhelming majority agreed with the statement.In contrast about 30% of the youth were neutral about the capability of youth and adults working together at the start of the program.After the program, over 90% of the youth agreed with the concept.The primary activity used to measure this objective was related to CERT, specifically, "I am knowledgeable about CERT (Community Emergency Response Team)".A review of youth and adult responses are shown in Figure 3. Analysis of the one-year follow-up data revealed that both youth and adults were unfamiliar with the CERT program at the beginning of the trainings.After the training almost all of the adults and most of the youth reported they were knowledgeable about CERT.

Figure 3
Knowledge about CERT: Comparison of Before/After Program Results

Discussion
All project objectives were clearly met as indicated in the evaluation findings.Objectives for the evaluation included: 1) increased knowledge and use of geospatial technology, 2) promoting and enhancing youth and adult partnerships, and 3) increased awareness and participation in personal, family and community emergency preparedness activities.
Participants recognized immediate knowledge gains in all aspects of the training.
As indicated, the highest ranked reported learning took place in the area of geospatial technology.As one participant stated in the final survey "Please provide more AES trainings.In a very short time I was given more training than I received in an Intro to GIS course in college."AES participants learned about their surrounding environments and asked important geographic questions in order to complete their project and benefit their communities.
While a main focus of the AES program was geospatial technology, an equally important component was fostering youth/adult partnerships, empowering youth to take leadership roles for the betterment of their community.The final retrospective survey revealed a long-term change in knowledge, attitude and behavior.Participants recognized that youth could be valuable resources, affecting positive change.A theme revealed in the qualitative data analysis and clearly stated by one adult represented in the sample was "Without the training, I would not have been able to help them (county response agency) see the advantage of working with youth." A cross-comparison of youth and adults revealed very interesting indications for partnerships benefiting communities.As a result of the training, adults reported that they believed youth were very valuable resources (see Figure 2 above) and that their work could be beneficial.Again, each training session heavily promoted an atmosphere for youth/adult partnerships.
This change in attitude pointed to the effectiveness of training efforts to build a sense of partnership in addressing important community topics.
A third focus of the training, to increase awareness and participation in personal, family and community emergency preparedness activities, took place beyond regular training sessions as teams implemented their community projects.A comparison of youth/ adult perceptions before and after the training revealed a significant increase in knowledge of the CERT.The long-term evaluation revealed an increase in CERT participation in the community.Said one participant "We are doing Teen CERT and working with the EOC (Emergency Operations Center).We certified 4 youth and 2 adults in CERT and CPR and AED (Automated External Defibrillators) training."

Summary and Recommendations
The purpose of the AES program was to develop a network of youth and adult teams that could assist their communities to be better prepared to stay safe during a disaster situation.During both the training and the implementation phase of the program, youth worked hand-in-hand with their adult counterparts to learn about community infrastructure, and how their emergency response agencies plan for disasters.By incorporating geospatial technology, teams created shelter site and evacuation maps where none had existed before, a needed skill identified as a result of several devastating hurricanes on the Gulf Coast The focus of these trainings was to educate youth and adults interested in emergency preparedness related to hurricane tragedies.The locations of the trainings were in communities where hurricane incidents have occurred.While training materials were originally designed to address specific hurricane concerns, training content is applicable to all types of natural and man-made disasters.As disasters affect every county in the nation, this training model could easily be adapted to fit all locations and all disasters, and would be of special interest to those working to address community emergency preparedness issues.As the AES data reveal, these youth and adults teams can be valuable resources in helping keep communities safe.
An example of adapting the AES curriculum to local communities needs is a project involving animal shelters.When researching the importance of human-animal bonds during emergencies, trained teams realized the need to map animal shelters sites and educate residents about the importance of animal disaster kits.Having plans in place for animal family members is a critical component to human safety.This safety issue was demonstrated in past disasters as humans refused to evacuate without pets when there was no capacity to accommodate the pets.As a result, team members have become the catalysts for starting animal response teams in areas previously uninformed about the need or unclear as to how to begin the process of addressing pet evacuation issues.As evidenced through the impact assessment, this program clearly encouraged and enhanced youth and adult partnerships to respond to critical community needs.As first responders and agency personnel are often overwhelmed in planning for and responding to a disaster, the addition of youth helped create needed resources to enhance community and safety well-being.
. Specifically, data were scrutinized in the following areas as they correlated to the program objectives: (a) Objective 1: Increase knowledge and use of geospatial technology for emergency preparedness.The Level of Knowledge subscale is looked at more closely to more clearly understand the long-term importance of the geospatial technology training to youth.(b) Objective 2: Promote and enhance youth and adult partnerships in emergency preparedness.Additional analysis of the one-year follow-up study was completed to more clearly understand the impact of youth and adults working together to benefit the community.

( a )
Level of Knowledge scale: Figure 1 below shows the knowledge gains reported in the final retrospective long-term survey regarding geospatial technology taught during the AES trainings.A further look at the Level of Knowledge section of the one-year follow-up survey is provided because increased knowledge and use of geospatial technology was the priority program objective; specifically, enhancing geographic literacy through the use of geospatial technology.Figure 1 shows the dramatic increases in knowledge gain regarding the geospatial technical training.

Figure 1
Figure 1Level of Geospatial Technology Knowledge Before/After Program

Figure 2
Figure 2Capability of Youth and Adults Working Together to Benefit the Community While the objectives of this program were accomplished, the potential exists to achieve further program impacts.Additional funding could support new face-to-face training in locations other than those impacted by hurricanes.Further program development could incorporate presentations made to responder organizations to showcase program impacts and encourage youth involvement in community emergency response activities.A series of online training modules could be offered for specific AES components, supporting further knowledge gain as well as reinforcing face-to-face training concepts.Additional funding would support curriculum development needed to expand the program.Supporting youth/adult teams in educating community leaders regarding the potential additional resources of involving volunteer teams in planning for emergencies is an important goal in program expansion.

Table 1
Ranking in Score Improvement on Topics Taught in a 5-state Emergency Preparedness Program a Differences between pre-test and post-test scores statistically significant at p<.01

Table 2
Ranking in Mean Scores for the Mid-Term Impacts of the Alert, Evacuate, and Shelter Program

Table 3
Ranking in Mean Scores for the Mid-Term Impacts of the Alert, Evacuate, and Shelter Program

Table 4
Ranking in Mean Scores for the Long-Term Impacts of the Alert, Evacuate, and Shelter Program