Preparing the Next Generation of After-School Educators : College Students ’ Perceived Learning and Civic Engagement Associated with the CASE Program

First-year evaluation findings from the University of California, Irvine Department of Education’s Certificate in After-School Education (CASE) program are reported in this paper. The goal of CASE is to promote positive youth development in diverse learners through education and training of the after-school workforce. CASE blends instruction across five, 10-week long courses with 70+ hours of fieldwork in local after-school programs (ASPs). CASE course and fieldwork enrollment, perceived understanding of course material, multicultural education, and civic interests and engagement were measured through student surveys. Students in CASE courses report higher levels of perceived course understanding (p < .01), civic responsibility (p < .01) and empowerment (p < .05) than students in the non-CASE courses. Students enrolled in CASE courses requiring fieldwork report greater perceived course understanding (p < .01) and academic engagement (p < .01) than CASE students without fieldwork. The findings suggest the program is achieving several of its early goals.


Overview
A recent review of literature on out-of-school time highlights both the growing popularity of and need for after-school programs (ASPs) (Mahoney, Parente & Zigler, 2009).In total, approximately 8.4 million children are currently enrolled in ASPs (e.g.After-school Alliance, 2009).There is a trend toward increasing ASP participation among all income levels though the trend is especially pronounced among the lowest income levels (Harvard Research Project, 2006).With this growing trend, the findings that participation in ASPs can lead to positive social development (Mahoney, Larson, Eccles, & Lord, 2005;Posner & Vandell, 1994) as well as improved academic performance (Grossman, Price, Fellerath, Jucovy, Kotloff, Raley, et al. 2002;Huang et al, 2000;Posner & Vandell, 1994) seems increasingly important.However, participation in an afterschool program does not always lead to positive outcomes.Whether or not a child benefits from after-school program participation depends on the quality of the program (Catalano, Berglund, Ryna, Lonczak & Hawkins, 1998;Durlak & Weissberg, 2007;Vandell & Pierce, 2001).Specifically, multiple studies have shown that program quality is a product of teacher quality (Mahoney & Stattin, 2000;Pierce, Hamm & Vandell, 1999;Smith, Peck, Denault, Blazevski & Akiva, in press).In light of the growing trend of ASP participation, it seems important to focus on improving ASP teacher quality which may be a productive step toward higher quality and greater benefits from ASPs.The current paper highlights the impact that staff quality has on potential benefits of ASP participation and the consequent need for professional development of ASP educators.Accordingly, the paper describes the first program of its kind aimed to comprehensively train ASP educators in a university setting and reports early findings on student's experience in this program.

The Value of After-School Programs
Research suggests that after-school program participation has the potential to lead to various positive youth development outcomes.For one, involvement in afterschool programs has been linked to improved academic performance (Grossman, et. al., 2002;Huang, Gribbons, Kim, Lee & Baker, 2000;Posner & Vandell, 1994).The improved academic performance may be a product of proper stimulation outside of school afforded by ASPs.Alternatively, studies have shown that unsupervised time can lead to negative academic outcomes (Synder & Sickmund, 1999;Weisman & Gottfredson, 2001) and that many children do not receive stimulation and adult support that they need to succeed in school (Eccles, Midgley, Wigfield, Buchanan, Reuman, Flanagan & Iver, 1993).Perhaps academic enrichment in many ASPs can account for improved academic performance (Cosden, Morrison, Albanese & Macias, 2001).Additionally, ASP participation can lead to positive social development including improved peer acceptance and decreased problem behaviors (Mahoney et. al., 2005;Posner & Vandell, 1994;Synder & Sickmund, 1999;Weisman & Gottfredson, 2001).ASP participation has also been linked to lower rates of obesity (Mahoney & Lord, 2005).To the extent that the developmental system works as a whole, benefits are synergistic.For example, gains in cognitive functioning support social well-being and vice-versa.While many students have been shown to benefit from ASPs, research suggests that low income and minority students who are at risk of academic failure may be especially likely to benefit from ASP participation.For example, low achieving students, black students, Hispanic students, and English language learners have been found to show greater gains in math achievement compared to other students involved in ASPs (Welsh, Russell, Williams, Reisner & White, 2002).Further, Lauer and colleague's meta-analysis of ASPs showed that test scores of low-income and at-risk youth improved significantly in both reading and mathematics after participation in after-school programs (Lauer, Akiba, Wilkerson, Apthorp, Snow & Martin-Glenn, 2006).Therefore, providing children with sufficient out-of-school stimulation may help to level the playing field for children who may not otherwise receive out of school stimulation conducive to academic achievement in US classrooms (Rothstein, 2004).
However, not all after-school programs yield these important benefits.Research shows that the extent to which ASP participation facilitates positive development depends on the quality of the after-school program staff (Catalano, Berglund, Ryna, Lonczak & Hawkins, 1998;Durlak & Weissberg, 2007;Mahoney, Stattin & Lord, 2004;Pierce, Hahm & Vandell, 1999;Rosenthal & Vandell, 1996;Smith, Devaney, Akiva & Sugar, 2009).Given the important potential benefits of after-school programs and because benefits of after-school programs depend on staff quality, facilitating the ability of staff to provide high quality service seems critical.
Evidence suggests that appropriate professional development provides the means to prepare ASP staff (Hall & Cassidy, 2002;Puzio, 1987;Smith, et. al, 2009;Wayne, Yoon, Zhu, Cronen & Garet, 2008).Teacher training has been linked to more confident and successful teachers and has shown to help teachers understand the diverse perspectives of learners (Causey, Thomas & Armento, 2000).Specifically, the fieldwork component in teacher training is linked to preparing educators to create successful learning environments for diverse populations (Rowe, 2003).The most recent review of professional development literature suggests that teachers who are products of successful professional development tend to have a positive effect on student achievement (Yoon, Duncan, Lee, Scarloss & Shapley, 2007).
However, professional development for ASP educators has not been a priority.Neither formal education nor training is required of many ASP staff (Boufard & Little, 2004).If available at all, ASP staff training models are usually limited to workshops, professional meetings, or online webinars (U.S.Department of Health and Human Services, 2007).Such workshops have been criticized as an ineffective process to prepare K-12 educators (Garet, Porter, Desimone, Birman & Yoon, 2001).In an attempt to fill the educational void in ASP educator training, the University of California, Irvine initiated a comprehensive program designed to train ASP educators in a university setting.

Overview of the Certificate in After-school Education Program
The CASE program is designed to train ASP educators though a blend of required coursework and related fieldwork in ASPs.The program is currently in its second year of operation and is focused on pre-service training and education aimed at university students.However, the program also offers the coursework to staff in the community through university extension courses and therefore provides in-service training opportunities as well.CASE students are required to complete five courses including: • a course on the foundations of out of school learning; • a choice between courses on human development or multicultural education; • a course on academic curricula relevant to after-school settings; • a course on expanded learning curricula (i.e.athletics, arts, technology etc.) and • an elective course chosen from either the academic or expanded learning categories (See Table 1 for complete list of CASE courses).
The fieldwork may be completed at any of the six ASPs that chose to partner with CASE and met CASE criteria (for a detailed explanation, see Mahoney, Levine & Hinga, 2010).The type of partnership between UC Irvine and selected ASPs can be labeled as a universitycommunity (U-C) partnership.A U-C partnership is defined as an explicit agreement between a community based organization and an academic unit to engage in a common project or goal, which is mutually beneficial (Suarez-Balcazar, Harper & Lewis, 2005).The goal of CASE U-C partnerships is to provide local ASPs with trained volunteers while also providing CASE students with hands on experience at ASPs under scaffolded conditions.One of the goals of the program is to positively impact CASE students.The potential benefits to CASE students include: increased understanding of course material, diverse cultures, and service learning ideals through the blend of coursework and hands on (fieldwork) experience.

CASE as a Community of Practice and Social Learning System
The current paper explains how CASE creates a Community of Practice, which may create additional learning benefits to CASE students, in comparison to students not involved in the CASE program.A Community of Practice is a type of community created by the quest for a shared ambition (Wenger, 1998).The shared ambition of CASE, is the desire to learn about the after-school education field.There are three crucial elements in a Community of Practice (COP): • the domain -A COP has an identity defined by a shared field of interest; • the community -Through pursuit of a common domain, COP members engage in shared activities and build relationships that allow them to learn from each other; • the practice -Members of a COP develop a shared repertoire of resources based on sustained interaction and experience.
The CASE program creates each of these three elements.COPs are thought to have many benefits.Of particular interest are two benefits of COPs that are important for ASP educators: 1) a deep understanding of material and 2) an intensified mutual commitment to the domain of interest (Wenger, 2000).
The CASE program allows students to form a COP which may provide CASE students with the potential for a deep understanding of ASP education and an intensified mutual commitment to ASP service through the blend of coursework and fieldwork.
Second, the design of CASE allows students to learn through experience.Experiential learning occurs through the blend of coursework and fieldwork afforded by CASE courses and fieldwork opportunities.This blend provides students with knowledge to be ASP educators (through coursework) as well as practical experience (through fieldwork).Specifically, the blend of knowledge (through coursework) and experience (through fieldwork) allows student engagement (interactions with others), imagination (how one sees oneself as being connected to a broader community) and alignment (coordination of perspectives and actions to reach higher goals) (Wenger, 2000).
Through the blend of coursework and fieldwork, CASE students are able to engage in both, the academic and practical components of ASP education.Academic engagement provides students with a knowledge base and theoretical background pertinent to their careers as ASP educators.While they learn these theories and implications they are also able to engage in interactions with ASP students and staff, and are in turn able to decide for themselves which coursework components pertain to their practical experience.Additionally, CASE provides students with the opportunity for alignment of ideas learned in courses with actual fieldwork experience.Lastly, through theories of learning in courses and hands-on fieldwork experiences, CASE students are able to imagine themselves in the role of an after-school educator.The CASE program employs experiential learning to provide CASE students with the potential for a deep understanding of ASP education.The following subsections, highlight the potential benefits of CASE as a COP and social learning system.

Understanding Course Concepts
The content offered through CASE courses covers concepts integral to successful ASP educators.To supplement classroom learning, CASE students are provided opportunities to learn experientially through direct work in ASPs.The comprehensive nature of the CASE program (including an interrelated coursework regimen and the fieldwork component which aligns with coursework) should help students feel like members of the ASP educator community and therefore allow them to place what they learn into a broader framework which helps with understanding (Wenger, 2000).Second, most CASE courses require a fieldwork component that is tied to ideas learned in courses.According to Wenger's COP theory, the fieldwork experience should allow students to align the ideas they learn through coursework with their fieldwork experiences, which should help students have a deeper understanding of their coursework, including: perceived understanding of course material and perceived ability to apply course concepts.

Understanding Diverse Populations
Effective educators need skills to work with diverse youth (Banks, 1995).However, multiple evidence sources suggest that it is the lack of direct and meaningful interaction with different cultures that prevents educators from a proper multicultural understanding (Cannella & Reiff, 1994;Fereshteh, 1995;Russo & Talbert-Johnson, 1997).Follo and colleagues (Follo, Hoerr & Vorheis-Sargent, 2002) found that educators must be immersed in other cultures to develop a multicultural understanding.Immersion can take the form of fieldwork (Russo & Talbert-Johnson, 1997) as implemented in the CASE program.Multicultural education is covered across CASE courses and is an integral aspect of fieldwork.The courses integrate ideas about how to educate diverse individuals and fieldwork sites consist of students from diverse ethnicities and socioeconomic statuses.Of additional importance is the reflection and dialogue between students and supervisors about multicultural experiences so that stereotypes and biases are not reinforced (Ooka, 1994).The face to face experience under direct supervision in fieldwork paired with the reflection and multicultural teaching in coursework should help students feel more prepared and comfortable to work with diverse individuals.

Service Learning Measures
Social learning through U-C Partnerships may also act to increase service learning measures among CASE students.Ample evidence has shown that students who participate in high quality service learning are likely to show multiple benefits.For one, students' empowerment tends to increase as they learn they can impact real social needs (Chung, 1997).Service learning tends to increase students' interest in furthering their education (Lewis-Charp, HanhCao, Soukamneuth & Lacoe, 2003) and may promote positive attitudes about civic engagement (Tannenbaum, 2007).Service learning may help students to understand community needs, develop ethic of service and civic responsibility and increase their desire to actively contribute to community (Billing, 2000).The mix between community service (ASP participation) and reflection (during coursework) through the CASE program seems to provide an ideal learning experience for the enhanced empowerment as well as academic, civic, and career engagement.

The Present Study
This study assesses whether college students' perceived course understanding, multicultural understanding, and service learning intentions are related to participation in CASE courses and fieldwork.To test the idea that students in the CASE program benefit from the COP created by CASE, students in CASE courses were compared to students enrolled in a university course offered in the same department (the Education Department) that is not part of the CASE program (hereafter referred to as the "control course").To measure the relationship between the above measure and fieldwork (or experiential learning), this study capitalizes on the design of CASE which requires fieldwork for some but not all CASE courses.The present study compares students in CASE courses with a fieldwork component to students in CASE courses without a fieldwork component along the above measures.
There are four main expectations for students in CASE courses (compared to the control course) and for students with fieldwork (compared to those without fieldwork).
1. First, we expect students in CASE courses to report greater understanding of course material and feel more prepared to apply course material than students in the control course due to the COP created by the comprehensive nature of the CASE program.
2. Second, we expect students in CASE courses with fieldwork to report greater understanding of course material and feel more prepared to apply course material than students in CASE courses without fieldwork, in accordance with the idea that fieldwork creates opportunities for social leaning.
3. Third, consistent with social learning theory, we expect that students in CASE courses with fieldwork will report greater comfort levels, desire to work with, and feeling of preparedness to work with individuals from diverse backgrounds than students enrolled in courses without a fieldwork component.
4. Fourth, we expect students in CASE courses to report higher service learning scores than students in the control course, due to the emphasis on the importance of service in the COP of CASE.
5. Fifth, we expect students enrolled in CASE courses with a fieldwork component to report a stronger orientation toward service learning than students in CASE courses without a fieldwork component because of the importance of service learned socially through fieldwork experiences.

Methods
Participants.Participants include 174 students enrolled in one of five CASE courses offered during the spring quarter of 2009 (81% of total students enrolled in these courses) and 50 students enrolled the control course (93% of students enrolled in this course) that was also offered through the Department of Education (see Table 1).Because CASE courses vary as to whether they involve fieldwork at an ASP, the sample can be further categorized as follows: 1) students enrolled CASE courses with fieldwork (n = 66); 2) students enrolled in CASE courses without fieldwork (n = 112); and 3) students enrolled in the control course (n= 50).Table 2 provides a list of the courses surveyed.The majority of students in these courses are full-time undergraduates and 77% were female.Procedure.During the last week of the spring quarter, students in each CASE course and the control course were asked to complete a survey during class time.The students were read the purpose of the survey and were made aware that survey completion was not mandatory nor did it impact their course grade.The surveys contained no identifying information.Students enrolled in courses without a fieldwork component were not asked questions pertaining to fieldwork experiences.The surveys were otherwise identical.

Measures
Basic demographic information, degree of perceived learning, multicultural appreciation, and service learning scores were measured through survey questions.An average score for perceived learning was based on students' answers to three questions regarding: understanding of course material; course usefulness; and feeling of preparedness to apply course concepts (e.g."How ready to you feel to apply the information you have learned in this course?"Each question was based on a four point scale.Cronbach alpha reliability was .64. An average score for multicultural understanding was based on each individual's average score based on answers to four questions.The questions included: amount of experience with; feeling of comfort working with; feeling of preparedness to work with; and desire to work with diverse populations.(e.g."How well prepared do you feel to serve low SES and minority populations?").Each question was based on a four point scale.Cronbach alpha reliability was .80.
To measure service learning domains, participants responded to questions from four scales taken from the Higher Education Service-Learning Survey created by the Higher Education Service-Learning Survey.(2000).The survey includes six statements pertaining to academic engagement, such as: "I find the content in school courses intellectually stimulating."There are nine questions specific to civic engagement, for example, "I am concerned about local community issues."Six questions pertain to career engagement, including "I feel well prepared for my future career."Lastly, there are eight statements aimed to measure empowerment, such as, "I can make a positive difference in my life."Responses to each statement are scored upon a four-point Likert scale.Statements either positively or negatively endorse each construct, so that in the case of negative statements, Likert-scale scores were reverse coded.Statements pertinent to each of the four categories were randomly mixed throughout the surveys.Cronbach alpha reliability coefficients for the four scales were as follows: academic engagement (.68); civic responsibility (.86); career engagement (.76); and empowerment (.69).

Results
Perceived Learning.Regression analysis uncovers that, consistent with our first hypothesis, students in CASE courses report higher average levels of perceived course understanding than reported by students in the control course (B = .27,p < .001)(See Table 3).Consistent with our second hypothesis, students enrolled in CASE courses with a fieldwork component reported higher levels of perceived learning than students enrolled in CASE courses without a fieldwork component (B = .23,p < .01)(See Table 4).Multicultural Understanding.Our third hypothesis was not met.Regression analysis failed to show a significant link between multicultural understanding for either students enrolled in CASE courses compared to the control course (See Table 3) or for students enrolled in CASE courses with fieldwork compared to those enrolled in CASE courses without a fieldwork component (see Table 4).

Discussion
This study reported early evaluation findings of the CASE program.The evaluation was intended to help understand the current level of implementation of the program and assess some of the associated impacts to identify areas where improvement may be needed.In addition, because of the considerable interest and need for education and training programs to better prepare the after-school workforce, it is hoped that the program design and assessment will be of use to parallel efforts occurring around the nation.Overall, the results suggest that university students enrolled in the CASE courses, particularly those with a fieldwork component, report high levels of service learning and civic engagement, compared to university students not enrolled in CASE courses.
To begin, consistent with our first hypothesis, findings suggest that students enrolled in CASE courses tend to score higher on aggregate scores of perceived learning.This link parallels past work by Lave and Wenger (1991) that explains that engagement, imagination and alignment of material leads to comprehensive understanding.This literature is also consistent with our second hypothesis and corresponding finding that students in CASE courses with fieldwork reported higher scores of perceived learning than students in CASE courses without fieldwork.It seems that the experiential learning, afforded by hands on fieldwork experience, relates to increased learning.
Next, our third hypothesis that perceived multicultural understanding levels would be higher for students in CASE courses versus the control and for students in CASE courses with fieldwork compared to courses without fieldwork was not supported.This may be a product of either a lack of reflection associated with hands on experience or an insufficient amount of exposure to diverse populations during fieldwork.Perhaps further studies may look at the degree of reflection as a mediating factor between CASE courses or fieldwork experience and scores of multicultural education.Conversely, the process of reflection may be adequate but the amount of multicultural education received over one quarter may simply be insufficient to multicultural understanding.Possibly, with a higher dosage of coursework matched with fieldwork, results may differ.Increased multicultural understanding may be linked to more than one quarter of CASE courses and/or fieldwork experiences.Perhaps, greater multicultural understanding will only be linked to the completion of all CASE requirements because it is the cumulative nature that is important.Further, the CASE curriculum includes a course specific to multicultural understanding which was not offered during the spring quarter measured here.Completion of the course on multicultural education may be a strong predictor of multicultural understanding.
Finally, our fourth and fifth hypotheses that service learning measures would be higher for students in CASE courses compared to the control course and for students in CASE courses with fieldwork compared to courses without fieldwork was partially supported.Findings suggest that students in CASE courses reported higher levels of civic engagement and empowerment than students in the control course, though there were no significant differences between the groups in academic or career engagement.Students in courses with fieldwork compared to those in courses without fieldwork only yielded higher scores of perceived academic importance.No other comparisons were significant.
The positive findings are consistent with the theory that the comprehensive nature of the CASE program lends itself to imagination of self as an ASP educator and therefore is linked to greater scores of civic engagement and empowerment (Wenger, 2000).And the service learning literature suggests that service learning experiences are linked to increased academic engagement (Lewis-Charp, et al., 2003).Although prior research does not suggest why service learning outcomes would vary across the measures, the lack of findings in some areas may be due either to the fact that the CASE program is at an early stage of development or that a short timeframe for evaluation was undertaken in this study.For example, perhaps service learning outcomes in other areas will be more apparent as students accumulate a greater dosage of CASE courses and/or fieldwork experiences over time.Alternatively, as the program matures, any given course may have a greater associated impact on student outcomes.

Limitations and Future Directions
This study represents the first step in examining experiences of students within the CASE program.As such, we discuss limitations and directions for future research.First, this study is not causal.While we did control for possible confounds (e.g.gender, ethnicity, and grade) we cannot claim that completion of CASE courses or fieldwork caused students to score higher on any measure.There are a number of potential confounds that may influence the established links between either enrollment in CASE courses or courses with a fieldwork component, such as baseline differences.Future research should employ methods more conducive to understanding causation.For example, pre-and post-test assessments should be undertaken to assess course-related growth over time.
Additionally, as mentioned for all measures, lack of findings on certain measures may be due to the early timing of this evaluation.Perhaps differences between CASE courses compared to the control course may be related to the completion of all multiple CASE courses, rather than just one.Similarly, it is possible that outcomes related to fieldwork experiences increase with the completion of greater amounts of fieldwork.Future studies should measure dosage of courses and fieldwork, as well as a combination of the two to determine if outcomes depend on additive exposure to CASE courses and/or fieldwork.In terms of all outcomes, if there is a link between dosage and outcomes it will be important to determine whether the completion of CASE program requirements provides sufficient dosage to yield beneficial results.
The measures may also be a limiting factor in the determination of more lasting impacts of the CASE program.Perception of learning is not equivalent to actual learning.For the purpose of this preliminary evaluation, perception of learning was an important measure because it may lead to greater confidence and illuminate certain processes that at least are linked to actual learning.However, while perception is important in this early evaluation, an additional aim of CASE is to prepare students who have a true understanding of course material which may be measured through course tests or other more objective measures.Likewise, reported multicultural understanding is important in this preliminary evaluation to understand early processes.Future studies may benefit from a more objective measure of multicultural understanding, possibly including observation of CASE student interactions with diverse children or survey questions that aim to measure multicultural understanding more objectively.

Conclusion
It is evident that institutions of higher learning can play a role in developing training and education for the after-school workforce.In this study, the differences between students in CASE courses and students in courses with fieldwork suggest that the CASE program is on the right track toward better preparing ASP educators with respect to their understanding of relevant course material and perceived academic, civic, and career engagement.Important next steps for the evaluation include replication of these findings using a more rigorous research design and examining how these sorts of outcomes may relate to changes in program quality and child outcomes.

Table 1
Amount and type of fieldwork required in CASE courses.Spring 2009 course are included in the present study.
* Whether fieldwork is included varies by instructor and is not included as part of the 70 hours of CASE required fieldwork hours.
Students who complete CASE courses share a common interest in the domain of After-school Education.Also, the comprehensive nature of the CASE program allows CASE students to learn from each other and build relationships as a community of ASP educators.Additionally, sustained interaction through fieldwork and coursework allows CASE students, professors and current after-school educators to create a practice marked by shared repertoires.The comprehensive nature of the CASE program (involving completion of multiple courses and fieldwork hours) creates the COP of CASE.The current evaluation focuses on how CASE students may benefit from CASE as a COP, compared to students who are not part of the COP formed by CASE.

Table 2
Descriptive statistics for study variables separately for students in CASE courses with and without field work and students in a control course.
Note: Percentages refer to percent of students within the column.Actual values refer to means within a column.

Table 3
Regression coefficients of CASE courses compared to the control course

Table 4
Regression coefficients of CASE courses with fieldwork compared to CASE courses without fieldwork.Service Learning Measures.Our fourth hypothesis was partially supported.Two reported service learning scores significantly differed between students in CASE courses compared to students in the control course: civic engagement (B=.19, p < .01)andempowerment(B=.12, p < .05)(SeeTable3).Regression analysis did not reveal significant differences in comparisons between academic engagement or career engagement between students in CASE and non-CASE courses.Lastly, our fifth hypothesis was also partially confirmed.One service learning measure, academic engagement, significantly differed between CASE students in courses with fieldwork versus students in courses without fieldwork (B=.18, p < .01).No significant differences were found between empowerment, civic engagement or career engagement between students enrolled in CASE courses with fieldwork, compared to those enrolled in CASE courses without fieldwork.