Lesson Study Model of 4-H Professional Development : Data-Driven Improvements to Educator Practice

Educator professional development in the 4-H Youth Development Program needs improvement. Lesson study, an iterative and educator-centered approach to professional development, has proven to be effective in school-based settings; however, research on the model in youth development programs is limited. The present study investigated the use of lesson study with professional Cooperative Extension staff, adult 4-H volunteers, and teenage 4-H volunteers in different 4-H contexts in 3 state programs. Findings revealed improvements in educators’ data-driven decision-making, content knowledge, lesson planning and implementation, and social connections across all contexts. To work effectively within the 4-H context, however, some adaptations to the model were necessary.


Introduction and Background
Every year, millions of U.S. youth participate in programs that support positive youth development (PYD) (Lerner, J. V., Phelps, Forman, & Bowers, 2009).Participation in youth development programs has been shown to improve self-esteem, academic performance, empathy and caring, leadership skills, and civic engagement (Lerner, R. M. et al., 2005;Scales, Benson, Leffert, & Blyth, 2000).Important factors in realizing these PYD outcomes include the practices and capabilities of the educators responsible for the implementation of programs (Borden, Schlomer, & Wiggs, 2011).Thus, helping to develop capable, confident educators is crucial in realizing desired youth outcomes.
Improving educator professional development is one of the most pressing and challenging issues in education (Bissonenette & Caprino, 2014).The most common approaches to educator professional development-episodic workshops, conference presentations, seminars-are characterized as "one-shot, fix-'em-up experiences" (Fleischer & Fox, 2003, p. 259).Referred to as "traditional" methods (Garet, Porter, Desimone, Birman, & Yoon, 2001), such strategies are considered to be ineffective because they are fragmented, lack educator-centeredness, and are decontextualized (Penuel, Fishman, Yamaguchi, & Gallagher, 2007).As such, there has been a call for alternative approaches to educator professional development using strategies that are iterative, educator-centered, data-driven, job-embedded, and collaborative (Guskey & Yoon, 2009).Professional development models with these features are referred to as "reform-based" and are considered more effective (Garet et al., 2001).
With over 370,000 4-H volunteer and staff educators nationally (U.S.Department of Agriculture (USDA), 2017), effective professional development is essential to help prepare these individuals to deliver high quality programming with youth participants.However, findings from a recent study revealed that most state 4-H professional development opportunities are one-time, episodic events that occur at a set time and location with content delivered by outside experts (Smith et al., 2017)-traditional methods-which are often ineffective in improving educator practice (Penuel et al., 2007).Thus, Smith et al. (2017, para. 21) argued that there is a demonstrated "need to introduce more reform-based professional development opportunities into the 4-H landscape in a systematic and intentional manner." Lesson study is a reform-based approach to educator professional development shown to be effective in advancing educators' knowledge, skills, confidence, and pedagogical practices (Lewis, Perry, Hurd, & O'Connell, 2006;Lewis, Perry, & Murata, 2006).The lesson study model involves educators working in communities of practice whereby they develop a reflective orientation on their teaching (Rock & Wilson, 2005).The lesson study process is constructivistbased, iterative, occurs over an extended period, and is data-driven (Lewis & Hurd, 2011).
Specifically, lesson study follows a cycle whereby educators work together to (a) study a curriculum and formulate learning goals, (b) select or adapt a specific curriculum lesson curriculum activity, (c) implement the selected lesson and collect formative data to inform learning and teaching, and (d) use formative data to reflect on and refine the lesson and its implementation (Lewis & Hurd, 2011).
In the United States, the lesson study model has been used successfully by classroom teachers and pre-service educators in science, language arts, social studies, and mathematics (e.g., Doig & Groves, 2011;Lewis, Perry, Hurd, & O'Connell, 2006;Marble, 2006).Research has shown lesson study to have positive effects on educators' knowledge, skills, and confidence (e.g., Rock & Wilson, 2005;Wiburg & Brown, 2007).In contrast, empirical literature on the use of lesson study in youth development programs is limited (Smith, 2008;Smith, 2013).In 4-H, only one study involving the use of lesson study with adult volunteers in rural, club-based settings has been published (Smith, 2013).Results from this investigation provided evidence of the promise of lesson study in 4-H; however, the inquiry was limited in its size and scope.
The purpose of the present study was to assess the viability of lesson study in multiple 4-H contexts.The study occurred in three states involving three types of educators commonly used in 4-H (professional staff, adult volunteers, and teenage volunteers), different lesson study group meeting formats (online and in-person), and sundry curricula.

Methods
We utilized qualitative case study methodology (Yin, 2013) and thematic analysis (Braun & Clarke, 2006) to explore the use of lesson study in different 4-H contexts.A case study is an indepth investigation of a contextualized phenomenon that occurs within specified boundaries (Creswell, 2009;Hatch, 2002;Patton, 2015).In this specific case study, we examined the use of lesson study (contextualized phenomenon) that was bounded by process (the use of lesson study), program (4-H), and time (duration of the study).The goal of our inquiry was to investigate the application of the lesson study model of professional development in three authentic 4-H contexts (club, school enrichment, and after-school) with different 4-H educators (program staff, adult volunteers, and teen volunteers).Our research foci were twofold: identifying benefits to the educators, as well as the strengths and weaknesses of lesson study in a 4-H context.
For this investigation we selected 4-H programs using a stratified purposive sampling process to represent the four regions of the United States as defined by the National Association of Extension 4-H Agents (Central, Northeast, Southeast, and West) (NAE4-HA, n.d.) and 4-H delivery modes.Our sampling technique was intended to provide maximum variation; that is, by including a wide range of extremes, we expected our findings to be more representative than if we had employed another non-random sampling technique (Patton, 2015).Specifically, we identified four states and invited state 4-H program leaders to participate; three states completed their participation and were included in the study.
The three cases in the present study varied based on the following factors: (a) geography, (b) 4-H delivery modes, (c) curricula, (d) types of 4-H educators, (e) lesson study group meeting modes (in-person or virtual), and (f) meeting frequencies.The number of educators involved in each case's lesson study group varied slightly (range = 5-7).The total number of educators was 18; the number of sites (clubs and classrooms) where curriculum lessons were implemented was 21 (see Table 1).
The approach to lesson study we used in this investigation was modeled after the Japanese lesson study cycle (Doig & Groves, 2011;Lewis & Hurd, 2011): 1. Review the curriculum and formulate goals.As such, each lesson study group was expected to establish a meeting structure and schedule in order to ensure that the lesson study cycle was implemented effectively and maximum benefits were achieved.Additionally, project researchers provided coaching and process expertise to lesson study groups on a regular basis; logistical support was provided by statelevel coordinators.

Data Sources
Educator and Youth Reflection Tools Two formative assessment tools were used by educators in all three states: a youth clover reflection sheet (see Appendix A) that provided 4-H youth participants the opportunity to reflect by responding to simple prompts related to their learning experience (adapted from Arnold & Gifford, 2014); and an educator reflection tool (see Appendix B) that utilized a plus/delta approach, a common, straightforward strategy for educators to reflect on their practice (Fanning & Gaba, 2007).We analyzed the data from these reflection tools for outcome purposes in addition to their being used as a source of formative data by educators throughout the lesson study process.

Focus Group Interviews of Educators
Post-project focus group interviews were held with educators (three focus groups total, one per site).We developed a semi-structured interview protocol to explore educator experiences on the usefulness, benefits, and drawbacks of the lesson study process and formative data tools.
Focus group interview questions or prompts were as follows:  Describe your experience with the [name of site] lesson study project that you just completed.
 Explain how, if at all, the [name of site] lesson study supported your growth as an educator.
 Describe the strengths and weaknesses of the [name of site] lesson study process.
 Discuss how, if at all, the data collection tools, including the educator reflection tool and youth clover reflection sheets, informed your group members in their lesson planning. Please provide any additional comments.
The focus group interviews drew upon educators' experiences in order to understand their perceptions of and attitudes toward lesson study (see Krueger & Casey, 2016).Focus group interviews were recorded and transcribed.

Data Analysis
We applied thematic analysis, a method of qualitative research that "involves the searching across a data set-be that a number of interviews or focus groups . . .-to find repeated patterns of meaning" (Braun & Clarke, 2006, p. 86).Based on our research foci, we began with three pre-determined themes: (a) benefits of lesson study to 4-H educators, (b) strengths of the lesson study process in 4-H, and (c) opportunities for improvement using lesson study in 4-H contexts.Specifically, we analyzed focus group interview transcripts collaboratively, one case at a time, by segmenting transcript excerpts into one of the three themes.Subsequently, we analyzed excerpts inductively within each theme to identify patterns.We organized excerpts into similar groupings, based on consensus, where there were conceptual similarities; these groupings formed subthemes.Subsequently, we utilized educator reflection tool and youth reflection sheet data for triangulation.Both educator and youth reflection tools served to support and/or provide negative case evidence for our subthemes.Table 2 provides a summary of data collected and analyzed from all three sites.

Rigor
In qualitative research, rigor is established primarily through trustworthiness, distinct from validity and reliability found in quantitative studies (Saumure & Given, 2008).Trustworthiness includes transparency and reliability of methods, credible representation of data, and reflexivity of the researchers themselves (Saumure & Given, 2008).Establishing trustworthiness, in part, relies on including "multiple perspectives on the issue under study" (Eisenhart, 2006, p. 577).
Our process was consensus-based and designed to emphasize diverse perspectives.Bringing our three perspectives to bear on representations of data strengthened the accuracy and authenticity of claims.Our team's background was diverse, including experience in qualitative research methods, positive youth development, and educator professional development.
Additionally, we employed two forms of triangulation: data triangulation (between three cases and with three data sources) and analyst triangulation (three investigators analyzing data; Patton, 2015).

Findings Theme 1: Benefits of Lesson Study to 4-H Educators
Educators shared their perceptions of the lesson study process pertaining to their own development.These included improvements to data-driven decision-making, content knowledge, lesson planning and implementation, and social connections.

Subtheme: Data-Driven Decision-Making
Lesson study meetings, which involved individual and group reflections and discussions guided by the formative data, aided educators in making data-driven improvements to their teaching (e.g., clarifying directions and reviewing material).Specifically, educators reported being more deliberate with respect to their lessons and more attentive to their learners.Representative responses from educators included: When you have that written reflection from the kids and you can go back, I think you can see if there's any areas where as an educator that I can improve.If there was something I wanted them to get from the lesson and they don't mention that on the reflection tool, then it lets me know that maybe I need to look at how it was presented to the kids.
I was actually pleasantly surprised at the ability of the [youth] to reflect on the day, and it really helps me to see that I had met my goals and that I was getting somewhere, we were moving. . . .It was helpful and it was always applied to the next week and adjusting my teaching.
See Figures A2 and A3 for examples of youth reflections.
Additionally, many educators found the use of their own reflections beneficial to improving their skills in making data-driven decisions (see Figure B2).One educator commented, "[The educator reflection tool] really brought insights to how to approach the lesson and really look into a lesson beforehand to make sure we understand it."Another observed: [Our written reflections] were the way we found out the that I could better explain that language to the youth." However, not all educators reported improving their content knowledge.For example, at one site, less experienced educators improved their content knowledge because of their interactions with other lesson study group members who were more knowledgeable and experienced; in contrast, the more experienced educators in another lesson study group reported no improvement in their content knowledge.These more-experienced educators suggested that they could have advanced their own content knowledge if ". . .we had someone who [was more experienced] and who was like a true programmer."

Subtheme: Lesson Planning and Implementation
Educators described improving their understanding of lessons, confidence in facilitating them, and improving the implementation of lessons they were teaching as a result of their interactions

Theme 2: Strengths of the Lesson Study Process
Educators were asked to reflect on what they experienced as strengths of the lesson study process as a professional development strategy.We report on four emergent subthemes related to strengths of lesson study process as they pertain to the 4-H contexts in this study.

Subtheme: Leadership and Roles
Educators appreciated the emphasis on egalitarian decision-making, distributed leadership, and defined roles used in lesson study.Many felt comfortable with the different roles and being able to take part in leading the process.Representative responses included: "I liked the way we . . .
[rotated roles].It gave everyone an opportunity to take part in a leadership role.I think that was very important that we got everyone's voice somewhere along the way." and "We all got a chance to be the leader once or maybe a couple times. . . .I think it made them gain more selfconfidence."

Subtheme: Lesson Study Group Meeting Format
Both in-person meetings and virtual meetings worked well for educators, with each group expressing a strong preference for the format they used during the study.For example, those individuals who used a virtual format expressed a desire for using that approach, while those meeting in person preferred that format.Educator reflections included: "Virtual was better than meeting in person.I would not have wanted to try to get us all together somewhere.I would not have participated if that was the case." and also "I like meeting in person.It feels like you have something you can clue into easily and . . .I feel like when we are all in the same room we are more comfortable with each other.

Subtheme: Role of Content Experts
The availability of content expertise was reported to be beneficial to educators, and having experts from outside the lesson study group or as group members proved to be valuable.One educator expressed that having a content expert as a member of their lesson study group helped advance understanding and build confidence.She stated: "I had no background in . . .Overall, educators reported benefits of capturing youths' perceptions using the youth clover reflection sheet.Some educators responded that youth completed the clover reflection sheets enthusiastically and saw them as an opportunity for the youth to reflect on their learning: Educators also saw the benefit of being able to assess youth learning through their selfreflections.One educator commented: "It was also very affirming for the trainers to see the kind of impact that they were having that we weren't really necessarily able to observe in any other way other than when they did the nice little reflection sheets." Responses were generally positive with respect to the educator reflection tool and the opportunities provided to reflect.This educator's comment was representative of the responses: I also think that it was good to do [the educator reflection tool] right after the session. .Educators were also asked to comment on areas of the lesson study process they thought could be strengthened.We highlight three subthemes that signal recommended areas for improvements identified by educators: timing and frequency of meetings, lesson implementation schedules, and formative data tools.

Subtheme: Timing and Frequency of Meetings
Reports were mixed on preferred timing (hour of the day) and frequency (e.g., monthly; every 2 weeks; once per week) of lesson study group meetings.In one lesson study group, educators expressed a desire for longer meetings.They accomplished this by expanding their last lesson study group meeting to one hour and forty-five minutes.One member said, "It was also nice not to have a huge time crunch so that each idea was not rushing through it."Another educator from the same lesson study group requested to meet on weekends and not weekday evenings.
This educator thought that if meetings were on weekends that they (teenagers) would have time to complete homework for school.

Subtheme: Lesson Implementation Schedules
Educators in one state reported varying schedules for the implementation of curriculum lessons posed a challenge for their group.In their opinions, this resulted in complicating lesson study planning somewhat.For example, one educator shared: "Some of us had already been in our clubs when we did this, so for some we had to backtrack . . .I think [a weakness was] just the timing of the clubs starting at different points."This was echoed by another who said, "It made it a little bit tricky because we did all started at different points." Educators in another state recommended that a resource be developed and included to help lesson study group members navigate the meeting process when implementation sequences vary.
Subtheme: Formative Data Tools Despite benefits described previously, some educators shared mixed reviews on youths' perceptions of the clover reflection sheet.Some responded that youth were frustrated when Key benefits to the majority of educators in this investigation included improved content knowledge, confidence in and understanding of lesson planning and implementation, and social connections.Strengths included distributed leadership whereby discussions were egalitarian in nature, the use of outside content experts to help build competence and confidence, and the use of formative data to inform the revisions of lessons and improvements to teaching.These outcomes reflected similar results described in the literature as it pertains to the use of lesson study in school-based settings.For example, Rock and Wilson (2005) showed that teachers increased their professional confidence, improved their subject matter knowledge, and recognized their enhanced peer collaborations through lesson study; Marble (2006) described improvements among participating pre-service teachers relative to their teaching skills and their abilities to develop and implement effective science lessons; and Dudley (2011) described how teachers engaged in lesson study used formative data to inform decisions about teaching and learning.
As a fundamental element of lesson study, reflective practice creates an environment for educators to challenge preconceived notions about teaching and learning, and thus is a positive attribute of effective professional development (Dantonio, 2001;Lewis & Hurd, 2011).
According to Osterman (1990), reflective practice helps educators develop ". . .new knowledge about professional practice, and to a broader understanding of the problems that confront [them as] practitioners" (p.134).In this study, reflective practice was supported by the educator reflection tool and youth clover reflections.These tools helped educators reflect on their practice and proved to be beneficial for most participants.However, some educators found these tools used in this investigation to be redundant (educator reflection tool) or not developmentally appropriate (youth clover reflection sheet).Thus, we recommend that other strategies to collect formative data be explored (e.g., observations, question prompts, curriculum artifacts, discussions, presentations).
Out-of-school time programs like 4-H differ from schools in a number of ways.Most notably, they occur after school, on weekends, or during school breaks, and are typically led by one adult staff member or volunteer educator; program frequency and duration are variable; and, 4-H clubs or projects are generally located over a broad geographic area (Eshach, 2007;Norland, 2005;Walker & Dunham, 2002).Recognizing these important differences, we adapted the traditional Japanese lesson study model to work within the 4-H context.Instead of a thirdparty observer as the main source of formative data (Fernandez, 2002;Lewis & Hurd, 2011), educators used written reflective data from youth participants (see Appendix A) and their own post-implementation reflections (see Appendix B) to inform lesson revisions and modifications to teaching.Overall, the adaptation complemented the traditional Japanese model by ensuring that sources of formative feedback were available to educators at both the individual and group level.
The traditional lesson study model in school-based settings also places an emphasis on reteaching a revised lesson by another member of the lesson study group (Fernandez, 2002;Lewis & Hurd, 2011).Re-teaching is encouraged in that it provides opportunities for educators to see the impacts of the data-driven decisions they made in revising a lesson (Lewis & Hurd, 2011).While the concept of re-teaching a revised lesson was not emphasized in this study, it occurred in some instances due to the variable implementation schedules among 4-H educators (e.g., weekly; bi-weekly; every three weeks; monthly).However, in this inquiry the variable implementation schedule posed a challenge for one lesson study group whereby some group members believed it complicated their lesson planning.In contrast, it was reported in a previous study with 4-H (Smith, 2013) that varying lesson implementation schedules were beneficial.In the aforementioned investigation, formative data from lesson study group members whose implementation schedules were shorter (e.g., weekly; bi-weekly) were used to inform lesson revisions; subsequently, group members whose implementation schedules were longer (e.g., every three weeks; monthly) re-taught the revised lessons.Given the complex nature of varying implementation schedules among 4-H educators, we recommend that future inquiries using lesson study in out-of-school time programs place an emphasis on re-teaching revised lessons and investigate strategies to capitalize on opportunities where it is possible.
We also recommend two other avenues for future inquiry on the use of lesson study in out-ofschool time programs: meeting modes and youth learning.Although the lesson study group in State #3 (Southern United States) met one time in person, there was not a systematic effort by any of the three lesson study groups in this pilot test to utilize blended learning.Blended learning involves regularly scheduled meetings over an extended period of time, some of which are in-person while others are online (Hino & Kahn, 2016), and represents an instructional option for lesson study that should be explored in the future.According to Hino & Khan (2016), blended learning opportunities have a "major role to play in Extension educational programming."Additionally, while lesson study is intended to improve educator abilities and confidence, it is also intended to help improve learner outcomes.Thus, future studies may want to examine or compare learner outcomes in various subject matter domains.One possibility may be comparing learner outcomes from educators in a lesson study community compared to learner outcomes from educators participating in more traditional professional development.

Conclusion
Lesson study is a viable option for professional development in 4-H.The strategy incorporates systematic reflective practice to support teaching and learning, and can help program staff and volunteers use formative evaluation to support data-driven improvements to lessons, their teaching practice, program implementation methods, and content knowledge.Limitations to our investigation include a small sample size (both in the number of cases and the number of educators in each case), the short duration of the study, and limited delivery mode options, thus, we caution against generalizing beyond our study's context.Further research is recommended on the use of the model in additional 4-H contexts with larger sample sizes, additional 4-H delivery modes, varied content matter, and a wider age range of youth.To help accomplish this, the lesson study model is currently being tested nationally using a structured implementation strategy involving 10 land-grant universities.
Educators influence PYD outcomes, and thus, educator professional development is crucial in realizing youth outcomes.We believe the use of lesson study could be a viable professional development option for other youth development organizations.We encourage additional investigation into the use of lesson study in youth development programs.
2. Plan a lesson.3. Implement the lesson and collect formative data.4. Reflect on data. 5. Revise the lesson based on formative data collected.
[youth]    weren't really familiar with [food] labels at all. . ..[We]  added a lesson in to fix that. . . .It allowed us to adjust and build upon information, to strengthen [our lessons].Subtheme: Content Knowledge Educators reported improving their content knowledge related to the curricula they were implementing.This was exemplified in one educator's comment: "The meetings helped . . .[us gain] the education about knowing about nutrition [content]."Another educator commented, "I would say that I improved my knowledge of the vocabulary that goes with [the curriculum] so nutrition. . . .[Having] nutrition educators [in the group] . . .[helped me] check myself and make sure I knew what I was teaching."Additionally, one site had a mix of more-and less-experienced educators.The less-experienced educators felt having more-experienced educators as part of their lesson study group was beneficial.One representative comment included: "I felt that the experience of the other leaders was a great strength for me because they actually helped me with a lot of the problem solving."Subtheme: Formative Data Tools.Youth Clover Reflection Sheet and Educator Reflection Tool.

[
The youth] enjoyed filling them out and enjoyed reflecting on them.They were pretty positive on everything.I think it gave them a good opportunity to think about what they did, and what they learned and what was important of the whole lesson.
. .It is nice to slow down and think about what we did and talk about it as team before you move onto the next one, so that you may change things like when they come in.Another educator observed: "It was good to see what we needed to improve on and it helped to talk it through with the group when we are planning our next lessons to see what we could do differently."Theme 3: Lesson Study Process -Opportunities for Improvement completing them: "It was difficult to get the kids to really focus on short answers and present the answers," and "[The Reflection Sheets] felt like school.[They were] too much like being in class and [doing] additional work."Although most educators saw the benefits of the educator reflection tool, some indicated that the process was repetitive in nature.One educator reflected: ". . . it seemed kind of redundant in different places that I feel like I'm writing the same thing over and over."Another said, "[The educator reflection tool] was very repetitive and you are saying the same over and over."Discussion Building upon Smith (2013), this study provided additional evidence that the lesson study model of professional development has potential for use in the 4-H Youth Development Program.Specifically, the study provided insights into the use of lesson study in different 4-H contexts with a variety of types of curricula.Furthermore, the investigation demonstrated that lesson study has the potential to work with varying types of out-of-school time educators (teenagers, adult volunteers, and professional staff), meeting schedules, and meeting formats (online and in person).

Figure
Figure A2.Example Response: Clover Reflection Sheet From Nutrition Curriculum Activity

Table 1 . Case Descriptions State; U.S. Region Subject matter curriculum Educators Youth Delivery mode 1 Time period 2016 Delivery start; Meeting frequency Lesson study meeting mode
2 SPIN = 4-H special interest club.