Building Evaluation Capacity in Youth-Serving Organizations Through Evaluation Advisory Boards

Youth-serving organizations seek effective and cost-efficient solutions to build evidence and advance their impact. Some common challenges include choosing data systems or assessments, budgeting and planning for 3 rd -party studies, and refining measurement and outcomes when programs expand or change. Evaluation advisory boards (EABs) are a low-cost solution to add evaluation capacity and can be mutually beneficial to both youth-serving organizations and evaluation experts. Previous research suggests that EABs may encourage meaningful use of data, support internal evaluators, and/or facilitate difficult conversations among stakeholders. However, there are very few examples of successful EABs in practice. This paper shares the perspectives of EAB members and organizational evaluation leaders from a large national after-school program, After-School All-Stars (ASAS), including (a) a description of the benefits of EABs, (b) how EABs may be especially helpful with the context of the COVID-19 pandemic, and (c) examples of youth- serving organizations’ EABs. The experiences and lessons learned by ASAS and its EAB are generalizable to other non-profit youth development programs. Recommendations for structuring EABs based on organizational goals are provided. The Research and Evaluation Strategy Committee (RESC) of the Expanded Learning Division (EXLD) at the California Department of Education (CDE) is composed of a diverse group of 12 to 15 program evaluators, researchers, program directors, and technical assistance providers across the state of California. Members from the RESC and the EXLD generally meet three times a year to provide strategic direction to the research arm of the EXLD. Topics discussed have included strengths and gaps of existing research reports produced by the EXLD; ways to hone future research to address those gaps; recommendations to help drive program quality in the field; and most importantly, ensuring that all research and evaluation efforts would be meaningful and relevant to practitioners in the field. Advisory board and EXLD conversations led to tangible results, including statewide development of a theory of change to guide strategic evaluation plans; revised data collection plans and instruments to meet the diverse needs of California programs; and guidance for using data to drive continuous quality improvement across California's funded programs.


Introduction
Youth-serving organizations face multiple challenges and restricted resources even in the best of times. Conducting quality program evaluations is a chronic challenge, and many programs do not have the resources or networks to engage qualified or competent evaluation professionals.
Day to day, programs may have the capacity to collect data to monitor their progress toward goals. However, there are critical times when external advisors may be particularly important to advance evaluation, such as when • designing or redesigning a program-describing activities and intended outcomes in logic models; • launching a program-finding or designing tools and systems to track youth experiences and outcomes; • implementing a program-setting up systems to analyze and review data and use it to inform decision making; • sustaining and improving a program-building evidence and seeking to design or redesign evaluation studies; and • responding and adapting to changes in funding, policies, or contexts that create challenges and lead to changes in program design or scale.
Many of these evaluation practices are encapsulated within the broader field of evaluation capacity building (ECB; e.g., Labin et al., 2012;Preskill & Boyle, 2008). ECB aims to develop practitioners' knowledge and skills and improve organizational processes and structures to create sustainable evaluation practice, which can strengthen the quality of implementation and outcomes.

54
The current article proposes evaluation advisory boards (EABs) as a promising strategy for building evaluation capacity within youth-serving organizations, particularly during times of crisis, such as the COVID-19 pandemic. The authors posit that EABs are an under-utilized yet high-potential strategy that could address these two inter-related goals of ECB initiatives: (a) supporting individual practitioners through informal training, coaching, and technical assistance; and (b) informing the larger organizational systems and structures that support high-quality, sustainable, evaluation practice (Preskill & Boyle, 2008). In this article, the authors define evaluation advisory boards, discuss ideal board composition and organizational readiness factors, summarize the potential roles of EABs, and identify the benefits and limitations of EAB development, using several real-world examples of EABs from youth development nonprofits.
Ultimately, EABs offer a low-cost solution to help youth programs access external advisors regularly and flexibly in response to their typical ongoing and evolving evaluation needs. The COVID-19 pandemic created new levels of disruption across youth development agencies (Ettekal & Agans, 2020), presenting challenges at a speed and level of complexity that underscored ways EABs could assist in this context too.

Facing Challenges in the Context of COVID-19
The onset of the COVID-19 pandemic in 2020 uprooted youth, families, and the youth development organizations and institutions that serve them (Arnold & Rennekamp, 2020). As stressed by Ettekal and Agans (2020), "During the pandemic, youth ecological systems are in turmoil, encountering changes in the routines and habits associated with daily life, interruptions in information flow and communication patterns across settings, and intense and abrupt changes in rules and regulations" (p. 3). Public policy groups predicted that youth and young adults would suffer the most in terms of adverse economic and social consequences related to the pandemic (Fine et al., 2020). Furthermore, school closures and social distancing measures disrupted youth routines, increased social isolation, and impeded access to services usually provided by schools and community-based organizations (Lee, 2020). Children with risk factors, particularly those with exceptional mental health needs, in foster care, at risk of neglect or abuse, or with complex medical conditions, face many potential negative impacts from COVID-19 (Wong et al., 2020).
COVID-19 appears to impact historically marginalized racial and ethnic minority groups more severely from medical (Kirby, 2020) as well as social and economic standpoints (Laurencin & McClinton, 2020). Many in these communities rely on the programs and services offered by youth-serving organizations and institutions.
However, most youth-serving organizations have faced a massive loss of revenue during the pandemic. In general, nonprofits tend to operate with minimal reserves and are forced to significantly reduce, alter, or cease operations in the event of significant revenue declines. It is within this broader ecological context that youth-serving organizations have been challenged to operate, pivoting to diverse virtual or blended program and service delivery models (Afterschool Alliance, 2020; Boys and Girls Clubs of America, 2020) while cutting expenses deemed nonessential (Ettekal & Agans, 2020) in an already competitive fiscal environment (Keller, 2010). Yet, these types of challenges are precisely where program evaluators in general, and EABs in particular, can provide necessary guidance, support, and insight to ensure that programs can do the greatest good with organizations' dwindling resources. In the next section, the authors define EABs in greater detail and provide examples of the kinds of supports they can provide.

Building
Although EABs have been referred to by different names (e.g., evaluation advisory committee, evaluation task force, evaluation working group, evaluation steering committee, and evaluation team; Preskill & Tzavaras, 2006), EABs generally provide insight, strategic support, and advice on program design, measures, implementation, impacts, and use of evaluation findings. As defined by Baizerman et al. (2012), evaluation consultation/advisory groups are "an intentionally organized and managed formal structure composed of competent and willing individual members who have agreed to proffer (offer) useful advice on how to create, conduct, and use one or more evaluation studies" (p. 11). They indicate that proffered advice is used to "enhance the quality of an evaluation (technical concerns), the conduct of the evaluation (process concerns), and the use of evaluation findings (utilization concerns)" (pp. 10-11). Thus, it is important to emphasize that EABs are concerned with offering advice and guidance but are not parties that conduct a significant part of the actual evaluation-related work; that is mainly left to organization staff and evaluator contractors.
This paper expands this EAB definition by incorporating evaluative thinking into organizational practices. Evaluative thinking (Berry & Sloper, 2020) is a critical concept in evaluation and embedded conceptually in the American Evaluation Association Guiding Principles and Joint Committee Standards (Patton, 2018). Buckley et al. (2015) define evaluative thinking as, "critical thinking applied in the context of evaluation, motivated by an attitude of inquisitiveness and a belief in the value of evidence, that involves identifying assumptions, posing thoughtful questions, pursuing deeper understanding through reflection and perspective taking, and informing decisions in preparation for action" (p. 378). Evaluative thinking not only entails critical thinking but also creative thinking, inferential thinking, and practical thinking (Patton, 2018). EABs can strengthen evaluative thinking in youth-serving organizations, as well as facilitate greater attention on the "positionality, assumptions, motivations, and biases'' (Vo et al., 2018, p.44) of youth-serving organizational leaders by capitalizing on practitioners' motivation, interests, curiosity, and immediate needs, especially within the context of continuous quality improvement (Berry & Sloper, 2021). EABs can improve organizational capacity by maximizing external credibility (Mattessich, 2012), improving the validity of findings (Grembowski, 2001), promoting methodological credibility (Baizerman et al., 2012), encouraging the use of results (Mattessich, 2012), and even decolonizing the research process if community-based advisors are involved (Johnston-Goodstar, 2012). Done well, EABs can create space for youth-serving organizations to critically challenge internal systems and assumptions, improve their efficiency and effectiveness, and ultimately empower internal evaluation.
The structure, composition, size, and activities of EABs typically vary, allowing agencies to determine which type of board "best suit(s) them to function effectively for a specific project within a specific context" (Mattessich, 2012, p. 32). This flexible structure allows EABs to meet agencies where they are, quickly pivot as needs change, and provide rapid feedback in line with agencies' immediate needs. Over time, engaging in ongoing, responsive, and informative evaluation guidance can stimulate better evaluative thinking (Buckley et al., 2015) and lead to enhanced evaluation capacity building (see Preskill & Boyle [2008] for a comprehensive model of evaluation capacity building).
Historically, many youth-serving organizations have prioritized program and service delivery over strategic and intentional formative and summative assessments (Arnold & Cater, 2011).
Efforts to build evaluation capacity within youth-serving organizations have often been challenged by limited resources (Ouellette et al., 2020;Rennekamp & Arnold, 2009), poorly trained staff (Baughman et al., 2012), lack of an evaluation champion (Silliman et al., 2016), absence of clear staff expectations related to evaluation (Altarum Institute, 2012), little confidence in the validity of the evaluation tools being used (Lovell et al., 2016), and a need for an evaluation culture within the organization (Bourgeois & Cousins, 2013). In recent years, youth-serving organizations at the national, regional, and local levels have demonstrated increased interest in evaluation through the development, implementation, and resourcing of program theory of change logic models, quality improvement monitoring systems, and more rigorous formative and summative evaluation approaches (Arnold et al., 2016;Arnold & Silliman, 2017;Maley et al., 2016). Within this context of burgeoning evaluation acumen and organizational readiness within the youth development field, EABs have the potential to build capacity within various departments and functions without significantly increasing human resource overhead and other associated costs, particularly if EAB members perform such service as volunteers or for honoraria.
Although EABs are not intended to supplant the work of internal or external evaluation consultants, they can help organizations strategically prioritize how to maximize their often "shoe-string budgets" as discussed by Bamberger et al. (2006). EAB conversations with practitioners can be informed by the immediate needs of the organization, and when coupled with an evaluative thinking lens, EABs can contribute meaningfully to youth-serving organizations. They can also enhance capacity in multiple other important ways, including • providing real-time support and guidance; • fostering evaluative culture and evaluative thinking; • enhancing evaluative rigor, including articulating logic models, defining program quality, and advancing outcome measurement; • helping programs organize and express activities according to established theories of youth development; • sharing examples of successful evaluation practices and tools from other programs and contexts; • connecting program staff to youth development peers, experts, and literature; • aiding program staff in the identification of appropriate funding opportunities from a variety of sources (e.g., federal grant programs, foundation offerings) and developing strategies to guide the development of grant proposals and funding requests; • assisting with writing requests for proposals and determining suitable external evaluation partners, and advising on internal staffing strategies to execute an evaluation; • catalyzing organizational change management as an expert body; • identifying research or evidence gaps and strategies to fill them; and • validating the rigor of organizational efforts to external audiences.

Benefits of an Evaluation Advisory Board During a Period of Crisis
EABs can help programs develop evaluation systems that can improve implementation and impact, particularly during periods of financial stress caused by crises such as the COVID-19 pandemic. Through this work, programs may be better able to make the case to funders and donors that their services are necessary and worthy of continuation. EABs can also help organizations diversify their funding through securing competitive federal, state, and foundation research grants. Funding agencies will need to make hard choices, and some programs providing critical services to youth may not survive. Youth-serving organizations that build responsive evaluation capacity will be better positioned to sustain their efforts to survive the impact of diverse crises including pandemics. Having an EAB in place helps youth-serving organizations be nimble in adjusting a program's evaluation to fit current conditions. If an EAB is not in place, program staff and advocates may (under certain conditions) want to engage an EAB to help them focus on what can or should be done to adjust programs and document outcomes in response to current conditions. During the COVID-19 pandemic, the authors have witnessed multiple ways that youth-serving organizations had to make drastic changes to their staffing and service delivery approach. With the constantly evolving community needs during COVID-19, evaluation capacity must be nimble and responsive. The programs the authors work with have shifted the nature of activities they are providing (e.g., from running after-school programming to delivering food to families) and the methods they are using (e.g., from in-person to virtual tutoring). These shifts mean that many programs need to assess existing activities as well as track new activities, and therefore need to update the databases, systems, and tools they use. Beyond the content of what they track or measure, the authors have seen that many had to create and implement new methods of data collection, communication about the evaluation, and related processes. For instance, programs that relied on paper-based tools and in-person opportunities to encourage youth and teachers to complete surveys had to rapidly consider designing and deploying new electronic surveys and consider issues of access to technology and data security, many with little or no experience. Several programs had to lay off staff due to loss of funding, or they lost access to critical volunteers (e.g., university students) to deliver their programs to youth. The settings and partners in their efforts (e.g., schools, parks, universities) are also suspending or changing services.
Although many funders extended timelines for programs to fulfill their deliverables, these funders still expect data about the new work that programs are doing, which means alternative ways of collecting and analyzing data and reporting may be needed. With uncertainty about when, where, for whom, and how long program activities may change or need to open and close again, changes in program implementation and loss of staff create new and urgent needs to modify evaluation practices. In the authors' experience, complex strategic thinking and problem solving within evaluation tests the abilities of even the most experienced evaluators.
Usually, evaluators and program providers can sequence the design or selection of tools and systems; the collection of data, analyses, and interpretation; and the use of data in ways that are developmentally appropriate for a program. Amid a crisis such as the COVID-19 pandemic, evaluative thinking is needed but at a rate of complexity and speed that few organizations can navigate alone. EABs can provide critical support to organizations during this time, and help organizations position themselves well for innovative success during their recovery from the impact of the pandemic, what might be described as the "after times." Specifically, the authors offer four examples of ways that an EAB can support during this critical time: 1. If university professors serve on the EAB, they can connect the organization with graduate students for support. This could serve as a low-cost, temporary capacitybuilding effort to address evaluation needs when staff are stretched thin.
2. The EAB can provide counsel on evaluation methods for new program services (e.g., food delivery) either through their own expertise or through their professional network.
3. The EAB can provide strategic advice on prioritizing evaluative tasks.
4. The EAB can provide near real-time advice on research-based strategies, particularly for new program areas or methods of delivery (e.g., shifts to virtual tutoring).
Additionally, when evaluation capacity building is considered within the context of organizational change or crises, such as the COVID-19 pandemic, then organizational change models (Batras et al., 2016;Bundy et al., 2017) can also provide a conceptual foundation for such work. For example, Bundy et al.'s (2017) three-stage organization crisis management model identifies priorities youth-serving organizations may address at different stages of change/crisis, including pre-crisis prevention, crisis management, and post-crisis outcomes. Using such a framework, youth-serving organizations may consider how building evaluation capacity within each stage could advance program and organizational outcomes. Another example is organizational stage theory (Rogers, 2003), which identifies how organizations change over time across four stages (i.e., awareness of a problem and possible solutions, deciding to adopt a recognized innovation, modifying organizational structures to implement the innovation, and institutionalizing the innovation in the organization's activities). Evaluation capacity building through development of an EAB can be an innovation that organizations may adopt during times of change, or it can be a strategy that youth-serving organizations use to better understand their programming and identify needed innovations proactively.

Examples of Evaluation Advisory Boards
The following examples were identified based on the authors' youth development networks as

After-School All-Stars (ASAS)
ASAS is a nonprofit organization that provides comprehensive after-school programs through 19 chapters across the country. In 2015, ASAS created a National Evaluation Advisory Board (NEAB) to assist in the development of a national framework for evaluating their after-school programs. Former members of the NEAB, who represent the authors of this paper, included youth development and educational researchers, evaluation consultants, professors, and afterschool experts who met quarterly to inform ASAS programming, strategy, and evaluation infrastructure. The NEAB was instrumental to ASAS change management efforts, driving transparency around organizational pain points and leading an increased focus and investment in rigorous, standards-aligned program quality. The NEAB's perspective as an external body magnified internal evaluators' voices to organizational leadership, critically important to manage change and enhance the internal capacity for evaluation. Among other initiatives, the NEAB's work informed ASAS decisions about evaluation staffing and reporting processes, data system and measurement tool development, the articulation of logic models, and the timing/staging of internal and external evaluation studies. ASAS experienced increased capacity building directly through its EAB, leading to enhanced internal evaluation expertise among staff, profound and vital shifts to the logic model and accordant program model, and organizational change management.
As a specific example, the NEAB served as a strategic sounding board in ASAS' North Star program model change-management process. Led by internal evaluation staff, the North Star process convened an impact working group composed of stakeholders throughout the organization who reviewed key data points, program implementation, and program staff practices to reexamine the organizational program model, with an eye toward redefining impact and integral program practices. In conjunction with the NEAB's theory of change support, the North Star process resulted in a new national focus on broadened impact with a socialemotional learning (SEL) emphasis, as well as a series of program pilots focused on staff retention and program quality. These included the launch of a validated SEL curriculum and instructional coaching for line staff to increase the consistent quality of instruction and relevance of the subject matter youth would experience. Early pilot data reflected promising trends in line staff retention, program quality, and youth SEL development at participating sites.

American Camp Association (ACA)
ACA is the leading nonprofit working to enrich lives through the camp experience. Providing

The Research and Evaluation Strategy Committee (RESC) of the Expanded Learning Division (EXLD) at the California Department of Education (CDE) is composed of a diverse group of 12
to 15 program evaluators, researchers, program directors, and technical assistance providers across the state of California. Members from the RESC and the EXLD generally meet three times a year to provide strategic direction to the research arm of the EXLD. Topics discussed have included strengths and gaps of existing research reports produced by the EXLD; ways to hone future research to address those gaps; recommendations to help drive program quality in the field; and most importantly, ensuring that all research and evaluation efforts would be meaningful and relevant to practitioners in the field. Advisory board and EXLD conversations led to tangible results, including statewide development of a theory of change to guide strategic evaluation plans; revised data collection plans and instruments to meet the diverse needs of California programs; and guidance for using data to drive continuous quality improvement across California's funded programs.

An Alternative Perspective on EABs
The EAB examples reviewed above were selected as they represent established EAB models that may serve as exemplars useful to other youth-serving organizations. While these examples represent national-level organizations, the experience of a smaller organization in developing an EAB is not discussed in this paper.
It is important to note that not all efforts at initiating an EAB are successful. One organization the authors identified in preparing this article, which preferred to remain anonymous, provided a note of caution. Although the attempted formation of the EAB for this organization was led by a Ph.D.-level research and evaluation staff member, with a thoughtful design (i.e., one that identified goals, benefits, tasks and responsibilities, and types of members), the effort failed.
The broader organization did not have a culture of valuing external input on evaluation and research, and that became a major obstacle. In part, this was because the research and evaluation function within the organization was relatively new and still being established and had not been part of the fabric of the organization previously. There was no active opposition to the idea of an EAB, but it was not integrated into the organization's work, and there were questions regarding why outside input was needed at an early stage. This example highlights key elements needed to support the success of EABs, namely: (a) an organizational culture that values evaluation, or is curious to learn more, needs to be present; and (b) the timing of when an EAB is introduced needs to align with broader organizational readiness and priorities.

Building an Evaluation Advisory Board
Effective board member recruitment is critical to a high-performing EAB. Content expertise and diversity in roles, lived experiences, and perspectives are key factors. For example, ASAS selected EAB members with varied areas of expertise specific to ASAS's program domains, including summer, health, and positive youth development. Furthermore, ASAS recruited researchers, external evaluators, and internal evaluators to bring various perspectives in advancing organizational evaluation capacity. In addition to content and experience, methodological expertise may be an important recruitment factor (e.g., including members with quantitative, qualitative, and mixed methods foci). Representation is also essential when considering EAB membership, as members may include organizational staff, recipients of programs and services, and community stakeholders. Diversity of opinions, experiences, and expertise, in addition to a broad demographic representation, contribute to the robust diversity of a group (U.S. Department of Education, 2008). It is also crucial for EABs to have input from those who have personal knowledge or experience of programs and communities. The larger the EAB, the greater the possible range in content experience, methodological expertise, and potential external reach of the board in engaging relevant stakeholders for organizational involvement. Larger boards, however, also demand greater management and coordination efforts and risk member disengagement (BoardSource, 2017).
Engaging board champions early in the process of creating an EAB helps facilitate recruitment because board member candidates are more likely to join a body with enthusiastic colleagues who appeal to their work interests. These early champions can also be helpful in establishing a recruitment approach to identify qualified candidates recommended by fellow members (Cohen, 2012), while also being intentional about engaging diverse members who bring a variety of perspectives. In doing so, youth-serving organizations can be mindful of how their research and evaluation environments (including, for example, the creation of an EAB) offer equitable opportunities (Outley & Blyth, 2020). A final factor in recruitment is selecting members who will work well together (Mariama-Arthur, 2017). Ultimately, participation in an EAB will usually be a non-salaried experience and members should enjoy the experience if organizations seek to have an engaged board with high member retention. The authors recommend the following strategies for youth-serving organizations interested in developing an EAB: • Identify and prioritize areas that would be most helpful for leveraging external expertise (e.g., planning how to evaluate new programs or new ways of delivering programs).
• Look for opportunities to "build" something new that would advance the organization's mission (e.g., a new culture of evaluation, new ways of using data for organizational learning, or new methods to measure youth development work and outcomes).
• Take advantage of the momentum from rapid change to address organizational needs around evaluation, advocacy, funding, partnerships, and sustainability.
• Locate additional and diverse funding opportunities and ways evaluation can support new and existing grants.
• Consolidate data already collected, analyze it, and explore ways to use it.
• Implement basic evaluation approaches to test and learn from pilots of new ways of delivering programs (e.g., virtual and blended approaches) to inform future programming efforts.
If you are feeling overwhelmed and not sure where to begin with building an EAB, the authors recommend starting small. Using your network and the network of those around you (e.g., a supervisor or leader at your organization), find one or two EAB members, then hold an initial meeting. Through that effort, you are likely to better understand what you are truly seeking to accomplish through an EAB and whose expertise will be needed. Then, you can leverage the networks of your first board members to reach out to additional members. An additional, initial recommendation would be to reach out to local professional groups for board member recommendations. For example, contacting your local American Evaluation Association chapter would yield a list of members with varied skills and experiences. A third practical tip would be to connect with a program officer with whom you and/or your organization hold a relationship.
Foundations are tapped into the research, evaluation, and provider communities and should be willing to make introductions to further your capacity and bolster their investment in you.
Developing and maintaining an EAB is a relatively low-cost endeavor with significant possible organizational returns. For instance, EAB meetings can be conducted efficiently at a distance, resulting in limited travel-related expenses. Many researchers, evaluators, and community members are interested in offering their time pro-bono to organizations, if they perceive that their input can make a significant and positive impact on the lives of youth and their families.
Furthermore, university-affiliated researchers and evaluators typically have community service requirements, which EABs can fulfill if they are voluntary. It is important to caution that since EABs are primarily advisory in nature, there are technical evaluation-related tasks (e.g., survey validation, data analysis) that will generally not be suitable to them, unless commensurate compensation is provided. In these cases, the assistance of external evaluation consultants may be necessary.
Many organizations have been challenged to adopt innovative practices of service delivery that are markedly different from what they typically provided due to COVID-19. Organizations need to maintain the flexibility to switch between online, face-to-face, and hybrid service delivery models for an indeterminate time. While youth-serving organizations may have experience and an evidence base regarding effectiveness in relation to the practices they have used with youth and families; determining the impact of new or modified modes of delivery poses a significant challenge. EABs can provide insights and guidance related to program design, alterations, and program testing, and examine the effectiveness of newly developed efforts. While external evaluators may vary by specific projects, EABs provide the opportunity to have consistent guidance, over time, from experienced evaluators for organizations committed to developing an evaluative thinking culture.
The opportunity to benefit from external perspectives and expertise while building internal capacity is not specific to evaluation. Beyond EABs, similar advisory structures focused on other organizational functions ranging from curriculum to marketing could prove of great value to youth-serving organizations. For example, the Standards and Curriculum Advisory Board for the National Financial Educators Council advances educator and learner standards while reviewing lesson plans and providing resources within financial literacy (National Financial Educators Council, 2019). Particularly in a post-COVID-19 setting, where resource scarcity may limit fulltime positions, content-specific advisory boards could present an efficient structure for organizations to stretch capacity. As organizations consider this structure, it may be advisable to consider where expertise could be garnered for functions unrepresented within the organization's governance board or operations team (BoardSource, 2016).

Conclusions
EABs present a strategic opportunity at a time when youth-serving organizations are grappling with increased need, rapid changes to program model delivery, and in many cases, reduced capacity to deliver programs and evaluate the efficacy of what is and is not working in this new context. Under these conditions, EABs may allow organizations to bolster capacity in a low-cost manner and increase expertise to engage in more rigorous and rapid forms of evaluation to meet the needs of the moment. Similarly, an EAB structure could be used to guide youthserving organizations in other expert capacities, such as virtual curriculum development, that may not be otherwise advanced by a nonprofit governing board.
It is the experience of the authors, from both the organizational and EAB member perspectives, that EABs hold great potential for youth-serving organizations. The authors' efforts on the ASAS EAB increased national program coherence across a large nonprofit, deepened and clarified efforts to bolster program quality, and furthered a range of evidence-based program initiatives that enhanced impact for youth served. Moreover, as board members, the authors learned from each other, formed new professional and enjoyable social connections, and appreciated having a new way to serve the broader goal of youth development. These potential benefits to programs, youth, experts, and much more can be harnessed through a well-designed EAB.
Further research is needed to elucidate ideal contextual and enabling factors for EAB effectiveness. Focus areas could include matching EAB composition and focus to organizational need, capacity, and will; ensuring individuals with diverse perspectives and backgrounds are recruited for participation in EABs; effectively managing EABs; and evolving EABs' purpose and membership in response to changes in the organizations they serve.
A limitation of the current paper was that, in addition to the extant research literature, it was based on the personal experiences of the authors and the three EAB examples were a descriptive rather than an empirical investigation of a broader sample. Furthermore, the EAB examples did not include an in-depth analysis of organizational data or a comprehensive review of internal reports. Future empirical study is needed to determine the impact of EABs on youthserving organizations' use of evaluation, programming implementation and quality, and strategic planning. Descriptive studies would be useful to examine the extent to which youthserving programs currently use EABs and how they are used. Future research is also needed to clarify the role of EABs in relation to evaluation capacity building and teaching evaluative thinking. Both are critical areas that EABs can serve to bolster, but research is needed for organizations to rigorously assess EAB effectiveness in meeting related needs.