Book Review—Measure, Use, Improve! Data Use in Out-of-School Time

Measure, Use, Improve! Data Use in Out-of-School Time offers out-of-school time (OST) professionals practical lessons and approaches to measurement and data use. Editors Christina A. Russell and Corey Newhouse assembled notable contributors who offer conversant perspectives on evaluation systems within diverse OST organizations. Its 4 thematic sections address issues and provide action-oriented solutions from OST programs, evaluation intermediaries, and funders. Key discussions across chapters include (a) moving forward using data to inform continuous quality improvement, (b) supporting youth development professionals, (c) ensuring equity and inclusion in the evaluation process, and (d) recognizing what it takes to move forward.

The evaluation design involves defining youth and stakeholders' program outcomes, outlining specific achievable objectives, and considering barriers to achieving outcomes. Continuous quality improvement (CQI) regularly addresses practices and how they make, or do not make, the program better, with a focus on improving the lives of youth. Evaluation takes time, and the authors suggest that the plan be multi-year and inclusive of staff, administrators, and stakeholders to ease the evaluation process, making it routine, and creating an environment of evaluative curiosity.
Reconciling organizational needs with funders' expectations may initially appear challenging.
However, funders and administrators are shifting their visions to support embedded evaluation for CQI. Trust between OST administrators and funders is key to advancing discussions on funding alignment, assessing internal capacity for multi-faceted evaluation planning, and being up-front about evaluation priorities. Part II: Building Blocks for Evaluation covers making data and tools meaningful to OST programs, selecting the right data system, being inclusive of cultural contexts and perspectives, allowing local ownership of data, and effectively employing youth participatory evaluation. YDP or practitioners (staff, managers, administrators) must be intentionally prepared to deliver highquality, equitable, and culturally relevant programs. Strategies and tools to make data meaningful are presented starting with a discussion on using a program logic model and theory of change. Coupling a well-crafted logic model with a theory of change may clarify the evaluation path. A look at data systems for small to middle-sized programs addresses the current system's ability versus what is needed and how uneven "tech-spertise" among staff plays into ongoing evaluation. Solutions and practical tips are offered such as stay mission focused, skip procrastination and shame zones, and allocate resources and staff to be effective. Table 2 outlines key content in each chapter.  Providence After School Alliance in program self-assessment (Yoo et al., 2019). Every Hour Counts offers 10 recommendations for effective data selection, collection, and management.
Youth Participatory Evaluation (YPE) includes young people in evaluation practices of planning, data collection, analysis, and community reporting (Chapter 9). YPE helps young people build thinking, leadership, and communication skills while expanding their social capital. The process of using YPE to improve programs and direct positive community change are discussed including organizational buy-in, planning for YPE, recruiting and involving youth, and preparing staff to engage youth as colleagues.
Part III: Developing Systems of Evaluative Thinking examines the use of CQI by OST intermediaries to build YDP's evaluative thinking skills. Evaluative thinking and strategies are described, along with case studies from several intermediaries, followed by recommendations to implement data-informed CQI. Notably, intentionally gathering youth feedback is an effective aspect of CQI. Table 3 outlines key content in each chapter.  Part IV: Using Data and Evaluation to Improve Staff Capacity addresses the professional development of and support needed for the YDP. One staff retention study strongly indicates YDP believe in their organizations and the positive impacts they have on the lives of youth (Angbah, 2018). Building staff capacity requires organizations regardless of size, to have an onboarding and retention plan embedded into the CQI scheme. CQI is greatly enhanced by YDP competencies that inform the hiring process, develop a scope and sequence for professional learning focused on skill building, and identify career trajectory for staff. Table 4 outlines key content in each chapter. Measure, Use, Improve! Data Use in Out-of-School Time brims with lessons, practices, processes, and recommended resources. It is an ideal tool for professional development training for after-school programs and youth-serving organizations. OST intermediaries and networks could employ Measure, Use, Improve! to guide a system-wide evaluation process. OST administrators could create study teams focusing on relevant chapters. Even well-established youth development organizations could use selected chapters to benchmark their CQI plans.
The authors have provided a rich collection of references for further reading.