Process Evaluations of Positive Youth Development Programmes for Disadvantaged Young People: A Systematic Review

Authors

DOI:

https://doi.org/10.5195/jyd.2022.1156

Keywords:

adherence, at-risk youth, community, complex settings

Abstract

This systematic review identified 10 process evaluation studies of positive youth development (PYD) programmes for disadvantaged young people, and aimed to assess the quality of reporting, methods used, and barriers and enablers to delivering programmes as intended. Four databases were searched: Web of Science, Psych INFO, Scopus, and Embase. Results indicated the methods used and quality of the process evaluations were highly varied. Numerous barriers (sessions feeling too much like school, lack of behaviour management skills, lack of funding, and logistical challenges) and enablers (collaboration with the local community, meeting young people’s needs, and communication) to delivering programmes as intended were identified. There is a clear need for improvement in design and reporting of process evaluations (e.g., more mixed method design of process evaluations, information on staff training, authors’ philosophical standpoint) in studies of PYD programmes for disadvantaged young people alongside a greater awareness of barriers and enablers to programme delivery. Doing so will enable programme outcomes to be appropriately attributed to what is actually delivered and generate more holistic understanding of the extent and reasons that programmes are delivered as intended. This will support more effective programme design, implementation, and sustainability of future PYD programmes for disadvantaged young people.

Author Biographies

Grace Tidmarsh, University of Birmingham

Grace Tidmarsh is a Doctoral Researcher in the School of Sport, Exercise and Rehabilitation Sciences at the University of Birmingham, UK. She is social scientist with interests relating to strengths-based sport psychology, process evaluation of interventions, and girls’ engagement in physical activity/sport.  

Janice L. Thompson, University of Birmingham

Janice L Thompson, PhD, FACSM, FECSS, is Professor of Public Health Nutrition and Exercise at the University of Birmingham. She is a mixed-methods researcher with over 30 years of experience examining factors that influence diet- and physical activity-related lifestyle choices. She uses this information to develop and evaluate community-based culturally tailored lifestyle interventions. She is a Fellow of both the American College of Sports Medicine (ACSM) and the European College of Sport Science (ECSS). In 2015, she was awarded the ACSM Citation Award in recognition of her significant contributions to research and scholarship in exercise science.

Mary L. Quinton, University of Birmingham

Dr Mary Quinton is a Lecturer at the School of Sport, Exercise and Rehabilitation Sciences. Dr Quinton's research interests include how positive youth development approaches in sport and community settings improve emotional regulation, social connectedness, and mental well-being in young people. Dr Quinton uses sport psychology and pedagogical theories to understand how young people’s backgrounds and support needs influence their development and motivation to learn and “perform” in sport, education, and homelessness settings. She is particularly interested in involving communities and those with lived experience in research and turning knowledge into action and impact.

Jennifer Cumming, University of Birmingham

Jennifer Cumming, PhD, CPsychol, AFBPsS, FHEA, is a Professor in Sport and Exercise Psychology at University of Birmingham. She is a community-engaged researcher who works collaboratively with third sector organisations and public services to co-develop and/or evaluate interventions to address important societal issues and translate knowledge into practice. Her programme of research investigates how individuals learn to effectively self-regulate and develop their mental skills for maximising potential and improving mental health, well-being, and social inclusion. 

References

Atfield, G., & Green, A. E. (2019). The impact of the youth obligation on disadvantaged young prople. Warwick Institute for Employment Research, Trust for London, and Centre Point. https://centrepoint.org.uk/media/4208/6-the-impact-of-the-youth-obligation-warwick-university-report.pdf

Bahari, S. F. (2010). Qualitative versus quantitative research strategies: contrasting epistemological and ontological assumptions. Sains Humanika, 52(1).

Barnes, M., Green, R., & Ross, A. (2011). Understanding vulnerable young people: Analysis from the longitudinal study of young people in England. https://www.gov.uk/government/publications/understanding-vulnerable-young-people-analysis-from-the-longitudinal-study-of-young-people-in-england

Bonell, C., Dickson, K., Hinds, K., Melendez-Torres, G., Stansfield, C., Fletcher, A., Thomas, J., Lester, K., Oliver, E., & Murphy, S. (2016). The effects of positive youth development interventions on substance use, violence and inequalities: systematic review of theories of change, processes and outcomes. Public Health Research, 4(5), 1-218.

Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., Brozek, I., & Hughes, C. (2014). Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Medical Research Methodology, 14(1), 42. https://doi.org/10.1186/1471-2288-14-42

Born, T., B., & Aldridge, H. (2015). Poverty amoung young people in the UK: A report by NPI for StreetGames. https://npi.org.uk/publications/children-and-young-adults/739/

Borrelli, B. (2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry, 71(s1), S52-S63. https://doi.org/10.1111/j.1752-7325.2011.00233.x

Bowers, E. P., Johnson, S. K., Buckingham, M. H., Gasca, S., Warren, D. J., Lerner, J. V., & Lerner, R. M. (2014). Important non-parental adults and positive youth development across mid-to late-adolescence: The moderating effect of parenting profiles. Journal of Youth and Adolescence, 43(6), 897-918.

Brunton, G., Thomas, J., O’Mara-Eves, A., Jamal, F., Oliver, S., & Kavanagh, J. (2017). Narratives of community engagement: A systematic review-derived conceptual framework for public health interventions. BMC Public Health, 17(1), 944. https://doi.org/10.1186/s12889-017-4958-4

Chegini, Z., Arab-Zozani, M., & Janati, A. (2019). Patient and health professional perspectives about engaging patients in addressing patient safety: A systematic review protocol. Open access Macedonian Journal of Medical Sciences, 7(9), 1561-1565. https://doi.org/10.3889/oamjms.2019.280

Collins, C., Lavender, M., Brown, S., Sheffield, P., & Andrew Aligne, C. (2013). Cyclopedia: Sustaining a positive youth development program through community partnership. International Journal on Disability and Human Development, 12(4), 457-465. https://doi.org/10.1515/ijdhd-2013-0208

Coser, L. R., Tozer, K., Van Borek, N., Tzemis, D., Taylor, D., Saewyc, E., & Buxton, J. A. (2014). Finding a voice: Participatory research with street-involved youth in the youth injection prevention project. Health Promotion Practice, 15(5), 732-738.

Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 337, a1655. https://doi.org/10.1136/bmj.a1655

Creswell, J. W., Shope, R., Clark, V. L. P., & Green, D. O. (2006). How Interpretive Qualitative Research Extends Mixed Methods Research. Research in the Schools, 13(1), 1-11.

Cronley, C., & Evans, R. (2017). Studies of resilience among youth experiencing homelessness: A systematic review. Journal of Human Behavior in the Social Environment, 27(4), 291-310.

Cyril, S., Smith, B. J., Possamai-Inesedy, A., & Renzaho, A. M. (2015). Exploring the role of community engagement in improving the health of disadvantaged populations: a systematic review. Global Health Action, 8(1), 29842.

Cuthbert, C. A., Farragher, J. F., Hemmelgarn, B. R., Ding, Q., McKinnon, G. P., & Cheung, W. Y. (2019). Self‐management interventions for cancer survivors: a systematic review and evaluation of intervention content and theories. Psycho‐Oncology, 28(11), 2119-2140.

Damon, W. (2004). What is positive youth development?. The Annals of the American Academy of Political and Social Science, 591(1), 13-24.

Danish, S., Forneris, T., Hodge, K., & Heke, I. (2004). Enhancing youth development through sport. World Leisure Journal, 46(3), 38-49.

Dowrick, A. S., Wootten, A. C., Murphy, D. G., & Costello, A. J. (2015). “We used a validated questionnaire”: What does this mean and is it an accurate statement in urologic research?. Urology, 85(6), 1304-1311.

Gadsby, B. (September 2019). The long-term NEET population. Impetus. https://www.impetus.org.uk/assets/publications/Youth-Jobs-Gap-The-Long-Term-NEET-Population.pdf

Gavin, L. E., Catalano, R. F., David-Ferdon, C., Gloppen, K. M., & Markham, C. M. (2010). A review of positive youth development programs that promote adolescent sexual and reproductive health. Journal of adolescent Health, 46(3), S75-S91.

Goldberg, H. E. (1979). A program for disadvantaged youth in an Israeli development town: An evaluation. Anthropology & Education Quarterly, 10(1), 21-42.

Gould, D., & Carson, S. (2008). Life skills development through sport: Current status and future directions. International review of sport and exercise psychology, 1(1), 58-78.

Gwadz, M., Freeman, R., Leonard, N. R., Kutnick, A., Silverman, E., Ritchie, A.,Bolas, J., Cleland, C. M., Tabac, L., Hirsch, M., & Powlovich, J. (2019). Understanding organizations serving runaway and homeless youth: A multi-setting, multi-perspective qualitative exploration. Child and Adolescent Social Work Journal, 36(2), 201-217.

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P. (2018). The mixed methods appraisal tool (MMAT) version 2018 for information professionals and researchers. Education for information, 34(4), 285-291.

Iachini, A. L., Beets, M. W., Ball, A., & Lohman, M. (2014). Process evaluation of “Girls on the Run”: Exploring implementation in a physical activity-based positive youth development program. Evaluation and program planning, 46, 1-9.

Johnston, K. A., & Lane, A. B. (2019). An authenticity matrix for community engagement. Public Relations Review, 45(4), 101811.IWGYP. (2016). Pathways for Youth: Strategic Plan for Federal Collaboration. https://youth.gov/pathways-youth-strategic-plan-federal-collaboration

Kazi, M. (2003). Realist evaluation for practice. The British Journal of Social Work, 33(6), 803-818. https://doi.org/10.1093/bjsw/33.6.803

Kenyon, D. B., McMahon, T. R., Simonson, A., Green-Maximo, C., Schwab, A., Huff, M., & Sieving, R. E. (2019). My journey: Development and practice-based evidence of a culturally attuned teen pregnancy prevention program for native youth. International Journal of Environmental Research and Public Health, 16(3), Article 470.

Kirkpatrick, D. L. (2006). Evaluating training programs : the four levels / Donald L. Kirkpatrick, James D. Kirkpatrick (3rd ed.). Berrett-Koehler, 2006.

Knoll, G. M., Pepler, D. J., & Josephson, W. L. (2012). The Toronto youth outreach worker program for transitional aged youth 12–24: Process evaluation. Canadian Journal of Community Mental Health, 31(2), 83-101.

Krabbenborg, M., Boersma, S., & Wolf, J. R. (2013). A strengths based method for homeless youth: Effectiveness and fidelity of Houvast. BMC Public Health, 13(1), 359. https://doi.org/10.1186/1471-2458-13-359

Kuosmanen, T., Fleming, T. M., Newell, J., & Barry, M. M. (2017). A pilot evaluation of the SPARX-R gaming intervention for preventing depression and improving wellbeing among adolescents in alternative education. Internet Interventions, 8, 40-47. https://doi.org/https://doi.org/10.1016/j.invent.2017.03.004

Larson, B. P., & Chung, K. C. (2012). A systematic review of peer review for scientific manuscripts. Hand, 7(1), 37-44.

Lerner, J. V., Phelps, E., Forman, Y., & Bowers, E. P. (2009). Positive youth development. In R. M. Lerner & L. Steinberg (Eds.), Handbook of adolescent psychology: Individual bases of adolescent development, (3rd ed., Vol. 1, pp. 524-558). John Wiley & Sons. https://doi.org/10.1002/9780470479193.adlpsy001016

Lerner, R. M., Lerner, J. V., Almerigi, J. B., Theokas, C., Phelps, E., Gestsdottir, S., Naudeau, S., Jelicic, H., Alberts, A., Ma, L., Smith, L. M., Bobek, D. L., Richman-Raphael, D., Simpson, I., Christiansen, E. D., & von Eye, A. (2005). Positive youth development, participation in community youth development programs, and community contributions of fifth-grade adolescents: Findings from the first wave of the 4-H study of positive youth development. The Journal of Early Adolescence, 25(1), 17-71. https://doi.org/10.1177/0272431604272461

Lerner, R. M., Lerner, J. V., Urban, J. B., & Zaff, J. (2016). Evaluating programs aimed at promoting positive youth development: A relational development systems-based view. Applied Developmental Science, 20(3), 175-187. https://doi.org/10.1080/10888691.2015.1082430

Li, J., & Julian, M. M. (2012). Developmental relationships as the active ingredient: a unifying working hypothesis of "what works" across intervention settings. American Journal of Orthopsychiatry, 82(2), 157-166. https://doi.org/10.1111/j.1939-0025.2012.01151.x

Liu, H., Muhunthan, J., Hayek, A., Hackett, M., Laba, T.-L., Peiris, D., & Jan, S. (2016). Examining the use of process evaluations of randomised controlled trials of complex interventions addressing chronic disease in primary health care—A systematic review protocol. Systematic Reviews, 5(1), 138. https://doi.org/10.1186/s13643-016-0314-5

Lockwood, C., Munn, Z., & Porritt, K. (2015). Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. International Journal of Evidence Based Healthcare, 13(3), 179-187. http://joannabriggs.org/research/critical-appraisal-tools.html

McConkey, R., & Abbott, L. (2011). Meeting the professional needs of learning support assistants for pupils with complex needs. Procedia - Social and Behavioral Sciences, 15, 1419-1424. https://doi.org/https://doi.org/10.1016/j.sbspro.2011.03.305

Mihalic, S. F., Fagan, A. A., & Argamaso, S. (2008). Implementing the lifeskills training drug prevention program: Factors related to implementation fidelity. Implementation Science, 3(1). https://doi.org/10.1186/1748-5908-3-5

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The, P. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

Moola, S., Munn, Z., Tufanaru, C., Aromataris, E., Sears, K., Sfetcu, R., Currie, M., Qureshi, R., Mattis, P., Lisy, K., & Mu, P.-F. (2020). Explanation of analytical cross sectional studies critical appraisal In E. Aromataris & Z. Munn (Eds.), JBI manual for evidence synthesis. JBI. https://synthesismanual.jbi.global

Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., Wight, D., & Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ : British Medical Journal, 350, h1258. https://doi.org/10.1136/bmj.h1258

Newman, T. J., Lower-Hoppe, L., Anderson-Butcher, D., & Paluta, L. M. (2020). Process evaluation examining the implementation of a sport-based positive youth development program. Journal of Youth Development, 15(6), 70-90.

Parry, B. J., Quinton, M. L., Holland, M. J. G., Thompson, J. L., & Cumming, J. (2021). Improving outcomes in young people experiencing homelessness with My Strengths Training for Life™ (MST4Life™): A qualitative realist evaluation. Children and Youth Services Review, 121, 105793. https://doi.org/https://doi.org/10.1016/j.childyouth.2020.105793

Pas, E. T., Cash, A. H., O'Brennan, L., Debnam, K. J., & Bradshaw, C. P. (2015). Profiles of classroom behavior in high schools: Associations with teacher behavior management strategies and classroom composition. Journal of School Psychology, 53(2), 137-148. https://doi.org/https://doi.org/10.1016/j.jsp.2014.12.005

Pierce, S. (2017). “Better people make better athletes”: Promoting a positive youth development approach with young athletes. In C. J. Knight, C. G. Harwood, & D. Gould (Eds.), Sport pyschology for young athletes (pp. 129-137). Routledge.

Quinton, M. L., Clarke, F. J., Parry, B. J., & Cumming, J. (2021). An evaluation of My Strengths Training for Life™ for improving resilience and well-being of young people experiencing homelessness. Journal of Community Psychology, 49(5), 1296-1314. https://doi.org/https://doi.org/10.1002/jcop.22517

Rangiwhetu, L., Pierse, N., Chisholm, E., & Howden-Chapman, P. (2020). Public Housing and Well-Being: Evaluation Frameworks to Influence Policy. Health Education & Behavior, 47(6), 825-835. https://doi.org/10.1177/1090198120917095

Rossman, G. B., & Wilson, B. L. (1985). Numbers and words: Combining quantitative and qualitative methods in a single large-scale evaluation study. Evaluation Review, 9(5), 627-643. https://doi.org/10.1177/0193841x8500900505

Scott, S. D., Rotter, T., Hartling, L., Chambers, T., & Bannar-Martin, K. H. (2014). A protocol for a systematic review of the use of process evaluations in knowledge translation research. Systematic reviews, 3(1), 149. https://doi.org/10.1186/2046-4053-3-149

Shek, D. T., Dou, D., Zhu, X., & Chai, W. (2019). Positive youth development: Current perspectives. Adolescent health, medicine and therapeutics, 10, 131-141. https://doi.org/10.2147/AHMT.S179946

Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., Petticrew, M., Rycroft-Malone, J., White, M., & Moore, L. (2021). A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ, 374, n2061. https://doi.org/10.1136/bmj.n2061

Tidmarsh, G., Thompson, J. L., & Cumming, J. (in press). Assessing the fidelity of delivery style of a mental skills training programme for young people experiencing homelessness. Evaluation and Program Planning.

Tingey, L., Larzelere-Hinton, F., Goklish, N., Ingalls, A., Craft, T., Sprengeler, F., McGuire, C., & Barlow, A. (2016). Entrepreneurship education: A strength-based approach to substance use and suicide prevention for American Indian adolescents. American Indian and Alaska Native mental health research, 23(3), 248-270.

Tufanaru, C., Munn, Z., Aromataris, E., Campbell, J., & Hopp, L. (2020). Chapter 3: Systematic reviews of effectiveness. In E. Aromataris & Z. Munn (Eds.), JBI manual for evidence synthesis. JBI. Available from https://synthesismanual.jbi.global

Tuli, F. (2010). The basis of distinction between qualitative and quantitative research in social science: Reflection on ontological, epistemological and methodological perspectives. Ethiopian Journal of Education and Sciences, 6(1).

Weiss, L. A., & Westerhof, G. J. (2020). The happiness route: Finding alternatives to the problem-based approach in social work for vulnerable groups. The Journal of Positive Psychology, 15(5), 666-669. https://doi.org/10.1080/17439760.2020.1789713

Wenz-Gross, M., & Upshur, C. (2012). Implementing a primary prevention social skills intervention in urban preschools: Factors associated with quality and fidelity. Early Education and Development, 23(4), 427-450. https://doi.org/10.1080/10409289.2011.589043

Wiltshire, G. (2018). A case for critical realism in the pursuit of interdisciplinarity and impact. Qualitative Research in Sport, Exercise and Health, 10(5), 525-542. https://doi.org/10.1080/2159676X.2018.1467482

Zimmerman, M. A., Stewart, S. E., Morrel-Samuels, S., Franzen, S., & Reischl, T. M. (2011). Youth empowerment solutions for peaceful communities: Combining theory and practice in a community-level violence prevention curriculum. Health Promotion Practice, 12(3), 425-439. https://doi.org/10.1177/1524839909357316

Downloads

Published

2022-06-15

Issue

Vol. 17 No. 2 (2022)

Section

Feature Articles

License

Authors who publish with this journal agree to the following terms:

  1. The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
  2. Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
  3. The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution 4.0 International License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
    1. Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
    with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
  4. The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
  5. Authors are permitted and encouraged to post online a prepublication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work. Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
  6. Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
  7. The Author represents and warrants that:
    1. the Work is the Author’s original work;
    2. the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
    3. the Work is not pending review or under consideration by another publisher;
    4. the Work has not previously been published;
    5. the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
    6. the Work contains no libel, invasion of privacy, or other unlawful matter.
  8. The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.

Revised 7/16/2018. Revision Description: Removed outdated link.