Youth Perspectives on Virtual After-School Programming During the COVID-19 Pandemic

After-school programs provide a range of support for students. During school closures due to the COVID-19 pandemic, many after-school programs were also forced to close or to find new ways to provide services to youth, such as through virtual after-school programming. We surveyed 244 youth who participated in virtual after-school programs about their access to virtual programming as well as their experiences. We considered their pre-closure experiences as well. We also surveyed 8 program directors of after-school programs who were providing virtual programming. We found that Internet access hindered the ability of more than 1 in 4 students to access the programs. Pre-closure program experiences, including ongoing relationships with program staff and positive peer relationships contributed to more positive experiences with virtual programming. Whenever students were able to access the programs, they generally reported positive experiences. This work has implications for after-school program providers, parents, and policymakers.

The Education Trust-West (2020) reported that 50% of low-income and 42% of families of color in California reported a lack of technology to access distant learning.
Resilience in the face of difficult circumstances, like the Covid-19 pandemic, can protect youth from negative outcomes. Authentic Connection (2020) presented a model of resilience that contains three domains: relationships at home, relationships at school, and attributes of each youth. Key relationships in the school domain include relationships with teachers and adults, as well as with peers. Warm and supportive relationships with teachers protected high school youth against declines in mental health during the pandemic (Authentic Connection, 2020).
Also, Canadian adolescents reported lower amounts of loneliness during the pandemic when they used video chatting technology and text messaging to interact with friends (Ellis et al, 2020).
Relationships with adults and peers are also important resilience factors for youth enrolled in after-school programs. After-school programs can provide youth with a safe place to go during after-school hours, academic support, a place to interact with friends, and an opportunity to create supportive and caring relationships with adults (Afterschool Alliance, 2017;Catalano et al., 2004;Halpern, 2002;Jones & Deutsch, 2011). Settings that are engaging and foster positive peer and youth-staff relationships can promote resilience and are also linked with favorable youth outcomes, like increased academic grades (Fredricks et al., 2004;Hirsch et al., 2000;Huang et al., 2007;Leos-Urbel, 2015). Journal of Youth Development | http://jyd.pitt.edu/ | Vol. 16 Issue 5 DOI 10.5195/jyd.2021.1063 Unfortunately, like schools, many after-school programs closed to in-person programming during the pandemic. After-school programs that largely serve minority and economically disadvantaged youth, especially those that are federally funded through the 21 st Century Community Learning Centers grant (Afterschool Alliance, 2014;U.S. Department of Education, 2004), may play a role in helping support youth who have been disproportionately affected by the COVID-19 pandemic if they can continue to offer some form of programming. A number of programs have adapted in the face of school closures and have developed creative strategies in an effort to continue serving youth through virtual or socially distanced in-person programming.
For example, one organization (STEAM Outreach Center) stepped up after New Mexico closed schools and partnered with educators and other community stakeholders to provide educational support (Morales et al., 2020). Using innovation and flexibility, they were able to provide a mixture of in-person and virtual programming for K-12 students.
Given the novelty of the situation caused by the pandemic, there is a lack of research on what these virtual after-school experiences are like for youth. Adapting the design of an evaluation of 21 st Century Community Learning Centers (CCLC) in Georgia, we explored youths' experiences in programs that were able to transition to virtual programming. We surveyed program directors to see what types of programming they offered, and we surveyed youth about their access to and experiences in the virtual programs. We draw on findings from research on community resilience (Ungar, 2011), which has consistently found that the social capital in place prior to events that disrupt whole communities is a fundamental resource for resilience following the event. We were interested in how youths' in-person after-school experiences prior to the pandemic were related to their experience in virtual programming. We begin with a descriptive analysis of the sites that were able to offer virtual programming and provided data for this study. We then examine how youth perceptions of their pre-pandemic program experiences and the quality of their Internet access were associated with their virtual programming experiences.
In addition, we examine how two key resilience factors, peer relationships and staff-youth interaction in the virtual setting, were associated with access to and experiences with virtual programming. Specific research questions were as follows: 1. Are program attendance and experiences prior to school closures and Internet connectivity associated with online program participation, staff communication online, and online interactions with friends in the program? 2. Are program attendance and experiences prior to school closures and Internet connectivity associated with the quality of virtual program experiences?

Method Program Directors
When schools closed due to the COVID-19 pandemic, the Georgia Department of Education provided us names of programs receiving 21 st Century Community Learning Centers funding that had made adjustments in order to continue to provide programming to their youth. To capture program directors' perspectives on how the switch to virtual programming was going, we created a survey using Google forms and sent it to program directors via email, following IRB approval. The email instructed directors to complete the survey if they consented, and to close out their browser at any time if they wished to end participation. We asked directors to provide demographic information of the youth their sites served, the types of technology they were using to reach youth, what types of services/activities they were providing, and the barriers they were facing. We sent surveys to 15 program directors and received responses from eight of them.

Youth Participants
As part of a larger project to enhance evaluation of 21 st CCLC after-school programs in Georgia, surveys were administered to program participants to assess their after-school program experiences. The survey was completed by 244 youth. These participants came from nine 21 st CCLC grantees across the state of Georgia. See Table 1 for demographic and participation information.
Though we did not collect demographic information such as youth ethnicity or socioeconomic status (SES), programs that receive federal funding under the 21 st CCLC grant serve large proportions of low SES and ethnic minority youth. Funding priority is given to applicants serving students attending Title I schools where students from low-income families comprise 40 percent of the total enrollment (U.S. Department of Education, 2004).

Procedure
Program sites were given the option to administer paper and pencil versions of the survey to youth, or to have youth complete the survey via Qualtrics. Eight programs chose to complete the survey via Qualtrics, and one site administered paper and pencil surveys. All procedures were approved by the university IRB, including a waiver of youth assent and parent permission.
We also included statements about the voluntary nature of the survey, and the right to skip questions or end participation at any time in the instructions.

Measures
The survey was an expansion of a three-item survey that has been administered to youth in all Georgia's 21 st CCLC programs for the past several years. We added questions about program participation, and we also adapted scales from previous studies to capture the constructs of youth engagement in program activities, adult support from program staff, and social competence. We were limited in the number of items that could be included in the final survey;  (Akiva et al., 2013;Kuperminc et al., 2018) items were used to create a school performance scale to include as a covariate to help isolate the effects of the after-school experiences variables as being above-and-beyond in-school experiences. These items were adapted from the Self-Description Questionnaire (SDQ) in the Early Childhood Longitudinal Study, which reported an alpha of .83 (Pollack et al., 2005); for example, "I get good grades in school." This scale had an internal consistency of α = .75 in our sample. All survey items were on a 5-point scale ranging from 1 (strongly disagree) to 5 (strongly agree).
We modified the response options for participation after school closures because some programs did not meet synchronously or at the same regular times as they did when schools were still open (See Table 1). We used three survey items to explore access to virtual programming and four items to assess virtual programming experience (See Table 2). We examined the three access items individually and conducted an exploratory factor analysis of the four experience items, which indicated the presence of a single factor. Item scores were averaged to create a virtual programming experiences scale with adequate reliability (α = .83).

Plan of Analysis
We first provide a descriptive analysis of the programs as reported by the program directors.
This description also includes the program director's perceptions of barriers for student participation. We also provide descriptions of youth experiences with access to virtual programming. for the other outcomes. A robust maximum likelihood estimator was used, and missing data were fitted using full information maximum likelihood (FIML). All analyses controlled for site variation by including dummy-coded variables representing each site. In each model, at least some site-level effects were observed. We also controlled for gender, grade level, and school performance.

Description of Virtual Programming
Program directors for eight 21 st CCLC grantees completed surveys describing how their sites shifted to providing virtual programming following the school closures. Four of these were rural, one was suburban, and three were urban. Three sites served primarily elementary school students; one served all high school students, and the remaining four served a mix of elementary, middle, and high school students. Reflecting the emphasis of 21 st CCLC on serving low-income populations, program directors estimated that 90% of the youth they served qualified for free or reduced-priced lunches at school.
On average, program directors estimated that 60% of youth in their programs had Wi-Fi access and a device they could use to participate in programming. Most of the school districts where the sites were located were able to provide devices (e.g., Chromebooks) and arrange for Internet access by providing hotspots, making arrangements with local Internet providers, or developing lists of locations with Wi-Fi access. In many cases, however, there were not enough of these resources for all students that needed them. Most of the programs were able to continue providing academic and enrichment programming in a virtual format. Adaptations to the virtual format included increasing social-emotional learning activities, coming up with virtual field trips, having staff be available to students throughout the day, and posting activities and videos online that could be viewed at any time. Programs used a variety of formats to engage youth online, including Google Classroom, Zoom, and social media platforms (Facebook, Instagram, Snapchat). All programs provided a mix of live, interactive, and asynchronous activities.  with 27.4% of youth disagreeing (or responding with "neutral") that they had good Internet access. Whereas a strong majority (89%) of youth agreed that adults at the after-school program communicate with them, only about half (50.5%) responded that they were able to interact online with their friends at the program.
Youth participation varied across sites that were able to offer virtual programming (see Table   1). One in 4 participants (25.3%) reported that they did not participate in virtual programming, whereas about 1 in 3 (36.0%) reported that they participated "a lot of times." Youth who participated in virtual programming at least once, tended to report positive experiences, with 74% to 90% agreeing or strongly agreeing that they were able to concentrate, felt connected with adults, felt a sense of belonging, and were able to get help on schoolwork.

Access to Virtual Programming
Three regression models assessing youth access to virtual programming were examined: frequency of online participation, communication with staff during online programming, and interaction with peers during online programming. In each of these models, the dependent variable was regressed on youth reports of their average daily program attendance prior to school closure, their retrospective ratings of program experiences (social competence in the program, engagement in program activities, and support from program staff), and the quality of Internet access. We also controlled for program site, grade, gender, and school performance.
Results of regression models assessing access to virtual programming are presented in Table 3.

Virtual Programming Experiences
Two models were examined to assess virtual programming experiences. The first included all the predictors from previous models, while the second added the three variables assessing virtual programming access as additional predictors. Results of regression models assessing virtual programming experience are presented in Table 4. We've reported the standardized beta coefficient (ß), which represents the relative amount of change (in standard deviation units) in the outcome variable for every standard deviation change in the predictor variable (Tabachnick & Fidell, 2019). For example, in Table 4, we reported that grade was significantly associated with adult communication with a beta weight of -.12. This means that with every standard deviation increase in grade, there is a .12 standard deviation decrease in adult communication.
The first model regressed virtual program experiences on youth grade in school, gender, youth reports of their average daily program attendance prior to school closure, their retrospective ratings of program experiences (social competence in the program, engagement in program activities, support from program staff), and the quality of their Internet access. We also controlled for program site, grade, gender, and school performance. The second model added communication with program staff about the online program and interactions with friends from the program online as two more predictors in the model. The 57 youth who reported never participating in the program online were excluded from these analyses, for a subsample of n = 187 youth who reported some degree of online program participation.
Overall, greater online communication from program staff and online interaction with friends from the program appear to explain why youth with better Internet access rated virtual program experiences more positively. In both models, youth who retrospectively reported more engagement in program activities and support from program staff prior to school closure rated their virtual program experiences more positively. In the first model, youth with better Internet access also rated their virtual program experiences more positively. In Model 2, youth who agreed more strongly that staff communicated with them about online programming and that they interacted more with their friends from the program online rated virtual program experiences more positively. In this model, the effect of quality of Internet access was smaller, and it dropped from statistical significance.  Note. N = 187

Discussion
The results provide insight into how virtual programming is being implemented, received, and affecting youth. We were interested in how program attendance and pre-closure experiences were related to virtual program access (including internet access) and virtual program experiences. Students reported high attendance before the shift to virtual programming.
However, most of the students who reported high attendance before school closures did not access programming virtually, as only one third of virtual program participants reported attending "a lot of times." In spite of these differences in attendance before and after school closures, we found that pre-closure program attendance was not related to any outcomes. We did find, however, that program directors and youth both reported that internet access was related to youths' abilities to participate in virtual programming. When youth were able to participate, they tended to report positive experiences.
We also found that different facets of pre-closure program experiences were significantly associated with virtual programming experiences. Youth who reported having more support from program staff pre-pandemic agreed more strongly that staff communicated with them about the online program. Also, youth with higher program engagement and higher ratings of adult support pre-closures rated virtual programming experiences higher. These findings are supported by work done by Ungar (2011) showing that resilience fostered by community factors before a major event help to buffer negative effects in the midst of and after a negative event.
These findings also align with previous research about components that make after-school programs of good quality. Having high student engagement and having strong, supportive staff-youth relationships are related to positive after-school youth experiences (Kuperminc et al., 2019;Leos-Urbel, 2015).

Limitations
Several limitations merit consideration. First, we relied on retrospective accounts of pre-closure after-school experiences to examine their role in explaining youths' experience with virtual programming. Therefore, we did not have truly longitudinal data with which to examine these associations. Second, we used convenience sampling to gather a sample comprised only of programs that were able to do virtual programming by the time of our survey. The 15 grantees that we contacted comprised just over 10% of the grantees in the state. Of those who were able to provide virtual programming, only nine of 15 participated in our study. Finally, the low internal consistency of the social competence scale likely attenuates the estimate of any association of social competence with virtual programming experience.

Future Directions and Implications
Moving forward with this work, more needs to be done to continue addressing the technology and Internet deficits experienced by many youth and their families. Schools, companies, and other organizations have worked together to address this problem. For example, during the pandemic, some Internet service providers offered Internet access through hotspots to children and youth and pledged to continue reducing the Internet access gap (Katz, 2020). Collaboration between companies, organizations, and the community can help to provide youth with the tools they need to access services like virtual after-school programs. Also, to use virtual programming as a means to promote resilience, focused attention should be paid to increasing student engagement, promoting positive staff-youth relationships, and facilitating peer interactions. More research is needed on how to promote these relationships and facets of after-school programming most effectively in a virtual format. The first step in this process would be to use programs, like those in this study, that have been able to connect with students using virtual programming, as resources from which to build off of successes that they have achieved. Some organizations, like Afterschool Alliance, have provided examples of successful virtual programming, along with resources for parents and educators (Afterschool Alliance, n.d.). These resources support the varying influences of engagement such as peer interactions and addressing the digital divide. Also, in trying to promote these relationships, it is important to consider the needs of caregivers at home and in other settings (school, afterschool, etc.) as well because adults in caregiving roles are also experiencing significant stress from the pandemic (Luthar & Mendes, 2020).

Author Note
This work was funded by the Georgia Department of Education. We have no conflicts of interest to disclose. Since this research was conducted, two co-authors have changed affiliations. Chris Henrich is now at The University of Alabama at Birmingham (chenrich@uab.edu) and Scot Seitz is now with Emory University (scot.raymond.seitz@emory.edu). Correspondence about this work may be sent to Candace Evans at cevans56@gsu.edu.