Youth and Adult Perceptions of a New Technology in California 4-H: The Online Record Book

Youth development research suggests that adult volunteers have the potential to influence the how and when youth engage with technology as a legitimate form of program participation. The adults’ comfort levels with technology, coupled with the historical structure of the youth development organization itself, shapes which technologies are made available to, adopted by, or perceived as relevant or useful to youth. This study explores how adults and youth members of California 4-H perceived the 4-H Online Record Book (ORB), an electronic version of the traditional 4-H record book. Survey data from 236 self-identified users and non-users of ORB provided feedback about their use of and feelings towards ORB, with youth expressing more positive feelings towards ORB and adults feeling more negative about the system. Youth however, felt supported by adults to use ORB.


Introduction
Community-based youth development organizations rely on adult volunteers to plan and facilitate programming with young people (often with youth as partners).Adults have tremendous influence and impact on young people's participation and experiences that may shape the lives of youth.A sustained relationship with a caring adult, other than a family member, is often described as one of a few critical factors in promoting positive youth development.For example, Lerner, et al. (2011) discuss the "BIG 3" program elements, the first being positive and sustained adult-youth relations, described as "relations between a young person and an adult who is competent, caring, and continually available, for at least a year, such as a mentor, coach, or teacher" (p.51).The National Research Council and Institute of Medicine (2002) emphasize the role of supportive relationships where adults provide guidance to youth and call this a "critical feature of any developmental setting" (p.94).
Youth development programs often provide access to opportunities youth may not get to experience elsewhere; for example, access to technologies and learning experiences to learn and use technology.But even while technology, particularly in the form of computer, web, and mobile devices, have become ubiquitous, particularly among youth (Ito, et al., 2008;Rani, & Kumari, 2013), the mere use of technology does not equate with one's abilities to understand the nature of technological systems, or to adapt and apply the tools of technology to complex, real-world problems (National Research Council, 2002).Emerging research suggests tremendous potential for integrating technology with youth development approaches to improve youth technological literacy and positive technological development (Bers, 2012).However, there are varying levels of comfort with technology; youth explore more with technology than do adults (Herring, 2008;Kolondinsky, Cranwell, & Rowe, 2002).Further, adults often have more difficulty using technology and use various modes of technology less than younger users (Herring, 2008).In addition, older adults may have skeptical attitudes about the benefits of technology and difficulties learning new technologies (Smith, 2014).
At the surface, this appears to be an odd paradox: youth development programs should promote youth technological literacy, and adults have the potential to influence the how and when youth engage with technology as a legitimate form of program participation.Yet adults are often uncomfortable and tend to use technology less than youth.But there is more to this account: another compounding factor is the institutionalized culture of many community-based youth development programs; several have endured over a century; e.g., Girls, Inc. formed in 1864 (Girls Incorporated, 2014);4-H started in 1902(Wessel, & Wessel, 1982); and Girl Scouts began in 1912 (Christiansen, & Girl Scouts of the United States of America, 2011).The period in which an organization is founded is highly correlated with its present structure, described as structural inertia (Hannan, & Freeman, 1984).The persistent organizational resistance to changing features (Carroll, Hannan, & Pólos, 2002), where "traditionalizing forces, the vesting of interests, and the working out of ideologies may tend to preserve the structure" (Stinchcombe, 1965, p. 168).The reinforcing nature of structural inertia is often preserved by the adult volunteer organizational leaders and seen in what researchers have termed the "Founder's Syndrome."A Founder is someone who "provided decisive leadership at a critical point in an organization's history, but did not change along with the evolving needs of the organization" (Huff, & Pleskac, 2012).Founders, often longtime participants resistant to change (typically adults), may serve to hinder the progress of the organization, preferring to stick to dearly-held traditions, best described by the maxim: "we've always done it this way."One of the issues with Founders Syndrome in youth development organizations is that an "old timer" may oppose organizational transformation even when such transformation may make program content more relevant for youth.
In summary, while youth development programs may want to integrate technologies into their programming (and should be encouraged to do so), several challenges may serve to prevent its broad adoption: adults who may not be familiar or comfortable with technology and/or obstruction from adults, Founders, who want to preserve tradition.

4-H Youth Development Program and the 4-H Record Book
The 4-H Youth Development Program, one example of a long-standing youth-serving program, has many historical traditions and rituals with roots back to its beginning in 1902, particularly the 4-H Record Book (Reck, 1951;Wessel, & Wessel, 1982).Throughout the past hundred years, 4-H youth have recorded their participation, personal development, and 4-H income and expenses through the 4-H Record Book (Forero, et al., 2009).The Record Book is a program tool to help youth reflect on their learning experiences, set goals, learn record keeping skills, promote self-reflection skills, and promote positive youth development (California State 4-H Office, 2013;Diem, & Devitt, 2003).
In 2010, California 4-H began to transition the paper-based Record Book to an online system, which was launched in September 2011.The Online Record Book (ORB) was programmed to mimic the traditional data collection from the paper forms but added two components: a social media-like interface and the ability to earn "spark points" for completing evaluation outcome surveys.It was initially thought that co-opting the traditional paper-based Record Book, moving it to a web-based format, and adding features to it, would help alleviate issues of Founders' Syndrome; in other words, we were not changing the Record Book so much as offering an online access point for youth to complete the process.However, in the three program years since the unveiling of the system, administrators received anecdotal feedback, primarily of a negative nature (i.e.,"ORB has too many glitches"; "I do not like anything about ORB"; "How do you know if the youth is the one really doing the Record Book?").As less than 13% of eligible youth were using the system in the most recent program year, ORB administrators and program staff wondered how youth were being encouraged or supported by their adult mentors to use the new web-based system.
Given the organizational desire to strengthen the use of technology in youth development programs, yet recognizing adults are not always as comfortable with technology, and knowing Founder's Syndrome may be present in a longstanding organization, our study examined youth and adult perceptions and use of the Online Record Book.Understanding service and satisfaction (of both users and non-users) would help ORB administrators explore adult and youth experiences while also informing system (re)development (Melovic, Mitrovic, Markovic, Nesic, & Vajčnerová, 2015;Warnock, 1992).California 4-H members had the option of using ORB for three years, yet coupled with the cultural inertia of the Record Book, we expected varying degrees of buy-in from youth members and adult volunteers.Therefore, we posed the question: What were the perspectives of 4-H members and their adult volunteer educators of the Online 4-H Record Book? to first gauge user satisfaction of ORB, and second to determine if Founders Syndrome was limiting the adoption of the new system.

Methods
We utilized a mixed-methods design to assess user satisfaction.The survey instrument featured both closed questions (with quantitative Likert scales) and two open-ended responses to explore participant experiences with and perceptions of ORB.Mixed methods research is an approach that relies on data gathered using both quantitative and qualitative methods and the integration of findings (Cresswell, & Plano Clark, 2010).Mixed methods is one tactic for triangulating results and improving validity and credibility of findings and often leads to more insightful understandings of the phenomenon.In our study, both types of data were collected on the same instrument; a process Cresswell and Plano Clark (2010) refer to as the convergent parallel design.

Data Collection
We administered a 15-item survey in July 2014.The online survey was both embedded in ORB and sent as a separate email to the 4-H youth and adult volunteer list to ensure completion by those not using ORB.This study was approved by the University of California, Davis Institutional Review Board.

Measure
The survey was adapted from several measures of usability of web-based systems (e.g., Chin, Diehl, & Norman, 1988;Lund, 2001).The survey was divided into four sections: use of ORB, perceptions of ORB, likes and dislikes of ORB, and demographics.
Use of ORB was assessed by asking participants four questions about their use of ORB: did they use ORB (response options: yes/no), what years they used ORB (2011, 2012, 2013; participants could check all that applied), if they used ORB to complete a record book, to earn spark points for completing surveys or to social connect with other 4-H members (participants were asked to check all that applied), and an open-ended questions asking for other reasons they used ORB.
Perceptions of ORB were measured by two items asking about the usefulness and general perception of ORB ("the Online Record Book is a positive addition to the 4-H program").Youth participants were then asked four additional questions about the adult support they receive with ORB (e.g., "At least one of my 4-H project leaders has encouraged me to use ORB").All questions were on a five-point Likert-style response scale (1=strongly disagree to 5=strongly agree).
To learn about likes and dislikes of ORB, participants responded to two open-ended questions: "What do you like the most about ORB?" and "What do you dislike most about ORB?What is your biggest challenge with ORB?" Finally, we asked for participant's gender and status in the 4-H program (youth member, adult volunteer, parent, or staff), as well as where they typically access the internet.The latter question was to help us determine if those who do not use or support ORB were accessing the system in a different manner from those that do use or support the system (e.g., from a location with a less reliable internet connection, from a location other than home).

Sample
The final sample consisted of 236 participants; 196 youth and 40 adult volunteers and parents.Respondents were predominantly female.Table 1 provides a summary of the participants.Note: gender percentage based on those that responded to the gender question.

Data Analysis
Quantitative.Quantitative analyses were conducted using SPSS version 22. Frequencies were used to examine youth and adults uses of ORB.To understand perceptions of ORB, we ran descriptive statistics on the six perception questions (two for adults).In addition, for the two perception questions asked of both, youth and adults, independent samples t-test were conducted to compare youth and adult mean scores.Gender differences were explored as well: for the questions relating to the use of ORB, Chi-Square tests of proportions were run, and for questions about the perceptions of ORB, independent samples t-tests were conducted to compare means between males and females.All effects sizes were calculated using Cohen's d.
Qualitative.Participant responses to the two open-ended questions varied in length from one word to paragraphs of text.Qualitative analysis was grounded in the data (see Corbin & Strasuss, 2007) and conducted in two steps.First, a set of codes was developed by reviewing responses and labeling each instance of a delineable idea or concept with a code.Second, the resulting codes were then reapplied to all responses.Multiple codes were often generated from and/or applied to responses with more than one idea or concept.Both questions were analyzed separately and youth responses were analyzed separately from adult (volunteers and parents) responses.The codes were grouped into themes.

Quantitative Findings
Use of ORB.For youth, 89% of youth reported using the ORB and half (50%) used ORB in the most recent (2013-14) program year.Over half of respondents (58%) reported using ORB primarily to complete their Record Book and to complete the spark points surveys.Only 6% of youth used ORB primarily to take spark points surveys.Youth also reported using ORB to organize and track their activities.For adults, 52% reported using ORB and a third of adults used the ORB in all years since ORB launched (38%).Adults reported using ORB for Record Book completion (to support their youth members), as well as approve youth record books and track project meetings and activities.
Perspectives of ORB.In general, youth found ORB to be useful and a positive addition to the 4-H program.In looking at adults modeling the use of ORB for youth, youth felt that at least one of their project leaders and parents or guardians talked to them about using ORB and encouraged them to use ORB.Adults, however, did not find ORB to be useful or a positive addition to 4-H.There was a statistically significant difference between the youth and adult perspectives; youth were more positive in their views about ORB than were adults ("use" t(230)=6.82,p<.001, Cohen's d=1.10; "positive addition" t(230)=5.88,p<.001, Cohen's d=0.99).See Table 2. * Likert-type questions coded with 1=Strongly Disagree to 5= Strongly Agree.
Demographic differences.We asked participants where they typically accessed the internet; 93% of participants access the internet at home.There were no significant differences in use or perceptions of ORB by where participants accessed the internet (home, mobile device, public library, etc.).Not surprisingly, those participants who reported using ORB had significantly higher scores on both questions (results not shown but are available from the first author).
No gender differences were found in uses of ORB by gender, but were found for perceptions.Specifically, male youth tended to agree more strongly than female youth that they felt ORB was useful (t(148)=2.05,p<.05; Cohen's d =.31) and was a positive addition to the 4-H program compared to their female 4-H peers (t(186)=2.10,p<.05; Cohen's d =.33).
Additionally, males tended to agree more strongly than females that at least one parent or guardian talked with them about ORB (t(168)=3.26,p<.01; Cohen's d =.50) or encouraged the use of ORB (t(167)=2.38,p<.05; Cohen's d =.51).See Figure 2. No gender differences were found for adults.

Qualitative Findings
Responses to the two open-ended questions resulted in approximately 8,500 words total.After coding the responses, codes were grouped into themes.Themes are presented below with the number of times that topic (codes) arose in the responses and a percentage of code applications; note that not all participants provided responses and some responses were labeled with two or more codes.

What do you like the most about the Online Record?
Youth (7 themes; 166 code applications).The most common "like" was the online format of the record book (n=84, 51%) and the possibilities afforded by technology including keeping information organized and on track, no hand-writing required, the ease of entering data, and the professional look.Youth reported comments such as "I like how it is a place to keep all of your information for future reference" and "It shows me all the work that I have done though out the year in one place."The second major theme was a general positive attitude towards ORB (n=46, 28%) represented by comments including, "It is convenient", "I find it to be a much quicker and easier method then the paper record book", and "It is easily accessible."Other themes included: general negativity (e.g., "I like nothing about ORB"; n=13, 8%), social networking (n=11, 7%), outcome surveys (n=5, 3%), environmentally-friendly (n=4, 2%), and better introduction of Record Books to new members (n=3, 2%).One young person remarked, "You don't need to print as much stuff, saving more money on paper and printer ink" while another said "I like that you can see other members participating, and sometimes it gives you motivation to work hard on your own work." Adult (4 themes; 29 code applications).The online format was the major "like" (n=17, 59%) including the professional look, keeping information organized, and tracking accomplishments.
One adult said, "Having all of the information in one spot.I can check what my child has done and watch her type her information in when she is working on it."Other themes included general negativity (n=7, 24%), general positive feedback (=4, 14%), and the ability for digital signatures (n=1, 3%).A general positive comment was "Online record keeping can be completed anywhere with internet.I like that the ORB helps kids prepare for adult jobs where this type of recording keeping is important."

What do you dislike most about the Online Record Book?
Youth (16 themes; 186 code applications).The most frequent "dislike" were specific technical glitches (n=80 responses, 43%) which were categorized into 16 specific issues/codes (the top were general glitches, data entry returning to the wrong page, formatting issues when printing, problems uploading files, and problems deleting activities).Other major themes included navigation and user interface issues (n=22, 12%), difficulty and time to enter information (n=19, 10%), the Record Book forms themselves (not specific to ORB; n=18, 10%), nonspecific complains of ORB being complicated (n=15, 8%), general positivity (n=8, 4%), no mobile app available (n=5, 3%), not enough training (n=4, 2%), and prefer hand written forms (n=4, 2%).The remaining seven themes consisted of 6% of code applications including: county-specific issues, social networking, spark score surveys, user support, limited computer access, general negativity, and negativity from adults.Regarding navigation issues, one youth commented that "Sometimes it is hard to find reports or print reports unless you go to the action center a certain way.I don't always know what goes where and I'm not sure what areas I should put my information in" while another observed "There is a lot of clicking to get back to places and trying to find things can be a tad challenging to say the least sometimes."Themes with limited responses (other: n=11, 6%) included: county-specific record book guidelines are a problem, social networking, outcome spark surveys, user support a problem, limited computer access at home, and general negativity.
Adult (12 themes; 47 code applications).Specific technical issues were the major "dislike" (n=24, 51%), categorized into 11 specific issues (the top were general glitches and log-on difficulties), followed by difficult and takes too much time to enter information (n=7, 15%), and limited computer access at home (n=5, 11%), general negativity (n=2, 4%), confusing and complicated (n=2, 4%), user interface issues (n=2, 4%), lack of features (n=1, 2%), navigation issues (n=1, 2%), not enough training (n=1, 2%), prefer hand written forms (n=1, 2%), privacy concerns (n=1, 2%), and problems using ORB for record book judging (n=1, 2%).One adult replied "ORB has been frustrating from the start.Too many glitches.It is only teaching the members technology and that the computer does everything for them.Our members are losing their ability to write."Another expressed frustration, "It is pretty much impossible for an adult to use much less a child!"One particularly negative comment (which was actually coded under ORB "Like" in the theme general negativity) was "My family dislikes the ORB and don't use it.It may be useful for some youth who dislike record books, but it has negated some of the learning from record keeping, rather than offer human support and assistance.From everything we understand, most youth find ORB frustrating."

Discussion
The survey results were able to provide useful information to respond to our research question: What were the perspectives of 4-H members and their adult volunteer educators of the Online 4-H Record Book?On average, youth rated the usefulness of ORB higher than adults and youth believed ORB to be more of a positive addition to the 4-H program; there were a statistically significant difference between youth and adults on both questions.In general, youth tended to share both likes and dislikes in proportion, taking the good with the bad, and generally being positive about the possibilities afforded by an online system, and frustrated with specific technical issues that interfered with the system's functions.Youth responses may demonstrate individual experiences with ORB and a more general acceptance, familiarity, and comfort with online technologies.Adults, on the other hand, were much more critical and shared in much more detail their frustration.Deeper reading of adult responses seemed to indicate an interpretation of ORB generated through multiple people and not individual experience themselves.In other words, adults did not generally use ORB or complete a Record Book, but were asked to support youth and parents in this process.Many adults made comments that indicated experiences from multiple people; e.g., "I am tired of hearing about all the problems;" "I tried to log on so I could tell the parents in my club … and I was unable to get online to see it;" and "People say that they enter information, and then it gets lost, and they have to enter it again.I don't want my child to waste her time entering information, and have it get lost."This could be demonstrative of adults being less comfortable with technology or more resistant to structural changes that can come with technological advances.Also, since adults served in a support role, they are called upon when something wasn't working right; meaning that adults hear about and interact with ORB when it is not working, rather than when it is.
One of our objectives was to identify the potential Founders' Syndrome limiting the adoption of ORB since only 13% of eligible youth used the system in the most recent program year.However, the results demonstrated that youth who use the system reported being encouraged to use ORB by a parent and/or an adult volunteer project leader.While these results do not represent the youth who did not respond to the survey (and who are likely not using ORB), our conclusion is that the Founders' Syndrome is not a primary issue.An alternative hypothesis, informed by the general negative tone from many of the adult respondents, stems from adults being placed in a technical support role, a position in which many 4-H adult volunteers are unfamiliar, uncomfortable, and unprepared.There is a clear need for additional training and support from 4-H staff to support 4-H volunteers with ORB.More support, resources, and training need to be provided to help adult volunteers become comfortable in a new role of technology support, a role many adults do not feel comfortable enough to begin with.

Limitations
Given the exploratory nature of the study, the survey was developed by heavily adapting existing measures, so future research might delve deeper into the customer satisfaction literature and use a validated measure.Second, over 60% (n= 151) of responses came from youth and adults using ORB and few demographic data were collected from participants.Therefore, this sample is not necessarily representative of the California 4-H population or of the youth and adults eligible to use the system.Finally, the initial purpose of the study was to understand youth and adults' use and perceptions of ORB.Only during data analysis did we discover the interesting youth versus adult effects so future research should be conducted to further explore these relationships.

Implications for Evaluation & Programs
Youth development programs provide youth with new opportunities, access, and support to become technologically literate.Because adult role models and mentors have such an influence on youth, we suggest that when implementing a new web-based technology which calls upon adult volunteers to support system adoption, two challenges may need to be mitigated: Founders' Syndrome and a general unfamiliarity (or lack of self-efficacy) of adults to support youth and parents in using the new system.
The implication is that the organization must be prepared to provide training and support to better prepare adult volunteers to help youth and their parents use the system in ways that validate their concerns.Another suggestion is to identify and empower tech-savvy youth to serve as "peer experts" and provide technical assistance to fellow youth.Recruiting youth peer mentors serves multiple goals of youth leadership development aligned with increasing technological literacy.This model of peer mentorship has well-documented outcomes in the context of youth development programming (e.g., Murdock, Lee, & Paterson, 2003).
Regarding the relationship between program development and evaluation, it is important to remember that the anecdotal feedback may not be representative of program members more broadly.Strong voices in the crowd, perhaps from Founders', may jeopardize the relevancy of the program by underutilizing potentially helpful changes.When instituting a new program or transition, it is important to anticipate and manage the expectations that both adult and youth members will present when new practices or tools are introduced.

Table 1
Online Record Book satisfaction survey demographics

Table 2
Youth and Adult Perspectives of ORB