Youth Worker Reasoning about Dilemmas Encountered in Practice: Expert-Novice Differences

This study examines one aspect of youth worker expertise, the ability to appraise and respond to the dilemmas of practice encountered in youth programs. To understand how expert youth workers differ from novices in their reasoning about these dilemmas, a sample of expert and novice practitioners read vignettes of dilemma situations and described their appraisals of the situations and how they might respond to them. Quantitative analyses confirmed four hypothesized differences between the two groups. Experts identified more considerations in the situations and generated more possible responses. Experts’ responses were also more likely to be youthcentered and address multiple considerations. Qualitative analyses suggested that these differences were attributable to the experts having more elaborate mental models that allowed them to understand varied human systems – individual youth, group dynamics, parents, funders, etc. – and to tailor their responses accordingly. The experts engaged in more forecasting of events, anticipating contingencies, and formulating decision trees that took possible contingencies into account. Although preliminary, the findings have implications for how youth workers are trained.


Introduction
The field of youth development is giving increased attention to the question of how best to equip practitioners who staff youth programs. Recent efforts have sought to clarify the essential knowledge and core competencies that are key to youth work practice (Astroth, Garza & Taylor, 2004;Stuart & Carty, 2006). Research in other fields of practice -including education, health professions, and engineering -suggests that the expertise of practitioners is defined, in part, by their skills in appraising and responding to the diverse situations, problems, and dilemmas they face in their work Fook, Ryan & Hawkins, 2000;Weiss, Kreider, Lopez, & Chatman, 2005). Similarly, an important element of youth workers' expertise may reside in their abilities to appraise and formulate responses to the array of dilemmas of practice they encounter in their daily work with youth in programs (Larson & Walker, 2010;Walker & Larson, 2006). This mixed methods study investigates how expert youth workers differ from novices in their reasoning about the dilemmas encountered in youth programs. Following methods employed to study expertise in other fields of practice , we focus on dilemma situations faced by front-line practitioners who supervise youth programs. A sample of peer-nominated expert and novice youth practitioners were asked to read vignettes of representative dilemma situations, then to describe their appraisals of the situations and how they might respond to them. We employed quantitative and qualitative analyses to identify differences between the reasoning provided by the two groups. The objective was to test four preliminary hypotheses and to use qualitative comparison to develop further theory and hypotheses about how experts differ in how they approach dilemmas of practice.

Dilemmas of Practice
We define dilemmas of practice as significant challenging situations and problems occurring in daily practice that call for decision-making by the practitioner, including whether and how to respond to the situation in ways consistent with professional obligations and goals. Across fields it is recognized that the dilemma situations encountered in practice are often complex, unstructured, and require pragmatic rather than formal reasoning. Different dilemmas present idiosyncratic considerations, as a function of the context, specific circumstances, the unique people involved, and the differing dynamics at work. In some situations, multiple competing professional obligations and goals may be in play; and these may shift over time as the situation evolves (Ross, Shafer, & Klein, 2006;Schön, 1983;Sternberg et al., 2000) The same appears to be true for the dilemmas encountered in youth programs. Youth workers encounter challenging situations that are unstructured and can involve competing relational, institutional, pragmatic, ethical, and other considerations (Banks, 1999;Camino, 2005;Halpern, 2009). These include, for example, situations that involve grappling with relating to youth as a friend versus an authority figure (Walker & Larson, 2006), struggling to keep youth's work in the program on track while keeping youth invested (Larson, Walker & Pearce, 2005), and feeling torn by funding requirements that conflict with youth's developmental needs (Jeffs & Banks, 1999). Youth work is embedded in complex overlapping ecological contexts, and youth workers are expected to be responsive to diverse people, points of view, and warrants across these contexts (White, 2007).
In order to investigate the range and nature of the dilemma situations faced by youth program leaders, Larson and Walker (2010) created a database containing 250 dilemmas reported by skilled leaders over the course of their daily work. These were obtained from ongoing interviews with 19 leaders in 12 urban and rural arts and leadership programs for high-school-aged youth. To obtain a framework for thinking about the diversity of these dilemma situations, Larson and Walker (2010) conducted a grounded theory analysis to categorize the dilemmas in their database. This analysis suggested that the dilemmas could be classified according to five different ecological systems in which the problem of the dilemma was situated. These systems included program activities, program norms, youth's personalities and relationships, the organization in which the program resides, and systems external to the program including community systems and youth's families. It should be noted that some dilemmas involved multiple systems, for example, a leader might have struggled with how the situation affected the progress of program activities, as well as program norms or the reactions of youth's families. These five categories are preliminary and incomplete, nonetheless they provide a provisional framework for identifying the diverse types of dilemmas that youth workers need to be able to think about in daily practice.
Practitioners' Expertise in Addressing Dilemmas of Practice What differentiates how "expert" youth practitioners appraise and respond to these dilemmas? We are aware of no prior research that systematically compares expert and novice program staffs' reasoning in these situations. Three decades of research on expertise across a wide range of disciplines, however, provides a foundation of knowledge about how experts think about and address situations in different fields ). This research consistently shows that expertise involves not just "knowing that" but "knowing how": it involves knowledge and skills both for understanding and for action (Sternberg et al., 2000). This research also indicates that there is typically no one "right" response to many dilemmas. Highly regarded experts in a field may have different responses to the same situation, as a result of different interpretations, values, and other factors (Dörner & Schölkopf, 1991).
This wider research on expertise, coupled with the literature on youth practice, suggested four initial hypotheses about differences in how novice and expert youth workers might reason about these dilemmas. The first was that experts would identify more considerations in dilemma situations. The wider research consistently finds that experts in a field have more situational awareness; they see more nuances in situations faced in practice (Endsley, 2006;Ross et al., 2006), partly because they look at situations from multiple perspectives (Fook, Ryan & Hawkins, 2000;Levin, 1995).
Second, the literature suggested that expert youth workers would generate more possible responses. Research across fields finds that experts have a wider repertoire of tactics (Ross et al., 2006). They are also more able to create novel responses adapted to the complexities of situations (Fook et al., 2000). In contrast, novices typically apply a smaller number of rote, context-free rules (Fook et al., 2000;Levin, 1995).
Third, we hypothesized that expert youth workers would be more likely to generate youthcentered responses. The general literature on expertise observes that in many professions decisions are shaped by values that define the mission of that field (Fook, et al., 2000). Being youth-centered has that status as defining value of the youth work profession (McLaughlin, 2000;White, 2007). Furthermore, qualitative research suggests that effective youth workers "see potential not pathology," put youth's interests first, and support youth's efficacy (McLaughlin, et al., 1994 p. 96). Larson and Walker (2010) found that the skilled leaders in their research found ways to solve dilemmas in their daily practice that involved engaging with youth, turning situations into opportunities for development, incorporating youth into the solution, and advocating for youth.
Fourth, we hypothesized that experts would be more likely to formulate responses that balance multiple considerations. The research on expertise indicates that experts are more able to address, adjudicate, or "balance" diverse considerations (Sternberg, 1998). Likewise, descriptive studies of effective youth program leaders suggest that they balance competing considerations, such as supporting and challenging youth or balancing the needs of individuals with those of the group (Grossman, Campbell, & Raley, 2007;Larson & Walker, 2010).
In addition to testing these four basic hypotheses, our aims included generating further preliminary grounded theory about the nature of youth workers' expertise. Description is an important early stage in any area of study. We felt it important to describe and begin to conceptualize the qualitative differences in the two groups' patterns of thinking about the dilemmas.

This Research
The strategy of this study was to compare the reasoning of peer-nominated novice and expert youth workers. This has been a common approach in research on practitioner expertise . Following the procedures of other studies, we employed written vignettes of dilemmas chosen to be representative of those encountered in practice; and research participants were asked to answer questions about these dilemmas and how they would respond (Ericsson & Smith, 1991;Harrington, 1995;Wilks, 2004). Quantitative analyses were used to test the hypotheses that the experts would differ from novices on the four features identified above. Exploratory qualitative analyses were then performed to identify underlying patterns that contributed to the differences found by these tests. As in other early research on practitioner expertise, strong relationships between participants' expertise and their age, education, and experience in the field (variables that often contribute the development of expertise) limit our ability to differentiate effects for expertise from those for these variables.

Sample
The sample consisted of 81 youth workers from organizations that provided programming for middle and high school aged youth. These programs were primarily in urban areas and served ethnically diverse urban youth. The sample included front-line staff from the national youth organizations (e.g., Boys & Girls Clubs, YMCA), city youth organizations (Parks and Recreation, Community Education), and community youth programs (e.g., leadership, arts, mentoring). This included 43 paid staff identified as expert youth practitioners and 38 identified as novices. The sample was recruited by leaders in the youth development field from three intermediary youth organizations in Minneapolis-St. Paul, New York City and the San Francisco Bay Area that provide support, training and resources to youth-serving organizations and programs. In most instances these leaders first worked with administrators of youth organizations in their city to identify "expert" practitioners (defined as "the most skilled, savvy practitioners; those we turn to when faced with a dilemma"), and these experts then nominated counterpart "novices" from their organization (defined as those still developing their understanding of youth development theory and practice). Twenty-seven matched pairs (an expert and novice) were identified in this way. In addition, 16 experts and 11 novices were identified without a pairing (i.e., a paired person was not identified or did not participate).
Not surprisingly, the experts were substantially older, had worked with youth for much longer, and had completed a higher level of education than their novice counterparts (Table 1). It also should be noted that the novices were not neophytes; they had an average of 2.6 years of paid work with youth, as well as an average of 1.6 years of volunteer work with youth. Nonetheless, the large differences in age, experience, and education -as found in other studies of practitioner expertise -should be kept in mind, and we will return to it later. Approximately half of the sample was from Minneapolis-St. Paul, and a quarter each from New York City and the San Francisco Area. Average length of employment at current organization (years) 1.6 4.7 Average length of employment in paid work with youth (years) 2.6 9.3 Average length of volunteer work with youth (years) 1.6 6.8 Percent who completed 4-year college degree or more 44 77 Percent who completed some college or a 2-year college degree 32 23 Percent who completed high school degree or less 24 0

Procedures
To evaluate how expert and novice youth workers reason about dilemmas of practice, each participant in the research read and responded to two vignettes of dilemmas situations via an on-line questionnaire. These two vignettes were randomly selected for each participant from a set of five real practice situations from Larson and Walker's (2010) database. The five vignettes (Appendix A) were chosen to be representative of each of the five ecological categories of dilemmas in that data base.
For each vignette, participants were asked to type out responses to five questions regarding the considerations and possible actions they would think about in the situation: Participants response to these to the two vignettes took about 30 minutes. Most of the data used to evaluate our first hypothesis came from question 1 and most of the data used to evaluate the other three hypotheses came from question 3. But in some cases pertinent information was also found in responses to the other three questions.

Data Coding and Analyses
The unit of analysis for both the quantitative and qualitative analyses was a participant's response to a single vignette.
Quantitative tests of the hypotheses. To test the hypotheses, we first created numeric scores representing the four features on which differences were expected. Coding rubrics were created to operationalize each feature. To assess number of considerations and number of possible responses (Hypotheses 1 & 2) the coding rubrics specified criterion for counting distinct instances of each. Thus, for example, the instructions for counting number of responses indicated that when a participant identified a set of responses that were linked in some way (e.g., actions ordered in deliberate sequence or alternative responses that were contingent on circumstances), each response should be counted as separate.
For the other two hypothesized features, holistic 4-point scoring rubrics were employed. Coders assessed the extent to which a participants' set of responses to the vignette contained elements of each feature. For Hypothesis 3, youth-centered responses were defined as those that gave youth and their developmental needs and interests focal importance (McLaughlin, 2000). Coders were instructed to first look for four dimensions of youth-centered responses identified by Larson and Walker (2010): a) engaging directly with youth, b) turning the dilemma into an opportunity for youth's development, c) incorporating youth into the solution or response to the situation, and d) advocating on behalf of youth as well as teaching youth to advocate for themselves.
Coders then scored a person's responses to the vignette on a scale from 0, "No comments about the youth's needs, interests and involvement," to 3, "Reference to all dimensions of youth-centered response, or depth on at least two dimensions." The coding instructions included prototypes for each of the four points on this scale, developed from transcripts from a pilot study.
For Hypothesis 4, responses that balance multiple considerations were defined as those that involved taking into account, addressing, or balancing multiple perspectives and concerns. This could occur, for example, when responses entailed: challenging yet supporting youth, accommodating individuals' interests and needs with those of the group, and balancing the needs of youth with the constraints imposed by funders, administrators, or another community organization. As with hypothesis 3, coders rated responses on a scale from 0, "No comments about taking into account, addressing to balancing multiple concerns," to 3, "Reference to all dimensions of multi-pronged response, or depth on at least two dimensions. Prototypes were identified for each point on the scale.
After initial training and practices sessions, two raters independently coded all four of these features for all the data. The raters agreed at a rate ranging from 50-70% across scales. Raters then compared codings and resolved disagreement through consensus. The raters were blind to respondents' novice/expert designation.
To test whether scores for these features differed between novices and experts, we used multilevel regression models that took into account the nested nature of the data (Raudenbush & Bryk, 2002). Models were tested separately for each of the four features. In initial analyses we used only the sample for which we had novice-expert pairings (N=27) and attempted to estimate three-level models with within-respondent (level 1), between-respondent (level 2), and between pair (level 3) effects. However, these models yielded insignificant and unreliable estimates which could be attributable to the small sample size and use of three levels.
Therefore the final analyses employed the entire sample and two-level models with within-and between-respondent effects. These analyses including dummy variables for four of the five vignettes, which were tested in Model 1. In Model 2, a variable was then added for a betweenrespondent variable, coded as expert (=1) versus novice (=0), and the test of the hypothesis was whether the effect of this variable was significant. To test the first two hypotheses, we used Poisson models to evaluate the dependent variables because number of considerations and number of responses are measured as counts (Cameron & Trivedi, 1998). For ease of interpretation, we report exponentiated coefficients (incident rate ratios, IRR), which are interpreted similar to odds ratios wherein a one unit change in the independent variable brings about a change in the incidence rate (count of the dependent variable). To test the last two hypotheses (for youth-centered and balancing responses), we use ordered logistic (proportional odds) regression and present results as odds ratios. Results of these tests for Model 2 are shown in Table 2. In Model 3, variables for participant's age, education, and years of experience were added as controls, and those findings are reported in the text. All models were estimated using STATA 10.0 (see Rabe-Hesketh & Skrondal, 2008). Qualitative hypothesis generating analyses. Exploratory qualitative analyses were aimed at better understanding the type of thinking lying behind the significant differences that emerged from the quantitative tests. For each significant hypothesis, the pertinent qualitative data were evaluated in order to identify underlying structural and thematic patterns related to the difference found between experts and novices. The iterative methods of grounded theory and related approaches were employed (Coffey & Atkinson, 1996). The pertinent data were first carefully read and open coded to generate preliminary ideas about possible differences. When differences were suggested, conceptual and operational definitions were developed to code instances of them, and comparisons were made. Our development of these definitions and interpretation of findings were influenced, in some cases, by the literatures on expertise and youth practice (Auerbach & Silverstein, 2003). The aim of these analyses was in part to generate new hypotheses for future research. The qualitative analyses progressed from empirical analyses, reported in the body of the section for each hypothesis, to theoretical postulation, presented in the final paragraph for each hypothesis (Strauss & Corbin, 1998).

Hypothesis 1. Number of Considerations
The first quantitative analysis tested the prediction that experts would identify more considerations in the dilemma vignettes than the novices (Table 2). In the initial step without expertise in the equation (Model 1), we found no significant differences in number of considerations that respondents reported between the five vignettes (i.e., all vignettes yielded a comparable number of considerations). In the test of the hypothesis (Model 2) we found that addition of the variable expert (vs. novice) accounted for a significant increase in the explained between-person variance. The incident rate ratio of 1.24 indicates that being an expert was related to a respondent identifying 24% more considerations in the dilemmas. In Model 3, in which participants' ages, education, and years of experience were added as controls, the effect for expertise became non-significant, indicating that this effect could not be differentiated from the effect of these other variables.
The qualitative analyses suggested two characteristics of the experts' appraisals that contributed to this difference. First, the experts differed from novices in identifying considerations that covered a wider range of ecological domains. These domains included the sponsoring organization (rules, norms, mission, funding), the psychological dynamics of the youth (personalities, emotions, maturity), the young persons' families' (parental consent, attitude, beliefs), the community, and macro-system factors (culture, religion, social class).
Second, the experts identified more considerations that went beyond immediate concerns in the situation. The considerations the experts described more often dealt with the situation's possible root causes (why the youth, staff, or people in the community acted as they did) and its future impacts on the youth and the program. It should be noted, however, that both of these differences (as well as those for subsequent hypotheses) are differences of degree. Most novices identified more than one consideration and a few novices described the kinds of widerranging concerns reported by the experts.
These differences are illustrated by one expert's appraisals of Vignette #4, in which the organization's CEO mandated that a music CD the youth were going to make would be focused on the topic of "role models," a topic the youth did not like. The novices' considerations for this vignette often focused solely on how to implement the CEO's decision, while the experts considered the motivations of the youth and CEO, as well as how the CD would contribute to the organization's future funding and reputation. For example, one expert's considerations included: 1) how youth might feel "trophied" and "taken advantage of," 2) why the community provides youth few role models, 3) the difficulties of attracting and retaining low income urban youth, 4) how the program's funders think, and 5) how to cultivate the relationship between the youth and the Executive Director.
This list shows both the diversity of ecological domains that this expert considered, as well as his/her concern with root causes of the dilemma and its future impacts.
A second illustration is provided by appraisals of Vignette #2 in which a young man, Charles, was disruptive to the group and disrespectful to staff. Novices were more likely to focus on the immediate issue of disciplining Charles, while experts had more broad ranging concerns, including the causes and possible sequela of the situation. For instance, one expert's considerations included potential causes for Charles's behavior (an undiagnosed condition, problems like drugs or gambling, difficulties he might be facing meeting basic needs like food and safety), Charles's perception of how his behavior impacted the group, parents' possible reactions, and the safety of program staff and participants.
In sum, the expert youth practitioners did not just generate more considerations, they formulated the dilemmas in terms of a wider swath of ecological systems and past-to-future time. Research across fields suggests that novices are more likely to focus on superficial features of a problem, while experts are more able to see its "deep structure" (Sternberg et al., 2000). Our findings suggest that in the field of youth practice this means that experts are more likely to conceptualize a problem from the perspectives of more people and levels of analysis.
The experts' thinking also involved reasoned speculation about possible and potential causes and impacts. This is consistent with research suggesting that expertise includes use of reasoned imagination to develop plausible hypotheses (Byrne, 2007;Dörner & Schölkopf, 1991). We suggest that experts' greater understanding of the "deep structure" of pragmatic problem involves developing theories about the processes at work.

Hypothesis 2. Number of Responses
In the initial step of the analyses for number of responses (Model 1), we found no significant differences between vignettes. The test of the hypothesis (Model 2), however, showed that addition of the variable for expert (vs. novice) contributed significantly to the equation. Similar to the findings for number of considerations, the experts identified a higher number of responses to the vignettes. The incident rate ratio indicated that experts generated 42% more responses (Table 2). In Model 3 the effect for expertise became non-significant, indicating that this effect could not be differentiated from those for participants' ages, education, and years of experience.
Our qualitative analysis suggested that the experts generated more responses due to three factors. First, consistent with the literature on expertise, they provided a broader range of responses. They appeared to have a larger repertoire of varied strategies. Second, they gave more responses because they described more contingencies that influenced their actions (i.e., if-then conditions). They described alternative responses dependent on specifics of the situation (if this happens, then…) and on one's objective. For example, in response to Vignette #1 involving a summer camp, one expert said, "If the camp is for educational purposes, the activities should …. If it is purely recreational, the activities should…" Third, the experts generated more responses because they more often described courses of action with multiple steps. These sometimes included branching situational contingencies. The following example of a highly elaborated plan by an expert in response to Vignette #2 illustrates a plan with multiple steps that depend on multiple contingencies (italics added to identify contingencies): When Charles got fired up … I would call for additional support help (if available) and separate Charles from the other members of the group (either ask the group to go around the corner with staff or ask Charles to go around the corner from the group. It is important to remove the audience). Give Charles a chance to cool off, as long as this is safe for staff (John and others) -this may mean slight acceleration before de-escalation. Once Charles starts to de-escalate, remind him of the agreement he made, ask him what else might be making him so upset. If Charles identifies other factors, develop a plan with him to assist in addressing those other factors. If Charles does not identify other factors influencing him, John would need to stay with him until he continues to de-escalate (historically it sounds like he does this) and then wait until Charles is in the state of remorse or apology. This is when Charles will be able to really "hear" John. Then a plan can be made with Charles to follow through with and act on.
One can see that this expert is able to envision an elaborate decision-making tree of anticipated contingencies and responses.
In contrast, the novices had less detailed responses with fewer contingencies and steps. Their responses were also more likely to emphasize adherence to rote, context-free rules or general slogans or truths. For Vignette #2, with Charles, one novice's response was: Do not allow youth to 'act out' and not respond to that behavior. Immediately responding to defiant behavior, for me, is the most pressing issue… As long as everyone follows the rules we can stay focused and attempt to have fun.
In response to Vignette #3 about a youth who dominated group discussions with violent stories, a novice emphasized: "Always stay positive. Make her talk about how she can improve her behavior." Observe that there is little reference to situational contingency in these responses.
As we found for Hypothesis 1, these results suggest that expert youth practitioners have more elaborate mental models of how situations unfold. In our instructions, we encouraged participants in the study to list many alternative responses to the vignettes ("Please brainstorm and list all the possible responses that come to mind"). But, although these experts demonstrated having a broad repertoire, they did not spend time generating widely different choices. Rather -similar research on expertise in other fields (Ross et al., 2006) -they worked from one or a few possible courses of action and thought through what was likely to happen and how they would respond to the most likely contingencies.

Hypothesis 3. Youth Centered Responses
Consistent with the third hypothesis, the experts reported significantly more youth-centered responses than novices (Table 2). In the initial step (Model 1), we found that the rate of youthcentered responses differed significantly by vignette (Vignette #2, about Charles, yielded lower scores for youth-centered responses). With this effect controlled, addition of the variable for being an expert (vs. a novice) in Model 2 was associated with a significant increase in explained variance. Experts described more youth-centered responses. In Model 3 the effect for expertise became non-significant, indicating that the effect for expertise could not be differentiated from age, education, and years of experience.
The qualitative analyses found that the most salient difference was that experts more often described responses that converted the dilemmas into opportunities for youth development.
Both novices' and experts' responses included addressing the immediate problem, however the experts responses also included using the situation to help youth learn. This is illustrated by responses to Vignette #5, in which the immigrant parents of a youth, Lydia, wanted her to quit the program. In response to this situation, both novices and experts recognized a need to communicate the value of the program to parents. But the novices often wanted to talk directly to the parents themselves, while the experts more often proposed strategies that included Lydia. One typical response by a novice was, "I would try to inform the parents more about the program, so they would see the benefits." In contrast, the experts more often proposed supporting or providing guidance to Lydia so she could handle the situation. One expert's youth-centered response involved "working with Lydia to help her answer the question 'what is this doing for you?' so she can try to convince her parents herself." Another emphasized helping Lydia learn "self-advocacy skills in navigating two very different worlds-family/home culture vs. youth program culture." Both novices and experts often proposed a three-way meeting including Lydia, her parents, and themselves. But the experts were more likely to envision coaching Lydia in advance, so that she would be able to speak for herself at this meeting.
This same contrast was evident in the response to other vignettes. For Scenario #4 in which youth were not motivated by the Executive Director's topic for their CD, one novice's concerns seemed to focus on getting the CD done himself, while experts more often wanted to use the situation to teach youth about how administrators and funders think, and even about grant writing. In response to the daycamp vignette (#1), novices suggested adult-centered responses, for example, "[The leaders] should remain controlled and confident; the project will meet great success if they simply maintain order and give a light (but stern) disciplinary 'helping hand' in such an effort." In comparison, one expert's described it as "a teachable moment" and said: I would bring the kids together in a series of ice breaking activities that make our commonalities and ultimate purpose in running the camp apparent. The point is to show youth that our intention is to learn, and roadblocks are OK, as long as we come together as a team … teams struggle too.
Facilitating youth development is a core objective of the profession. These findings suggest that novices readily shift their focus to more basic goals such as keeping order. But expert youth practitioners looked for ways to achieve both. They found creative solutions to engage and support youth in learning how the youth can help address the dilemma themselves. Our final hypothesis deals with this capacity of experts to address two goals at once.

Hypothesis 4. Balancing Multiple Considerations
The quantitative analyses also confirmed the fourth hypothesis (Table 2). In Model 1, we found significant differences between vignettes in how much they yielded balancing responses.
(Vignette #4 about the CEO and the CD yielded significantly higher scores for balancing than all four other vignettes). When the variable for expert (vs. novice) was added to the equation in Model 2, it accounted for a significant increase in explained variance. Experts scored significantly higher on the balancing scale than novices. However, this effect became nonsignificant in Model 3, indicating that it could not be differentiated from age, education, and years of experience.
The follow-up qualitative analyses suggested that this finding is a corollary of the experts identifying a wider range of considerations in the dilemmas. Their set of responses to each vignette were more likely to address multiple concerns, including balancing concerns that were in conflict with other. The dilemma in Vignette #3 was that a youth, Jackie, often dominated group discussions by sharing her violent experiences with the youth. The novices more often focused their responses exclusively on Jackie's behavior, typically on how to reduce her storytelling. The experts more often identified the situation as one in which the needs of Jackie and the group were in tension and both needed to be accommodated. One said her/his goal was: "trying to find a balance in making sure Jackie gets the right attention and that the other girls get heard." Similarly, in the situation between Lydia and her parents, the novices were more likely to dismiss the parents as simply being uninformed, whereas the experts emphasized a need to understand and respect their perspective.
In the following response to the daycamp vignette, one expert explicitly identifies multiple balancing acts that the leaders should try to achieve with their responses: Youth need to be able to plan activities and make their own decisions. They also need guidance and support from an experienced adult. If they are not led in a specific direction or motivated by a leader, they will have difficulties accomplishing their goals. The youth want to have ownership in planning the camp, which is important, but they also need to learn about what goes into planning a successful camp experience. By learning about the needs of children, the youth will be better prepared make plans and follow through with them.
This leader seeks to balance youth making their own decision with adult guidance and youth ownership with helping youth learn, among other things.
In sum, the experts tended to develop responses that addressed considerations across a wider range of people and ecological systems. The planned courses of action integrated concern for parental, organizational, pragmatic, and developmental goals and imperatives.

Discussion
This study suggests distinct differences in how expert and novice youth workers appraise and respond to dilemmas of practice encountered in youth work. The peer-nominated "experts" in our sample generated more considerations and more possible responses to vignettes of challenging situations. They were also more likely to formulate responses that were youthcentered and that addressed multiple considerations. Limits of the study need to be kept in mind. In particular the small data set did not allow us to differentiate these differences from difference in age, education, and experience between the two groups. Nonetheless these findings suggest some features that may define effective reasoning and response to dilemmas of practice. These findings may also have implications for how youth workers are trained.
The four features that significantly differentiated the reasoning of experts suggest elements of effective youth work practice. The experts' identification of more considerations suggests that skilled youth workers have a broader understanding of the issues and concerns embedded in youth practice. Their generation of more responses suggests that they have a wider repertoire of strategies to consider and choose from in ways that may be better fitted to specific situations. Their greater tendency to convert challenging situations into youth-centered learning experiences reflects skills to achieve the fundamental aims of youth work. And their greater development of responses that balance multiple considerations reflects a versatility to manage complex situations.
A central theme suggested by these four differences -most evident in our qualitative findingswas that the experts were distinguished by what might be called ecological intelligence, or "ecological rationality" (Gigerenzer, 2008). Whereas the novices more often applied context-free rules, the experts' appraisals and responses to the different vignettes reflected contextual sensitivity to the multiple people and ecological system dynamics that were pertinent to their work. The findings suggested they had knowledge and skills for understanding the varied systems -individual youth, group dynamics, parents, program managers, funders, etc. -and tailoring their responses accordingly. As found in other fields of expertise, they were able to take a pluralistic perspective, tolerate contradictions, and address or balance competing demands and obligations (Fook, et al., 2000;Levin, 1995;White, 2007).
This ecological intelligence appeared to involve, not solely knowledge of these different systems, but possession of elaborated mental models of how different systems worked and the leverage points for influencing them. These were evident in the experts more frequent formation of hypotheses about root causes of youth's or other people's behavior, hypotheses that, in real life situations, might help them better know what questions to ask or information to seek. These mental models were also evidence in the experts, greater engagement in forecasting events, anticipating contingencies, and formulating decision trees that took these possible contingencies into account. Ross and associates (2006) found that experts in other fields used their models of how things worked to run mental simulations and refine their course of action. That appears to be what these expert youth workers were doing. Whereas novice responses appeared to have a more narrow and less dynamic view of the dilemma situations that seemed to lead to more rigid and less nuanced solutions, we believe these mental models accounted for the experts' ability to generate more flexible and contextually-sensitive responses.
This preliminary study begins to identify some of the ways that expert youth development practitioners excel. But the study is limited by the sample and methods employed. The small size and the large differences between the two samples in age, education, and experience limited our ability to differentiate the effects of expertise from these other variables --which likely contribute to the development of expertise. Research with better matched samples is needed. Alternatives to use of peer nomination as a measure of expertise should also be explored. In addition, it should be asked how appraisals and formulations of responses to vignettes as a task in a study correspond to how practitioners act in real life situations. Priest and Gass (1997) describe how a rationale decision-making process can sometimes breakdown due to stress, haste, misinformation, and role overload. Another goal for future research is to understand how to transform less skilled or experienced youth workers into more proficient ones.
These findings suggest that it may be important to train youth workers in skills for ecological thinking; that new youth workers should learn to attend to the complexity of the real world of daily practice and to develop capacities to balance diverse and competing considerations while keeping youth at the center. It has been suggested that discussion and analyses of dilemmas should be a component of youth work training (Banks & Nohr, 2003). In other fields of practice-education, business, medicine-challenging situations and cases are often used to help trainees develop abilities to think about the real-world complexities of practice (Banks & Nohr, 2003;Weiss et al., 2005). Research in these areas suggest that "deliberate practice" that gives trainees, or even experienced professionals, with the range of situations encountered in their field is important to their development . This research also suggests that the goal is not to teach novices that there is one right solution for every situation, but rather to help them develop abilities to see the underlying complexity of situations and generate alternative responses to address the array of considerations.