Early Childhood Research Quarterly, 14, No. 4, 515–536 (1999) ISSN: 0885-2006
© 1999 Elsevier Science Inc. All rights of reproduction in any form reserved.
Multivariate Relationships Among Developmental Age, Global Engagement, and Observed Child Engagement Rene´e E. L. de Kruif and R. A. McWilliam University of North Carolina at Chapel Hill
The purpose of this study was to explore patterns of relationships among developmental age, teacher ratings of global engagement, and observed engagement in the classroom setting. Sixty-two children (age 9.5 to 63.6 months) were observed during free play, structured activities, and mealtimes. All children were administered a developmental test to assess their developmental age. In addition, teachers completed a rating scale to assess children’s typical engagement profiles. Canonical correlation analysis revealed two uncorrelated patterns of relationships among the variables. Function I reflected the positive relationship between children’s developmental age and high levels of engagement, and the negative relationship with lower levels of engagement. Function II represented the bivariate relationship between high levels of engagement regardless of developmental age.
Early childhood researchers (e.g., Buysse & Bailey, 1993; Jones & Warren, 1991; McWilliam, Trivette, & Dunst, 1985; McWilliam & Bailey, 1992, 1995) have proposed that high quality of engagement with the environment is a potentially critical mediating factor in young children’s learning. Specifically, these researchers suggest that when engagement is systematically promoted in the classroom, children are more likely to participate in developmentally appropriate activities with peers, adults, and materials (McWilliam et al., 1985; see Whaley & Bennett, 1991). Higher levels of engagement have also been related to high achievement levels and aptitude scores (e.g., McGarity & Butts, 1984; Rosenshine, 1978). Engagement has typically been defined as “the amount of time children spend interacting appropriately with the environment at different levels of competence” (McWilliam & Bailey, 1992, p. 234). The potential importance of engagement has led to extensive research over more than a decade, resulting in the conceptualDirect all correspondence to: R. A. McWilliam, University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Center, CB#8180, Chapel Hill, NC 27599-8180; Phone: (919) 966-7485; E-mail: [email protected]
de Kruif and McWilliam
ization and measurement of child engagement in increasingly elaborate ways. The purpose of this study was to add to the literature of child engagement by exploring multivariate relationships among developmental age and components of engagement as measured by two different methods: an observational method (i.e., the E-Qual; McWilliam, 1995) and a rating scale of “global” engagement (i.e., the CEQ; McWilliam, 1991). In early studies, engagement was conceptualized as participation in planned activities (Twardosz, Cataldo, & Risley, 1974) or as participation in manipulative activities (McClannahan & Risley, 1975), and researchers used the percentage of children participating in activities as an outcome measure (e.g., Krantz & Risley, 1977; Montes & Risley, 1975). A number of studies found that factors contributing to high percentages of engaged children included the physical environment, the social environment, and the teaching method. More specifically, researchers reported that a modified open classroom arrangement (Dunst, McWilliam, & Holbert, 1986; Twardosz & Risley, 1982), providing a variety of developmentally appropriate materials accessible to children (Krantz & Risley, 1977; Montes & Risley, 1975), opportunities for children to make choices (see Ostrosky & Kaiser, 1991; Whaley & Bennett, 1991), smooth transitions between activities (Doke & Risley, 1972; Rogers-Warren, 1982), carefully sequencing activities (Krantz & Risley, 1977), and incidental teaching (Warren & Kaiser, 1986) promote children’s engagement. These studies were helpful in determining which environmental factors influenced engagement but focused less on how children spend their time and how time spent was related to learning. More recently, research on engagement has shifted its emphasis from assessing the quantity of engagement to also investigating the quality of children’s engagement behaviors. Contemporary conceptualizations of engagement no longer view engagement as a dichotomous construct (i.e., time engaged versus time nonengaged). Rather, researchers are now more interested in the focus of a child’s engagement (e.g., with adults, with peers, with materials) and the level of this engagement. McWilliam (1995) has defined engagement levels with behaviors, such as pretend play, persistence, attention, participation, undifferentiated behavior, and nonengagement. Mastery-motivation researchers have included similar behaviors in their studies of this construct (see Brockman, Morgan, & Harmon, 1988; Messer, Rachford, McCarthy, & Yarrow, 1987). They have not, however, linked mastery motivation to engagement, and this literature has not yet led to improvements in instructional practices (McWilliam & Bailey, 1992). Whereas mastery motivation is generally operationalized as the time the child spends in goal-directed behavior, engagement also includes less goal-directed behaviors. Mastery motivation can therefore be considered to be a particular form of engagement and may be an important key to what happens during engagement that leads to learning (McWilliam & Bailey, 1992). The inclusion of quality or mastery levels in the engagement construct has aided researchers in better understanding how child engagement can be influenced by a number of different factors internal and external to the child. Researchers have investigated how child engagement is affected by involvement in activities (McWilliam & Bailey, 1995), and how engagement may differ across settings
(Blasco, Bailey, & Burchinal, 1993; McWilliam & Bailey, 1991, 1995), types of activities (McCormick, Noonan, & Heck, 1998), social and nonsocial behavior (McWilliam & Bailey, 1991), and mastery levels (MacTurk, Hunter, McCarthy, Vietze, & McQuiston, 1985). Measures of engagement are typically obtained through observation; researchers observe live or videotaped classroom sessions, using a coding scheme consisting of specific behaviors to determine the focus and level of engagement (see Carta & Greenwood, 1985; Greenwood & Carta, 1987; McWilliam, 1995; Malone, Stoneman, & Langone, 1994). Although this approach provides very specific information about children’s behaviors, it may not be the only relevant index of a child’s engagement. Another approach is to conceptualize engagement as a child’s typical or “global” behavioral pattern when interacting with the environment. Parents, teachers, or other adults who have broad knowledge of the child, gathered over long periods in a variety of settings, could provide information about this dimension of engagement. The potential for adults’ perceptions of the child’s engagement to influence their interactions with the child emphasizes the importance of an instrument to assess a child’s global engagement as viewed by the adult. Morgan and his colleagues (see Morgan, Maslin-Cole, Harmon, BuschRossnagel, Jennings, Hauser-Cram, & Brockman, 1993) have taken such an approach to the assessment of mastery motivation. They developed the Dimensions of Mastery Questionnaire (DMQ; Morgan, Maslin, & Harmon, 1987), which asks parents and teachers to rate how typically their children engage in behaviors that reflect mastery motivation. Preliminary work on measuring typical or global engagement levels has involved the use of the Children’s Engagement Questionnaire (CEQ; McWilliam, 1991), an instrument based on the DMQ. In a validation study, parents and professionals completed the 32-item CEQ for 108 children under the age of 6 years (McWilliam, Snyder, & Lawson, 1993). Results showed high congruence between parent and professional ratings of children, indicating that adults who have an ongoing opportunity to observe children’s behavior can judge their engagement equally well. Factor analysis of the CEQ items resulted in four underlying factors: Competence, Persistence, Undifferentiated Behavior, and Attention. No investigations have been published examining multivariate relationships between engagement components measured through global methods (e.g., the CEQ; McWilliam, 1991) and engagement components measured through observational methods (e.g., E-Qual; McWilliam, 1995). In addition to the influence of the environment on engagement, as mentioned earlier, the child’s intra-individual characteristics (e.g., mastery behavior, temperament, disability) influence engagement also. A number of studies have examined the relationship between developmental age and engagement. Malone et al. (1994) investigated how categorical and sequential play levels were related to developmental age and chronological age in a home context versus a classroom context. Patterns of association in both conditions were the same: nonplay, functional play, and exploratory play behavior (which were considered to be less sophisticated behaviors) were negatively related to developmental age, whereas more sophisticated behaviors such as constructive play and pretend play were
de Kruif and McWilliam
positively related to developmental age. In addition, developmental age was related to more sophisticated play sequences. Blasco et al. (1993) also reported a relationship between developmental age and the sophistication of play behavior. Typically developing children and children with disabilities in mixed-age classrooms who were developmentally more advanced, spent more time in purposeful play and social play and less time simply manipulating materials than children who were less developmentally mature. Finally, McWilliam and Bailey (1995) found that developmental age was positively associated with more interactions with peers. As developmental age increased, children with disabilities spent more time attentionally engaged with peers than did children without disabilities. Developmental gain appeared to reduce the effects of disability with regard to nonengagement. In conclusion, these studies indicate the importance of developmental age as a predictor in children’s engagement. The purpose of this study was to investigate patterns of relationships between two sets of variables including developmental age and components of the engagement construct as measured by the CEQ (McWilliam, 1991), and an observational method (i.e., the E-Qual; McWilliam, 1995). Specifically, the question explored in this paper is which combinations of developmental age and teacher ratings of global engagement components are associated with similar combinations of observed engagement components.
METHOD This study was part of a broader study investigating teachers’ interaction styles and the influence of these different styles on children’s engagement. All teachers were videotaped, and their behaviors as well as the behaviors of the children in their presence were coded through a computerized coding system. Although the present investigation focused on children’s engagement behaviors, we acknowledge that these observed behaviors might have been influenced by the presence of the teacher. A brief description of the teachers, in addition to the information we provide on the children who participated in the study, is included. Participants Children. Sixty-two children (31 males, 31 females) enrolled at a university supported childcare center participated in the study. The children ranged in age from 9.5 to 63 months. Forty-nine of the children were typically developing, and 13 children were classified as having special needs. Disabilities or conditions represented by these children included developmental delay, cerebral palsy, Apert’s Syndrome, autism, Prader Willi syndrome, and Williams syndrome. Table 1 presents the demographic information and the number of children in each classroom. Fifty percent of the children were Caucasian, 44% were African American, and 5% were from some other ethnic background, as determined from parents’ self-identification on center registration forms. The average SES of the children’s families was calculated with the Hollingshead (1975) four-factor for-
4 (1) 3 (1) 3 (1) 4 (1) 0 49.6 18.5–63.0 19.2 14.4–26.4 24.6 (18.0) 22.0–29.0 (17.0–19.0)
5 5 4 6 0 45.8 27.0–58.5 9.5 6.8–11.8 18.5 7.0–21.0
Toddler A (n ⴝ 7)
3 (1) 4 (1) 0 37.1 19.0–66.0 19.2 13.2–21.6 26.4 (21.0) 24.0–31.0 (14.0–28.0)
4 (1) 3 (1)
Toddler B (n ⴝ 7)
Numbers in parentheses represent the number of children with special needs. Hollingshead (1975) Four-factor index. c Numbers in parentheses represent M and SD for children with special needs.
Gendera Male Female Ethnicitya African American White Other M SESb Range M Age in Years Range M Battelle (BDI-DA)c Range
Nursery (n ⴝ 10)
3 4 0 47.9 41.5–57.0 33.6 30.0–37.2 45.4 39.0–51.0
Toddler C (n ⴝ 7)
Table 1 Demographic Information for Children in Different Classrooms
7 (1) 7 (3) 2 (1) 44.6 14.0–66.0 50.4 32.4–63.6 57.9 (45.4) 21.0–70.0 (16.0–43.0)
8 (4) 8 (1)
Preschool A (n ⴝ 16)
8 (1) 6 (3) 1 41.6 27.0–53.0 48.0 33.6–63.6 58.3 (41.5) 52.0–65.0 (27.0–56.0)
9 (4) 6
Preschool B (n ⴝ 15)
Engagement Relationships 519
de Kruif and McWilliam
mula and ranged from 14 to 66 (the ceiling). Since this was a sample of convenience, we had no control over the heterogeneity of the children. Although this variability could confound our findings, we appreciated the diversity presented. Teachers. The adults participating in the study consisted of 10 female teachers and 1 male teacher working in six classrooms at a university-supported child care center. Four teachers were African American and seven were Caucasian. Two teachers had master’s degrees, one had an associate’s degree, and the rest had bachelor’s degrees. Teachers ranged in teaching experience from 2 to 22 years (M ⫽ 13.64, SD ⫽ 7.99), and their experience with children with special needs ranged from 1 to 12 years (M ⫽ 8.20, SD ⫽ 4.62). Each classroom had an assistant, and the number of children in each classroom ranged from 7 to 16 (M ⫽ 9.00). Instrumentation Battelle Developmental Inventory (BDI). The BDI (Newborg, Stock, Wnek, Guidubaldi, & Svinicki, 1984) was administered to determine the children’s developmental age. The BDI is a standardized instrument for assessing children’s development in five domains (personal-social, adaptive, communication, motor, and cognitive domain). Test procedures are designed to collect information through presentation of a structured test format; interviews with parents, caregivers, or teachers; and observations of the child in natural settings. Each child was assessed using all five domains in order to establish the children’s approximate developmental age. Children’s Engagement Questionnaire (CEQ). Teachers completed the CEQ (McWilliam, 1991) for each child. The CEQ is a 32-item instrument designed to rate children’s global engagement. It is completed by an adult familiar with the child (in this case, the teacher) and demands their free-recall impression of the child’s levels of engagement with peers, adults, and materials (i.e., the rating is independent of time or context). The CEQ has a four-point rating scale to record whether the child’s behavior is (1) not at all typical, (2) somewhat typical, (3) typical, or (4) very typical. The instructions specify that “typical” means that the child spends quite a lot of time in the activity. Behavioral examples are provided for each item on the CEQ to further clarify the intent of the item. Previous research on the CEQ indicated the existence of four underlying factors: Competence, Persistence, Undifferentiated Behavior, and Attention. These factors have been found to explain 62.1% of the variance, and parents’ and professionals’ ratings of children were found to be highly congruent (McWilliam et al., 1993). The generalizability coefficient was found to be .84, and the alpha coefficient for internal consistency was .96. Coefficient alpha in the present study was .93. Table 2 presents example items for each of the factors. E-Qual Observational Coding System (E-Qual). Observed engagement was measured using a version of the E-Qual (McWilliam, 1995). Two kinds of engagement variables were coded: level of engagement (i.e., attention, undiffer-
Table 2 Sample Items on the CEQ Factors CEQ Factors Competence
● Tries to complete things even if it takes a long time to finish.
● The child knows how to put together simple jigsaw puzzles, sticks with it until it is completed. ● The child who does most of the things at the 2-year-old level plays with objects and people at the 2year-old level. ● When another child approaches, the child will talk to or play with him or her. ● The child tries to get the teacher to give him or her a toy. ● The child bangs the toy car over and over again on the high-chair tray. ● The child says “Ba-ba-ba-ba-ba.” ● When the mother moves about the kitchen, talking to the child, the child watches her. ● When other children are playing, the child follows their movements with his eye-gaze.
● Plays appropriately for his or her age.
● Plays with other children who try to play with him or her. ● Tries to get adult to do things.
Undifferentiated ● Plays with objects in a simple Behavior manner (i.e., repetitive, changing). ● Uses repetitive vocalization. Attention ● Watches or listens to adults. ● Watches or listens to other children.
A copy of the full instrument may be obtained from the second author.
entiated behavior, nonengagement, participation, pretend, and persistence), and a modifier indicating the type or focus of engagement (i.e., with the teacher, another adult, peers, materials, and self). To reduce the number of variables only level of engagement was used for analyses in the present study. All engagement codes were mutually exclusive; raters could not assign two codes at the same time. Two additional codes were used to indicate that the child was no longer present and to indicate when the child was in transition between two locations without being otherwise engaged. Neither of these two codes was used in the analysis. Table 3 presents brief descriptions of the observed behaviors included in the analysis. Detailed operational definitions may be requested from the second author. Procedures Videotaping. Observational data were collected through videotaped sessions of teachers interacting with the children. Each teacher in the study was videotaped for at least eight 20-minute indoor sessions. The camera focused on the teacher and a 6- to 10-foot area in front of him or her to capture the children most likely to be affected by teacher behavior. To obtain a balanced sample of teacher behaviors across activities, two of the sessions were taped during mealtime, two
de Kruif and McWilliam
Table 3 Brief Definitions of Levels of Observed Child Engagement from the E-Qual Level Persistence Pretend Participation Undifferentiated Behavior Attention Nonengagement
Definition Goal directed problem solving or repeated attempt. Talking in character, substituting objects or acting out a scenario. Actively involved with the environment (i.e., busy), but not in pretend play, not persistently, and not repetitively. Interacting with the environment without differentiating his or her behavior (i.e., in a repetitive manner). Watching or listening to features in the environment for at least 3 seconds. Unoccupied, waiting, staring, wandering, crying or aggressive behavior. None of the other behaviors are occurring.
during structured class activities, and the remainder of the sessions during unstructured class time. In the nursery, structured sessions were replaced with sessions taped during unstructured class time. The sampling times were not intended to represent exact proportions of the day during which each activity occurred (i.e., free play occurring exactly twice as often as meals or structured sessions), because each classroom and age level differ in their appointment of time. Structured sessions were considered to be sessions in which the teacher (a) selected materials, (b) had expectations for sustained play, (c) encouraged a limited range of behaviors, and (d) spent at least 2 minutes directing the activity. Large and small group times are examples of structured sessions. Another example we observed was when a toddler teacher sat at the table with all children in the room. Each child had a peg-board and a bowl or plate with pegs. The children were not allowed to leave or take out other toys, but were directed back to the table and the pegboard in front of them. Unstructured sessions were sessions in which the teacher (a) allowed free choice of materials, (b) allowed the children the freedom to come to and leave the activity, and (c) encouraged a variety of behaviors. These sessions varied from activities in which children asked the teacher to read a book to them, to activities in which children who were interested came to the art table to make animal masks of their choice with materials of their choice. Our intent was to capture the normal flow of the activities, and as a result, some of the videotaped sessions included more than one type of activity. For example, we once captured teacher-child interactions during a session starting with the last minutes of free play and the beginning of large group time. When this occurred, the session type was determined by the activity that took up 50% or more of the total session time. Every child in the study was taped in at least four sessions and for a minimum of 10 and a maximum of 60 minutes across all sessions. Since the sessions were not systematically organized by child but by teacher, the presence of the children in the sessions was somewhat random. Every child, however, was filmed at least once during a structured activity, once during an unstructured activity, and once
Table 4 Inter-rater Agreement—Observed Engagement Codes (E-Qual) Code
Attention Participation Nonengagement Undifferentiated Behavior Persistence Pretend High Level Engagement
.94 .94 .97 .98 .99 .98 .99
.69 .80 .34 .40 .30 .32 .32
.76 .84 .43 .49 .37 .37 .38
during mealtime. Three teachers needed 9 sessions, and another teacher needed 10 sessions, to obtain adequate samples of child behavior (i.e., enough time). All 93 sessions were taped within 8 weeks. Videotaping occurred between 8:00 a.m. and 12:30 p.m. Teachers were notified in the morning that they would be taped that day but had the option to defer taping at their convenience. The teachers were told that observations would focus on all children’s engagement and that group, individual, or caregiving activities were acceptable. Activities in the classroom were taped as they naturally occurred. After the third session, however, teachers were informed of the type of activity that still needed to be captured, prior to the day the taping would take place. Coding. The Observer™, a computer software program for observational studies with a Video Tape Analysis System (Noldus Information Technology, 1993), was used to code both teacher and child behaviors. In this study, analyses were limited to the coding of the level of child engagement behaviors. Exactly 900 seconds from each 20-minute taped session were electronically time-coded. Four graduate students were trained for 6 – 8 weeks to master the technique of micro coding on The Observer™ and to establish a minimum 80% level of agreement on all categories of the E-Qual Observational Coding System (McWilliam, 1995). Coders rated continuous behavior, using one of six levels of engagement followed by the type of engagement. Coders were trained, using sessions similar to those used in the actual study, until they had reached acceptable levels of inter-rater agreement. The coding team met regularly to answer questions about the coding system. Twenty-five percent of the formal observations were double-coded to monitor inter-rater agreement and to prevent coder drift. Table 4 presents interrater agreement for each code across the double-coded sessions. Low expected agreement and low kappas occurred in the codes for behaviors observed very infrequently. This phenomenon is characteristic of expected agreement and kappa coefficients. Computation of Scores. Factor scores on the CEQ were calculated by taking the mean of the items constituting each factor. Observed child engagement behaviors were coded from the videotaped teacher sessions. Because the focus of
de Kruif and McWilliam
the taping was on the teacher and some classrooms had more than one teacher in the room, some children were coded as many as 21 times (children in classrooms with three teachers and who were present for the sessions of almost all of the teachers), and others only 4 or 5 times (children with just one teacher in the room). Specifically, children in classrooms with one teacher were taped an average of 4.79 times (range 4.00 –7.00), children in classrooms with two teachers were taped an average of 9.86 times (range 8.00 –11.00), and children in classrooms with three teachers were taped an average of 14.41 times (range 4.00 –21.00, with one child being taped only 4 times). No differences in sessions were found as a function of ability status or gender. Since more frequent codes only increases the reliability of the observed behavior, we used all sessions in which a child was coded to compute an overall mean score for each observed behavior per child. Scores on each of the engagement behaviors represent the percentage of time the child was engaged in this behavior across sessions. Data Analysis To reduce the number of variables, the observed pretend and persistence categories, both cognitive advanced skills occurring very infrequently (0.41% and 1.14% respectively), were combined to create a single variable: high level engagement. Correlations were used to examine bivariate relationships between variables in the analysis. Canonical correlation analysis was then used to explore patterns of multivariate relationships between developmental age, four CEQ global engagement variables, and five observed engagement variables. Canonical correlation analysis may not be familiar to some readers (for an introduction see Thompson, 1991, in press) but can be related to regression. The basic strategy in canonical correlation analysis is to derive a weighted sum of linear combinations (or canonical variates) of two sets of variables in such a way that the linear combinations are maximized (Thompson, 1984). The first pair of canonical variates (i.e., function) selected has the highest intercorrelation possible. The second pair of canonical variates then selected accounts for the maximum amount of the relation between the two sets of variables unaccounted for by the first canonical function, and so forth. This process is repeated until the number of canonical correlations established equals the number of variables in the smaller set. In this study, developmental age and the four CEQ factors (competence, persistence, undifferentiated behavior, and attention) were selected to form the predictor set in the canonical analysis. Given the age variability in the sample, developmental age was included in this first set of variables. Five observed engagement variables (attention, undifferentiated behavior, nonengagement, participation, and high engagement) served as the second or criterion set of variables.
RESULTS Means and standard deviations for all 10 variables are presented in Table 5. Inspection of the correlation matrix (see Table 6) revealed a number of moderate
Table 5 Means and Standard Deviations on the Battelle, CEQ and E-Qual Variables
Battelle Developmental Age Child Engagement Questionnaire Attention Competence Persistence Undifferentiated Behavior E-Qual Observed Engagement Attention Nonengagement Participation Undifferentiated Behavior High Level Engagement Note:
3.05 2.65 2.92 2.54
0.67 0.78 0.63 0.92
33.13 7.13 52.79 3.58 1.56
9.29 4.74 11.03 6.04 1.30
Means for the CEQ are presented in terms of the Likert scale options of the CEQ scale (i.e., 1 ⫽ not at all typical, 2 ⫽ somewhat typical, 3 ⫽ typical, and 4 ⫽ very typical). Means for the E-Qual scores represent percentages of time observed.
to high correlations within and between the two sets of variables. Before describing the results of the canonical correlation analysis, we will provide a brief overview of the most important correlations. Developmental age was, as expected, highly negatively correlated with global undifferentiated behavior (i.e., teachers’ ratings of children’s typical repetitive behavior). It was moderately positively correlated with global competence and observed participation and moderately negatively correlated with observed nonengagement and observed undifferentiated behavior. Finally, developmental age was somewhat associated with global persistence. These relationships were all expected. We will focus on three other particularly salient variables: global competence, observed participation, and observed nonengagement. Global competence was the first factor derived from the CEQ (McWilliam et al., 1993). As in the original study, global competence was most strongly associated with global persistence, the second factor on the CEQ. Moderate positive correlations were found with developmental age and observed participation. Global undifferentiated behavior, observed nonengagement, and observed undifferentiated behavior were modestly negatively correlated with global competence. Again, these relationships were all expected. Among the observed scores, participation was the most commonly seen behavior (see Table 5). The relationships with developmental age and global competence reported above show that this variable is associated with maturity. It was modestly correlated with global persistence. Moderate negative relationships with global undifferentiated behavior, observed undifferentiated behavior, observed attention, and observed nonengagement confirm that this variable is associated with appropriate active play.
Developmental Age/Global Engagement 1. Developmental Age 2. CEQ Attention 3. CEQ Competence 4. CEQ Persistence 5. CEQ Undifferentiated Behavior Observed Engagement 6. Observed Attention 7. Observed Nonengagement 8. Observed Participation 9. Observed Undifferentiated Behavior 10. Observed High Level Engagement
1.00 .35 .57 .33 .19 ⫺.11 ⫺.19 .14 .24
⫺.08 ⫺.66 .69 ⫺.59 ⫺.10
1.00 ⫺.23 .68 .46 ⫺.87
.14 ⫺.66 .43 ⫺.52
1.00 .86 ⫺.57
.18 ⫺.63 .30 ⫺.40
.02 .52 ⫺.57 .54
1.00 ⫺.21 ⫺.60 ⫺.20
Table 6 Correlations Among Battelle, CEQ, and E-Qual Variables
1.00 ⫺.52 .49
526 de Kruif and McWilliam
Nonengagement is the negative equivalent of total observed engagement; if the child were engaged in any way, nonengagement would not have been coded. The moderate negative correlations with developmental age, global competence, and global persistence indicated that observed nonengagement is not associated with ratings of typical mature engagement. It is, however, associated with global undifferentiated and observed undifferentiated behavior, indicating that children judged to spend much time in repetitive behavior are also likely to be observed in a nonengaged state. Five canonical functions (i.e., pairs of variates) were derived from the canonical correlation analysis. Results of the analysis were interpreted according to the guidelines provided by Thompson (1991), who cautioned researchers not to rely solely on statistical tests when deciding which canonical functions to interpret. He pointed out that tests for significance as offered in statistical packages are not tests of the significance of single functions. Rather, in a set with three canonical correlations, the first test statistic is used to evaluate whether the complete set of functions is zero, or equivalent, that there are no linear relationships between the first and the second set of variables. The second test statistic evaluates the second and third canonical correlation coefficients, and only the third test statistic is a test of a single correlation coefficient (Thompson, in press). Because the significant tests do not provide information about the importance of individual functions, it is suggested that researchers use the squared canonical correlation coefficient (RC2) (describing the proportion of variance that a pair of canonical variates shares) as an indicator of effect size in addition to the significance test (Thompson, 1991). This coefficient is similar to a multiple R2 in regression. In the present study Wilks’ lambda prior to extraction of the first function was statistically significant, F (25, 194.67) ⫽ 5.08, p ⫽ .001 (see Table 7). Inspection of the canonical correlation and the squared canonical correlation indicated that the first pair of variates was strongly correlated and accounted for 70% of the shared variance. Wilks’ lambda prior to extraction of the second function was also statistically significant, F (16, 162.56) ⫽ 2.49, p ⫽ .002. The variates were moderately correlated and accounted for 37% of the shared variance. The high canonical correlation coefficients indicated that relationships between the two sets were unlikely to have occurred by chance. Applying the Wherry (1931) correction formula to the obtained canonical functions to account for “shrinkage” (Thompson, 1990), the first squared canonical correlation was .64. This indicated that, even with this conservative correction of the effect size applied, the variates of Function I still shared 64% of their variance. Applying the Wherry correction formula to the second function yielded a shrunken squared canonical correlation coefficient of .25. To interpret the extent to which the pairs of canonical variates contribute to the multivariate relationship, we used both standardized function coefficients and structure coefficients (see Table 7). Standardized function coefficients or canonical weights are similar to beta weights in regression and indicate the contribution of each variable to the variance of its respective within-set variate (Thompson, 1991). In both analyses these weights are usually not correlation coefficients and therefore do not necessarily range between ⫺1 and ⫹1. A given variable can have
Developmental Age Attention Competence Persistence Undifferentiated Behavior Rc2 Attention Nonengagement Participation Undifferentiated Behavior High Level Engagement
Variables .97 ⫺.06 .81 .66 ⫺.82 ⫺.01 ⫺.88 .80 ⫺.74 .03
.74 ⫺.14 1.25 .16 .02
.77 ⫺.05 ⫺.01 .34 ⫺.05
.94 .00 .66 .43 .68 .70 .00 .77 .64 .55 .00
Squared Structure Coefficient
.44 ⫺.27 ⫺.28 ⫺.16 .89
⫺.24 .46 .42 .66 .28
⫺1.22 ⫺.71 .46 1.25 .06 ⫺.82 ⫺.76 ⫺1.56 ⫺.73 .67
Table 7 Canonical Correlation Analysis for Battelle, CEQ, and Observed Engagement
.06 .21 .18 .44 .08 .37 .20 .07 .08 .02 .80
Squared Structure Coefficient
528 de Kruif and McWilliam
a function coefficient of near-zero when the variable does not contribute to the relationship between the variable sets or when variables in the same set are moderately interrelated. In the latter case, one or more other variables may contain the same information and the variable may be arbitrarily denied credit for providing this information (Thompson, in press). Structure coefficients are the correlations between measured or observed variables (e.g., developmental age, observed attention) and scores on the canonical functions derived by applying the canonical weights to the measured variables (Thompson, 1991). Thus structure coefficients help us to understand the nature (or structure) of the variables being related in the analysis. A measured variable is not noteworthy on a given function if both the standardized function coefficients and structure coefficients are near zero. As previously described and shown in Table 6, variables included in the present analysis within the given sets were relatively highly related with each other, suggesting that merely inspecting the function coefficients may not be reliable in indicating variable contributions. Hence, structure coefficients and squared structure coefficients were consulted for interpretation of the canonical variates. Because only the first two RC2’s were noteworthy and only the first two lambdas were statistically significant, Table 7 only presents the function coefficients, structure coefficients, and squared structure coefficients for these first two functions. Variables within the predictor set (developmental age and CEQ factors) that were correlated most highly with the first canonical variate were developmental age and global competence, global persistence, and global undifferentiated behavior (acceptable cut-off for correlation at .30; see Tabachnick & Fidell, 1996). Among the variables in the observed engagement or the criterion set, nonengagement, participation, and undifferentiated behavior were correlated with the first canonical variate. Specifically, this first pair of canonical variates indicated that children who were developmentally more mature and were rated by their teachers as typically engaging in competent and persistent behavior, but not typically engaging in undifferentiated behavior, were frequently observed participating. These children tended to display less nonengaged and undifferentiated behavior when observed by raters in the classroom. Near-zero values on both standardized function coefficients and structure coefficients indicated that global attention and observed high level engagement contributed little to the first canonical function. The high function coefficient and near-zero structure coefficient for observed attention indicated that this variable actually functioned as a suppressor variable. This means that children’s observed attention contributed little to the multivariate relationship between the two sets of variables but contributed an appreciable amount indirectly by making the other measured variables more related to each other. In conclusion, as indicated by large and homogenous squared structure coefficients ranging from .43 to .94, Function I appeared to be a general function including most of the measured variables. Variables most highly related to Function II were global attention, global competence, global persistence, observed attention, and observed high level engagement. Compared to other variables, global competence, global attention, and observed attention had relatively small function coefficients (e.g., .46, ⫺.71,
de Kruif and McWilliam
⫺.82 vs. 1.25, ⫺1.22. ⫺1.56), and relatively small squared structure coefficients (.18, .21, and .20 respectively), indicating that these variables contributed little to the second canonical function. Function II therefore primarily reflected the bivariate relationship between global persistence and observed high level engagement.
DISCUSSION The results of this study indicate the existence of two uncorrelated patterns of relationships among developmental age, global engagement, and observed engagement variables. First, more developmentally mature children who were rated by their teachers as typically engaging in high level engagement behaviors and less typically engaging in lower level engagement behaviors were indeed more actively involved with their environment and engaged less in lower level behaviors when observed in the classroom. Maturity and teacher-rated high level engagement, however, were not related to observed high level engagement. This finding was supported by the second function, which indicated that children who were rated as typically engaging in persistent behaviors tended to engage in observed high engagement, regardless of developmental age. In addition, children’s attention did not contribute directly to the multivariate relationship among developmental age and the engagement variables. In this section we will list each of these findings, point out possible limitations to the study, and discuss implications for practitioners, administrators, researchers, and personnel development faculty. The nature of the relationship of developmental age with the variables on Function I is generally consistent with findings from previous engagement studies that included developmental age as an independent variable (e.g., Blasco et al., 1993; Malone et al., 1994; McWilliam & Bailey, 1995). In these studies, developmental age was positively related to more sophisticated engagement behaviors and negatively to lower level engagement behaviors (e.g., nonengagement, undifferentiated behavior). Based on the literature, we hypothesized that observed high engagement, consisting of persistence and pretend play, would be related to the first function as well, together with CEQ persistent play, CEQ competence, and developmental age. A possible explanation why this was not the case is the frequency of occurrence of observed persistence and observed pretend play. Both were observed infrequently, which limited their range and may have influenced the low correlations of high engagement with the other variables in the analysis. Low occurrence of persistent behavior may have been an artifact of both our observational coding system and our videotaping methods. For example, a child who is completing a puzzle, looking for pieces, trying to fit pieces in the puzzle over and over again to find the right one is clearly engaging in persistent behavior. Our second-bysecond coding of each behavior, however, may have caused us to lose sight of this broader picture, leading us to alternate short intervals of persistence with short intervals of any of the other engagement levels, thereby reducing the time the child spent in persistent behavior. The age range of the children we were studying
might explain low occurrence of pretend behavior. Although most preschoolers engage in pretend play, most infants and toddlers do not. In addition, we observed that when teachers approached children who were engaged in pretend play and tried to join them in their play, the children often turned to the teacher, which resulted in a different level of engagement (e.g., attention, participation). Observed high level engagement was included in the second function, as was global persistence. The functions in a canonical correlation are uncorrelated. Function II, therefore, reflected the relationship between the variables after the variance associated with Function I was accounted for. Thus, Function II tapped into different relationships across the sets of variables from those represented in Function I. While Function I reflected an increase in sophisticated behavior and a decrease of lower level engagement behaviors when children become more developmentally mature, Function II primarily involved the bivariate relationship between global persistence and observed high engagement. The absence of developmental age in this second function suggested that there is a relationship between global persistence and observed high level engagement that is not age-related. This makes conceptual sense since both younger and older children may engage in persistent behavior (high level engagement consisted of observed persistence and pretend play). At the same time this finding is somewhat worrying given the fact that observed high level engagement only occurred 1.56% of the time. One would expect more mature children to engage in these behaviors at a much higher rate. One explanation might be the fact that children’s engagement was observed in the presence of the teacher. Few studies have examined the influence of adult involvement, and those that have typically examine adult involvement in structured or teacher-led activities (e.g., Karners, Johnson, & Beauchamp, 1989; McWilliam & Bailey, 1995). Because the current study was part of a research project designed to examine the influence of teacher interaction behaviors on child engagement (McWilliam, Scarborough, & Kim, 1999), the teacher was the focus of all taped observations and was therefore present (i.e., interacting with the child or with children in the same activity) in all sessions (i.e., in structured activities as well as free play and mealtimes). Teachers’ interactions with children may have equalized typical age and maturity advantages in terms of higher level engagement behaviors. As mentioned before, we observed how teachers changed the nature of the children’s activity when trying to join children in their play, often without being aware of it. This raises the question whether the presence of the teacher reduced the amount of time developmentally more mature children spent in sophisticated behaviors, increased the amount of time less mature children spent in more sophisticated behaviors, or both. Moreover, these findings emphasize the influence of teacher interaction behaviors on children’s engagement. Given the assumption that the quality of engagement is a critical mediating factor in children’s learning, it is important to help teachers to become aware of how their interaction behaviors may help children to improve the quality of their engagement. The finding that neither global attention nor observed attention directly contributed to the multivariate relationship among the variables included in the
de Kruif and McWilliam
analysis was surprising, especially because the children in this study spent a considerable amount of time attending (i.e., 33.13%) to a teacher or a peer. Based on the literature it was expected that an increase in developmental age would be related to more frequent engagement in sophisticated behaviors and less frequent engagement in lower level behaviors such as undifferentiated behavior and attention (McWilliam & Bailey, 1995; Blasco et al., 1993). Reviewing the literature on the development of attentional behavior in young children revealed reports of conflicting results as to how attentional behavior is related to more sophisticated behaviors. For example, Bornstein and Colombo and their colleagues (see Bornstein & Sigman, 1986; Colombo & Mitchell, 1990) suggested that the duration of looking is negatively related to better functioning in terms of novelty preferences, language development, and later performance on cognitive tests. These findings have been interpreted in terms of the speed and efficiency of processing; children who spend less time looking are more efficient in processing information. Or it may be that children who are more competent to act on their environment are less content passively observing (Ruff & Saltarelli, 1993). Other researchers, however, have reported a positive relationship between children’s attention and competence in play, future performance, and academic achievement (Jennings, Harmon, Morgan, Gaiter, & Yarrow, 1979; Palisin, 1986; Ruff, 1988). Greater attentiveness is likely to lead to new information about the environment that, in turn, provides a better foundation for children’s developing knowledge and cognition (Ruff, 1990). The confusion in the findings of the role of attention in young children’s learning is not new and is related to the way attention is defined in different studies. When attention is defined as the duration of looking at, but not manipulating or interacting with, external events or objects, attention is negatively related to better functioning. But, when attention is defined as looking at an object while manipulating it, duration of attention is a predictor of better cognitive functioning at later ages (Ruff & Saltarelli, 1993; Tamis-LeMonda & Bornstein, 1993). Since we defined attention as the amount of time children look at and listen to features in the environment without manipulating toys, our findings are not consistent with what is reported in the literature. Although attentional behavior is at times appropriate behavior for all children, it is a less active level of engagement and we would not expect more mature children to spend a large percentage of their time merely looking. Again, however, these findings may be typical when observing children’s engagement in the presence of the teacher. Observed attention was not directly related to the other variables in the analysis, but it contributed an appreciable amount by making the relationships among the other variables stronger. The findings of this study must be interpreted with regard to some limitations not yet addressed. First, our method of data collection (i.e., videotaping) may not have been as natural as intended. Although the teachers were free to choose what types of session were taped in the first couple of sessions, they were subsequently told which sessions were needed. In addition, the teachers were asked during mealtime to move around instead of sitting at one table to capture the interaction with as many children as possible in one session. On the other hand, we taped all
sessions in natural environments (as opposed to laboratory situations) and at natural times of the day (vs. out-of-context situations). Second, it has been suggested that canonical weights and loadings are most reliable for detecting the most important variables and for interpreting the canonical variates when there is a large ratio of sample size to the number of variables. Barcikowski and Stevens (1975) suggested a ratio of at least 42:1 for interpreting the first two sets of canonical variations and about 20:1 for interpreting only the first set of canonical covariates. Thompson (1990), however, reported that estimates of the canonical correlations and eigenvalues appear to be reasonably stable if the researcher uses at least 5–10 subjects per variable. With a subject to variable ratio of 6.2:1 we can be relatively confident that the results of our analysis are “reasonably stable.” In conclusion, this study shows that, as children mature, their engagement observed in the presence of the teacher becomes more competent and they shed some lower level behaviors. This seemingly obvious finding is important for assessment and intervention planning. With regard to assessment, teachers should assess both global and observed engagement. Rating children’s global engagement and observing their engagement in classroom settings gives caregivers (parents and teachers) information about functional activities from which they can develop intervention or curricular goals. With regard to intervention planning, teachers should be aware that children rated as less competent and persistent are likely to have problems with participation (i.e., teachers should plan activities that encourage children’s active involvement with adults, peers, and materials in the classroom). In addition, if a child’s global rated persistence is high, teachers should ensure that the child has lots of opportunities for high level engagement. These opportunities can include such practices as using problem solving materials, challenging activities, “sabotaged” situations, housekeeping centers, dramatic play activities, and informal pretend play interactions. Furthermore, these findings have implications for faculty involved in personnel development, administrators, and researchers. Personnel development faculty should (a) help teachers become aware of the importance of engagement in children’s learning, (b) introduce teachers to global and observational measures that are helpful in determining children’s level of engagement, (c) introduce teachers to developmentally appropriate practices that promote high levels of engagement for all children, and (d) help teachers become aware of the factors (such as their interaction behaviors) that influence engagement. Administrators should provide a structure for assessment of global engagement and observed engagement. Researchers should continue to explore links among intra-individual factors influencing engagement, especially between ratings of child propensity and observations of behavior. Acknowledgments: This research was funded by a grant from the Office of Special Education and Rehabilitative Services, U. S. Department of Education, Grant No. HO23C40015. The authors thank the participating teachers and children. Thanks also to Katherine Harville, Ellen Reilly, Anita Scarborough, Kimberley Sloper, Virginia Smith, Alyse Sweeney, Don Trull, and Rebecca Zulli.
de Kruif and McWilliam
Finally, the authors are appreciative of specific comments by two anonymous reviewers on an earlier version of this article.
REFERENCES Barcikowski, R., & Stevens, J. P. (1975). A Monte Carlo study of the stability of canonical correlations, canonical weights and canonical variate-variable correlations. Multivariate Behavioral Research, 10, 353–364. Blasco, P. M., Bailey, D. B., & Burchinal, M. A. (1993). Dimensions of mastery in same-age and mixed-age integrated classrooms. Early Childhood Research Quarterly, 8, 193–206. Bornstein, M. H., & Sigman, M. D. (1986). Continuity in mental development from infancy. Child Development, 57, 251–274. Brockman, L. M., Morgan, G. A., & Harmon, R. J. (1988). Mastery motivation and developmental delay. In T. D. Wachs & R. Sheenan (Eds.), Assessment of young developmentally disabled children. Perspectives in developmental psychology (pp. 267–284). New York, NY: Plenum Press. Buysse, V., & Bailey, D. B. (1993). Behavioral and developmental outcomes in young children with disabilities in integrated and segregated settings: A review of comparative studies. Journal of Special Education, 26, 434 – 461. Carta, J. J., & Greenwood, C. R. (1985). Eco-behavioral Assessment: A methodology for expanding the evaluation of early intervention programs. Topics in early Childhood Special Education, 5(2), 88 –103. Colombo, J., & Mitchell, D. W. (1990). Individual differences in early visual attention: Fixation time and information processing. In J. Colombo & J. W. Fagan (Eds.), Individual differences in infancy: Reliability, stability, and prediction. Hillsdale, NJ: Erlbaum. Doke, L. A., & Risley, T. R. (1972). The organization of daycare environments: Required versus optional activities. Journal of Applied Behavior Analysis, 5, 405– 420. Dunst, C. J., McWilliam, R. A., & Holbert, K. (1986). Assessment of preschool classroom environments. Diagnostique, 11, 212–232. Greenwood, C. R., & Carta, J. J. (1987). An ecobehavioral interaction analysis of instruction within special education. Focus on Exceptional Children, 19(9), 1–12. Hollingshead, A. B. (1975). Four-factor index of social status. Unpublished manual. Department of Sociology, Yale University, New Haven, CT. Jennings, K. D., Harmon, R. J., Morgan, G. A., Gaiter, J. L., & Yarrow, L. J. (1979). Exploratory play as an index of mastery motivation: Relationships to persistence, cognitive functioning, and environmental measures. Developmental Psychology, 15, 386 –394. Jones, H. A., & Warren, S. F. (1991). Enhancing engagement in early language teaching. Teaching Exceptional Children, 23(4), 48 –50. Karnes, M. B., Johnson, L. J., Beauchamp, K. D. F. (1989). Developing problem-solving skills to enhance task persistence in handicapped preschool children. Journal of Early Intervention, 13(1), 61–72. Krantz, P., & Risley, T. R. (1977). Behavioral ecology in the classroom. In K. D. O’Leary & S. G. O’Leary (Eds.), Classroom management: The successful use of behavior modification (2nd ed., pp. 349 –367). New York: Pergamon. MacTurk, R. H., Hunter, F. T., McCarthy, M. E., Vietze, P. M., & McQuiston, S. (1985). Social mastery motivation in Down syndrome and nondelayed infants. Topics in Early Childhood Special Education, 4(4), 93–109. Malone, D. M., Stoneman, Z., & Langone, J. (1994). Contextual variation of correspondences among measures of play and developmental level of preschool children. Journal of Early Intervention, 18(2), 199 –215. McClannahan, L. E., & Risley, T. R. (1975). Design of living environments for nursing-home residents: Increasing participation in recreation activities. Journal of Applied Behavior Analysis, 8(3), 261–268.
McCormick, L., Noonan, M. J., & Heck, R. (1998). An exploration of variables affecting engagement in inclusive preschool classrooms. Journal of Early Intervention, 21(2), 160 –176. McGarity, J. R., & Butts, D. P. (1984). The relationship among teacher classroom management behavior, student engagement, and student achievement of middle and high school science students of varying aptitude. Journal of Research in Science Teaching, 2(1), 55– 61. McWilliam, R. A. (1991). Children’s Engagement Questionnaire. Chapel Hill, NC: Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill. McWilliam, R. A. (1995). E-Qual Observational Coding System. Chapel Hill, NC: Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill. McWilliam, R. A., & Bailey, D. B. (1991). The effects of activity structure and age mixture on child engagement. Chapel Hill, NC: Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill. McWilliam, R. A., & Bailey, D. B. (1992). Promoting engagement and mastery. In D. B. Bailey & M. Wolery (Eds.), Teaching infants and preschoolers with disabilities (2nd ed., pp. 230 – 255). New York: MacMillan Publishing Company. McWilliam, R. A., & Bailey, D. B. (1995). Effects of classroom social structure and disability on engagement. Topics in Early Childhood Special Education, 15(2), 123–147. McWilliam, R. A., Scarborough, A. A., & Kim, H. (1999). Adult interactions and child engagement. Manuscript under review. McWilliam, R. A., Snyder, P., & Lawson, S. (1993). Dependability and factorial validity of the Children’s Engagement Questionnaire: A broad-based child status measure for early intervention. Paper presented at the DEC Conference, San Diego, CA. McWilliam, R. A., Trivette, C. M., & Dunst, C. J. (1985). Behavior engagement as a measure of the efficacy of early intervention. Analysis and Intervention in Developmental Disabilities, 5, 33– 45. Messer, D. J., Rachford, D., McCarthy, M. E., & Yarrow, L. J. (1987). Assessment of mastery motivation behavior at 30 months: Analysis of task-directed activities. Developmental Psychology, 23, 771–781. Montes, F., & Risley, T. R. (1975). Evaluating traditional day care practices: An empirical approach. Child Care Quarterly, 4, 208 –215. Morgan, G. A., Maslin-Cole, C., Harmon, R. J., Busch-Rossnagel, N. A., Jennings, K. D., Hauser-Cram, P., & Brockman, L. M. (1993). Parent and teacher perceptions of young children’s mastery motivation. Assessment and review of research. In D. J. Messer (Ed.), Mastery motivation in early childhood. Development and measurement and social processes (pp. 109 –122). London: Routledge. Morgan, G. A., Maslin, C. A., & Harmon, R. J. (1987). Dimension of mastery questionnaire. Unpublished instrument. Fort Collins: Department of Human Development and Family Studies, Colorado State University. Newborg, J., Stock, J. R., Wnek, L., Guidubaldi, J., & Svinicki, J. (1984). Battelle Developmental Inventory. Allen, TX: DLM Teaching Resources. Ostrosky, M. M., & Kaiser, A. P. (1991). Preschool classroom environments that promote communication. Teaching Exceptional Children, 23(4), 6 –10. Palisin, H. (1986). Preschool temperament and performance on achievement tests. Developmental Psychology, 22, 766 –770. Rogers-Warren, A. K. (1982). Behavioral ecology in classrooms for young handicapped children. Topics in Early Childhood Special Education, 2(1), 21–32. Rosenshine, B. V. (1978). Academic engaged time, content covered, and director instruction. In H. Leitenberg (Ed.), Handbook of behavior modification and behavior (pp. 453– 474). Englewood Cliffs, NJ: Prentice-Hall. Ruff, H. A. (1988). The measurement of attention in high-risk infants. In P. Vietze & H. G. Vaughan (Eds.), Early identification of infants with developmental disabilities. Philadelphia: Grune & Stratton. Ruff, H. A. (1990). Individual differences in sustained attention during infancy. In J. Colombo & J. Fagen (Eds.), Individual differences in infancy: Reliability, stability, prediction (pp. 247– 270). Hillsdale, NJ: Erlbaum.
de Kruif and McWilliam
Ruff, H. A., & Saltarelli, L. M. (1993). Exploratory play with objects: Basic cognitive processes and individual differences. In M. H. Bornstein & A. W. O’Reilly (Eds.), The role of play in the development of human thought (pp. 5–16). San Francisco, CA: Jossey-Bass Publishers. Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.), New York NY: Harper Collins. Tamis-LeMonda, C. S., & Bornstein, M. (1993). Play and its relations to other mental functions in the child. In M. H. Bornstein & A. W. O’Reilly (Eds.), The role of play in the development of thought. San Francisco, CA: Jossey-Bass Publishers. The Observer 3.0 [Computer software]. (1993). Wageningen, The Netherlands: Noldus Information Technology. Thompson, B. (1984). Canonical correlation analysis: Uses and interpretation. Newbury Park, CA: Sage. Thompson, B. (1990). Finding a correction for the sampling error in multivariate measures of relationship: A Monte Carlo study. Educational and Psychological Measurement, 50, 15–31. Thompson, B. (1991). Methods, plainly speaking: A primer on the logic and use of canonical correlation analysis. Measurement and Evaluation in Counseling and Development, 24, 80 –93. Thompson, B. (in press). Canonical correlation analysis. In L. Grimm & P. Yarnold (Eds.), Reading and understanding multivariate statistics (Vol. 2). Washington, DC: American Psychological Association. Twardosz, S., & Risley, T. R. (1982). Behavioral ecological consultation to daycare centers. In A. Jean & R. Slotnick (Eds.), Community mental health and behavioral ecology: A handbook of theory, research, and practice (pp. 147–159). New York: Plenum. Twardosz, S., Cataldo, M. F., & Risley, T. R. (1974). Open environment design for infant and toddler day care. Journal of Applied Behavior Analysis, 7, 529 –546. Warren, S. F., & Kaiser, A. P. (1986). Incidental language teaching: A critical view. Journal of Speech and Hearing Disorders, 51, 291–298. Whaley, K., & Bennett, T. C. (1991). Promoting engagement in early childhood social education. Teaching Exceptional Children, 23(4), 51–54. Wherry, R. J. (1931). A new formula for predicting the shrinkage of the coefficient of multiple correlation. Annals of Mathematical Statistics, 2, 440 – 451.