Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Oct 1.
Published in final edited form as: J Child Psychol Psychiatry. 2014 Mar 26;55(10):1162–1171. doi: 10.1111/jcpp.12233

Classes of Oppositional-Defiant behaviour: Concurrent and predictive validity

Robert R Althoff 1,2,3, Ana V Kuny-Slock 5, Frank C Verhulst 3, James J Hudziak 1,3,4, Jan van der Ende 3
PMCID: PMC4159429  NIHMSID: NIHMS570012  PMID: 24673629

Abstract

Background

Oppositional Defiant Disorder (ODD) has components of both irritability and defiance. It remains unclear whether children with variation in these domains have different adult outcomes. This study examined the concurrent and predictive validity of classes of oppositional defiant behavior.

Methods

Latent Class Analysis was performed on the Oppositional Defiant Problems scale of the Child Behavior Checklist in two samples, one in the U.S. (the Achenbach Normative Sample, N=2029) and one in The Netherlands (the Zuid-Holland Study, N=2076). A third sample of American children (The Vermont Family Study, N=399) was examined to determine concurrent validity with DSM diagnoses. Predictive validity over 14 years was assessed using the Zuid-Holland Study.

Results

4 classes of oppositional defiant problems were consistent in the two latent class analyses: No Symptoms, All Symptoms, Irritable, and Defiant. Individuals in the No Symptoms Class were rarely diagnosed concurrently with ODD or any future disorder. Individuals in the All Symptoms Class had an increased frequency of concurrent childhood diagnosis of ODD and of violence in adulthood. Subjects in the Irritable Class had low concurrent diagnosis of ODD, but increased odds of adult mood disorders. Individuals in the Defiant Class had low concurrent diagnosis of ODD, but had increased odds of violence as adults.

Conclusions

Only children in the All Symptoms class were likely to have a concurrent diagnosis of ODD. Although not diagnosed with ODD, children in the Irritable Class were more likely to have adult mood disorders and children in the Defiant Class were more likely to engage in violent behavior.

Keywords: Oppositional defiant disorder (ODD), longitudinal studies, validity, Child Behaviour Check List

There are many ways to conceptualize and study aggressive behavior in children. Conduct disorder (CD), oppositional defiant disorder (ODD) and attention-deficit/hyperactivity disorder (ADHD) are the most common reasons that children are referred for mental health services, with ODD and CD most often being associated with aggression [for review see (Loeber, Burke, Lahey, Winters, & Zera, 2000)]. ODD has often been conceptualized as the milder version and prodrome of CD because it includes symptoms that are close to ‘normal’ behavior (e.g., losing one’s temper, arguing with adults); however, it has become clear that ODD may not be as harmless as previously thought. Instead of serving as prodrome for CD, ODD exists on its own, and may play a role in the development of a wide range of child psychopathology, including anxiety, depression, CD and later the development of antisocial personality disorder (Loeber, Burke, & Pardini, 2009; Rowe, Costello, Angold, Copeland, & Maughan, 2010) with ODD alone at age 7–12 predicting poor social and interpersonal relationships at age 24 (Burke, Rowe, & Boylan, 2013). Furthermore, ODD as a long-term predictor of many other disorders holds in childhood and adolescence even when controlling for co-occurring disorders (Copeland, Shanahan, Costello, & Angold, 2009). In fact, dimensions of ODD appear to mediate the longitudinal relations between CD and later depression in girls (Hipwell et al., 2011).

In the last decade, studies of oppositional behavior have moved away from focusing only on children extreme enough to meet criteria for the DSM diagnoses of ODD or CD to examining the relations of symptom domains within ODD and other psychopathology. It is now widely appreciated (and reified in the DSM-5 criteria) that there are separable dimensions within ODD that distinguish “irritable non-defiance” and “defiant non-irritability” in keeping with findings in the literature of the distinction between Reactive-affective-defensive-impulsive (RADI) aggression vs. Proactive-instrumental-planned-predatory (PIPP) aggression (Steiner, Saxena, & Chang, 2003). Multiple studies have now examined a two- or three- factor model within the DSM criteria for ODD (Aebi, Plattner, Metzke, Bessler, & Steinhausen, 2013; Burke, Hipwell, & Loeber, 2010; Stringaris & Goodman, 2009; Stringaris, Zavos, Leibenluft, Maughan, & Eley, 2012; Whelan, Stringaris, Maughan, & Barker, 2013). There are small disagreements in these studies as to where to place certain items, but all agree on the separation of an irritable dimension from a headstrong and/or hurtful dimension [although a developmental model of disruptive behavior has been proposed by others (Wakschlag et al., 2012)]. Similar dimensions have been described in preschoolers (Ezpeleta, Granero, de la Osa, Penelo, & Domènech, 2012; Wakschlag et al., 2012), in other studies of school aged children (Aebi et al., 2010), and even in autistic populations (Mandy, Roughan, & Skuse, 2013).

Beginning with Stringaris & Goodman (2009), the importance of these dimensions is that they demonstrate differential prediction for future psychopathology with the irritable dimension predicting depressive or internalizing disorders and headstrong or hurtful dimensions predicting more disruptive behavior disorders (Aebi et al., 2013; Burke, 2012; Burke et al., 2010; Rowe et al., 2010; Stringaris & Goodman, 2009; Stringaris et al., 2012; Whelan et al., 2013). This is consistent with work on irritability as a construct demonstrating associations within preschoolers (Dougherty, Tolep, Smith, & Rose, 2013) and school-aged children (Stringaris, Cohen, Pine, & Leibenluft, 2009) between irritability and later depressive symptoms. Similar work examining treatment resistance showed that irritability was associated with treatment-resistant ODD while hurtfulness was associated with treatment-resistant CD (Kolko & Pardini, 2010).

The dimensions of ODD have mostly been examined from a variable-centered approach. That is, the symptoms themselves have been examined as predictors. Fewer studies have used a person-centered approach whereby children high on one dimension and not the other are differentially examined. Latent Class Analysis (LCA) is a form of bottom-up, person-centered data analysis that has been used to study classes of children across the continuum of behaviors. There have been studies in ADHD (Althoff, Copeland, et al., 2006; Neuman et al., 2001; Todd, Lobos, Sun, & Neuman, 2003), obsessive compulsive disorder (Althoff, Rettew, Boomsma, & Hudziak, 2009), dysregulated behavior (Althoff, Rettew, Faraone, Boomsma, & Hudziak, 2006), tic disorders (Nestadt et al., 2003), and alcohol use disorders (Rindskopf, 2006), among others. LCA has been used in two examinations of ODD symptoms. Burke (2012) conducted an analysis of ODD using a clinic-referred sample of 177 boys aged 7–12 and found a 3-class model consistent with the dimensional models. In a large, general population sample of twins, Kuny et al. (Kuny et al., 2013) performed an LCA of mother’s report on an oppositional scale of the Conners’ Parent Rating Scales Revised Short Forms (Conners, 2001) in Dutch twins (aged 7–12) from the Netherlands Twin Registry (Boomsma, 1998). The results of the LCA identified 4-classes of oppositional behavior across age groups – a high symptoms class, a low symptoms class, and two intermediate classes – one characterized by defiance but not irritability and the other characterized by irritability, but not defiance. Thus, person-centered approaches suggest a separation between children with irritability, defiance, none, or both. Burke further demonstrated that children with irritability were more likely to demonstrate anxiety or depression at age 18.

The current study was designed to test similar dimensions in childhood using a person-centered approach and to examine their utility and predictive power. Specifically, the research was centered around two aims – 1) a concurrent validity aim: comparing current diagnoses of ODD across the subtypes of behavior; and 2) a predictive validity aim: comparing the adult outcomes of subtypes of oppositional behavior. We hypothesized that 1) there would be four classes of children with oppositional defiant behavior consisting of a class with high symptoms across irritable and defiant domains, a class with only irritability, a class with only defiance, and a low symptoms class; 2) given the weight placed on headstrong/hurtful items in an ODD diagnosis, only the classes with high levels of defiance would be associated with concurrent ODD while irritability without defiance would not predict an ODD diagnosis; and 3) defiance in childhood would be associated with adult violent behavior, but that irritability without defiance would be associated instead with adult mood disorders and not adult violent behavior.

METHODS

Participants

Participants for this research came from three sources. For the fitting of latent class models, U.S. participants came from the Achenbach Normative Sample (Achenbach, 1991) and Dutch participants came from the Zuid-Holland Study, a seven-wave longitudinal study of behavioral and emotional problems in children (Verhulst, Akkerhuis, & Althaus, 1985). For the study of concurrent validity, U.S. participants came from the Vermont Family Study (Hudziak, Copeland, Stanger, & Wadsworth, 2004). The institutional review board approved all studies and all participants provided informed consent.

The Achenbach Normative Sample (ANS)

Participants were American nonhandicapped, clinically-referred and nonhandicapped, nonclinically referred children and adolescents from a national sample collected in 1999 (Achenbach & Rescorla, 2001). This sample has been described in detail elsewhere (Achenbach, Dumenci, & Rescorla, 2002). Briefly, data were obtained from home interview surveys with the parents of participants chosen to be representative of the contiguous 48 states. These surveys included the CBCL for parent or guardian report and other questions regarding demographics and the participant’s mental health and special education history. The sample consisted of 2029 children of whom 276 had been referred for mental health services in the preceding 12 months. 1073 (53%) participants were male. All children ranged in age from 6–18 with mean age 11.98 years (SD 3.53).

The Zuid-Holland Sample (ZHS)

The original sample of 2,600 children from 13 birth cohorts aged 4 to 16 was drawn from municipal registers that list all residents in the Dutch province of Zuid-Holland. More comprehensive details of the study can be found elsewhere (Hofstra, Van Der Ende, & Verhulst, 2001). The study initially contacted 2,447 parents of which 2,076 (i.e., 84.8%) provided data. The original sample was contacted every 2 years starting in 1983 and ending in 1991, at which time the participants were not contacted again until 1997 (Wave 6). The initial sample consisted of 49% boys and 51% girls. Demographic information at Wave 1 and Wave 6 is shown in Table 2 along with the measures collected across the time points. Differences between participants who dropped out of the study and those who completed the study were minor (Althoff, Verhulst, Rettew, Hudziak, & van der Ende, 2010). Overall levels of psychopathology in children in The Netherlands are demonstrably similar to U.S. populations placing them at the omnicultural mean for overall problems on the CBCL, like the U.S. (Ivanova et al., 2007), allowing for the study of patterns of problem behavior separable from symptom severity.

Table 2.

Descriptive Variables Zuid-Holland Longitudinal Study

Wave Cohort Year Ages Prop. Female Mean Age (SD) Years follow-up No. Completed CBCL No. Completed VIO No. (Prop.) + for VIO No. CompletedCIDI No. (Prop.) + for CIDI Mood Disorder
1 1983 3–17 .51 9.67 (3.72) 0 2076 - - - -
6 1997 18–32 .54 24.35 (3.75) 14 1388 114 (0.082) 1390 75 (0.054)
Open in a new tab

Note: CBCL = Child Behavior Checklist, VIO = Violence Measure, CIDI = DSM Interview, No. = number, Prop. = proportion, CIDI = Composite International Diagnostic Interview

The Vermont Family Study (VFS)

The VFS is a family-study of aggression and attention problems which has been described in detail elsewhere (Hudziak et al., 2004). Briefly, families participating in the study were recruited through the use of newspaper advertisements and posters, as well as by local pediatricians and psychiatrists practicing in a university-based outpatient clinic. A total of 205 families participated in the study and data were obtained for 168 probands (with either attention problems, aggressive behavior, or both) and 231 of their siblings (total N = 399 children; 212 boys and 187 girls; mean age 10.88 years; SD, 3.06 years). In the present study, all probands and siblings with complete mother-rated CBCL data were included.

Measures

Child Behavior Checklist [CBCL; (Achenbach & Rescorla, 2001)]

The CBCL is a 118-item parent report of child behavioral, emotional and social problems with items endorsed on a 3-point scale yielding a quantitative score for 8 empirically-based problem scales and 6 DSM-IV-oriented problem scales. The CBCL also has been shown to be a reliable measure with good internal consistency as measure by Cronbach’s alpha ranging between .72 (anxiety problems on the DSM-oriented scale) and .94 (aggressive behavior on the empirically-derived syndromes). The test-retest reliability is also very good, ranging between .80 (anxiety problem on the DSM-oriented scale) and .93 (attention deficit hyperactivity problems on the DSM-oriented scale). The DSM-IV problem scales were derived by expert clinicians who were asked to rate the items from the CBCL which most corresponded to certain DSM diagnoses. In the present study, the variables from the CBCL Oppositional Defiant Problems Scale (see Table 1) were used. This specific scale has a test-retest reliability of .85 in the US sample and .80 in the Dutch sample. It has good internal consistency as measured by Cronbach’s alpha .86 in the US sample and .77 in the Dutch sample, and has been shown to predict DSM-IV ODD (Achenbach & Rescorla, 2001). Good reliability and validity estimates of the Dutch version of the ASEBA checklists (i.e., CBCL, ABCL, and ASR) have been confirmed (Verhulst, van der Ende, & Koot, 1996) with Cronbach alphas ranging from .54 to .91 and test-retest reliability ranging from .74 to .89. The US version was used in the ANS and the VFS while the Dutch version was used in the ZHS.

Table 1.

Individual Items of Child Behavior Checklist Oppositional Defiant Problems Scale

CBCL Item Question
3. Argues a lot
22. Disobedient at home
23. Disobedient at school
86. Stubborn, sullen, or irritable
95. Temper tantrums or hot temper
Open in a new tab

DSM-IV Checklist (Hudziak, 1998)

To assess concurrent validity in the VFS the DSM-IV Checklist was used. The DSM-IV Checklist is a semi-structured clinical interview that was developed to allow clinicians to assess symptoms for common childhood psychiatric diagnoses according to the definitions and criteria of DSM-IV. It consists of questions that are directly quoted from the DSM-IV symptom criteria and was administered by trained mental health providers to parents and children. Test–retest reliability of this measure has been found to be adequate (Hudziak et al., 2004). Assessments for lifetime ADHD, Any Anxiety Disorder (Generalized Anxiety Disorder or Separation Anxiety Disorder), Conduct Disorder, and ODD were used. Kappa statistics for disruptive behavior disorders range from .43 for Conduct Disorder to .85 for ADHD (mean κ = .619). The intra-class correlation coefficient for ODD is .74.

Composite International Diagnostic Interview [CIDI; (Essau & Wittchen, 1993)]

To assess predictive validity in the ZHS, the CIDI was used. The CIDI was used for measurement of adult mood disorder outcomes. The CIDI is a comprehensive, fully-structured interview designed to be used by trained lay interviewers for the assessment of mental disorders according to the definitions and criteria of ICD-10 and DSM-IV. It is intended for use in epidemiological and cross-cultural studies as well as for clinical and research purposes. The diagnostic section of the interview is based on the World Health Organization’s Composite International Diagnostic Interview (Wittchen, Kessler, & Ustun, 2001). Adult DSM diagnostic data were collected at Wave 6 (14-years after Wave 1 data collection) using the computerized version CIDI (Essau & Wittchen, 1993). The CIDI has shown good test-retest and excellent inter-rater reliability for DSM diagnoses and is the most commonly used instrument for DSM diagnosis in epidemiological studies (Andrews & Peters, 1998). For the present study, the variable of interest (i.e., “any mood disorder”) was coded “yes” or “no” according to whether individuals at any time met diagnostic criteria for any DSM-IV mood disorder.

Self-Reported Violent Delinquency (Elliott & Huizinga, 1989)

To assess violent behavior in adulthood in the ZHS, a face-to-face interview was conducted at Wave 6 using a standardized questionnaire (Elliott & Huizinga, 1989). This questionnaire is based on a modified version of the questionnaire developed for the International Self- Report Delinquency Study (Junger-Tas, Terlouw, & Klein, 1994). The adaptation was done to make the questionnaire suitable for adults. The portion of interview used contained six questions on violent criminal behavior [items: (1) armed robbery, (2) threatening behavior with a weapon, (3) threatening behavior without a weapon, (4) violence within family, (5) violence outside of family, and (6) physical attack]. “Yes” or “no” responses to these questions indicated whether the participants had committed any of these acts in the previous 5 years. Responses were then recoded into one dichotomous variable assessing whether an individual endorsed any of the six violent behavior questions. The resulting variable “violent ever” (i.e., a person had positively endorsed at least one violent behavior question) was then used in subsequent analyses (yes: n = 114; no: n = 1277).

Data Analyses

Two separate, but parallel analyses were performed. To assess concurrent validity, latent classes from the U.S. sample were compared to DSM-IV ODD diagnosis. To assess the specific predictive hypotheses that the classes would differentially predict mood disorders versus criminal behavior in adulthood, latent classes from childhood in the Dutch sample were compared to DSM-IV mood disorder diagnoses and criminal behavior in adulthood.

Latent Class Analysis

To determine latent structure of the CBCL Oppositional Defiant Problems (ODP) scale, for both the ANS and the ZHS, latent class analysis (LCA) was performed on the five ODP subscale items. Latent class models were fit by means of an Expectation Maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977) with the program Latent Gold (Vermunt & Magidson, 2000). Based on previous models of oppositional behavior, models estimating 1-class through 5-class solutions were compared. To calculate the best fitting model, goodness of fit was assessed using Latent Gold’s bootstrapping algorithm with final models compared to chance. Models were compared using changes in the Bayesian Information Criterion (BIC) when moving from one class solution to the next. The BIC is a goodness-of-fit index that considers the rule of parsimony. Sex and age were entered as covariates. For the purpose of the LCA, trichotomous (0, 1, 2) responses from items of the ODP scale were entered as dichotomous variables (0 = 0 and 1 or 2 = 1). This is similar to previous procedures used in other studies (Althoff, Rettew, et al., 2006) and on the recommendation of the CBCL manuals (Achenbach & Rescorla, 2001). Further, we have performed extensive testing of various methods of categorizing data in previous studies and the results vary only slightly with different methods (Kuny et al., 2013). The resulting classes are mutually exclusive with each having its own particular pattern of item endorsement. This analysis results in two metrics (among others): (1) the probability of class membership for each individual and (2) the probability of item endorsement for each class. The class that is most probable for a particular individual was then used in subsequent analyses.

Concurrent and predictive validity

Once an adequate model fit was obtained in the ANS, the resulting class weights were used to fit the participants from the VFS into classes. Then, analyses were performed to test whether ODP-scale derived classes would be associated with DSM-IV checklist ODD diagnosis. Because there were participants clustered within families, logistic regression with family clustering was performed using the program Mplus (Muthén & Muthén, 2007). Class assignment was used as the predictor and lifetime DSM-IV Checklist derived ODD diagnosis was used as the outcome variable along with lifetime ADHD, CD, Anxiety and sex. Logistic regression was performed for each class.

For examining adult outcomes of the classes, we first fit the models to the ZHS at Wave 1, and used SPSS 17.0 to place the latent class membership into logistic regression analysis. The outcomes of CIDI mood disorder diagnosis and self-reported violence at Wave 6 were examined separately. Sex and class membership at Wave 1 were used as predictors (with Class 1 as a reference).

RESULTS

Cross-cultural Structure of Oppositional Behavior

LCA on both the Achenbach Longitudinal Sample (ANS) and the Zuid-Holland Sample (ZHS) identified an optimal solution of 4-classes across age groups co-varying for sex. In both analyses, moving from a 3-class solution to a 4-class solution improved the fit by decreasing the BIC, while moving to a 5-class solution worsened the model fit indexed by an increase in the BIC. In addition, dropping sex as a covariate did not improve the model fit and increased the BIC while dropping age as a covariate decreased the BIC, indicating that it could be dropped from the analysis. Final models with sex covariate and without age demonstrated adequate fit by bootstrapping. The final model fits are presented in Table 3. The classes, along with the percentage of the samples falling into the latent classes are shown in Figures 1a and 1b.

Table 3.

LCA model fits: Bayesian Information Criterion (BIC)

Number of Classes Achenbach Sample BIC Zuid-Holland Sample BIC
1 12853.5064 11090.90
2 11237.1209 9991.80
3 11171.6957 9970.63
4 11133.0365 9964.14
5 11173.7172 9995.32
Best 4-Class – reduced bivariate residuals 11133.0365 9958.44
Open in a new tab

Figure 1.

Figure 1

Figure 1

Open in a new tab

Figure 1a. Vermont Family Study Latent Classes

Figure 1b. Zuid-Holland Latent Classes

The classes in this analysis were similar to the classes from previous work (Kuny et al., 2013) and the classes were very similar across Dutch and US societies. Specifically, Class 1 was a no or low symptom class and Class 4 had high symptoms across all symptoms (labeled the Low Symptoms and High Symptoms Classes). Class 2 showed higher defiance at home (and at school in the US sample) and lower irritability. We labeled this the Defiant Class. For children in Class 3, parents endorsed the defiance items at a lower level compared to children in the Defiant Class or the High Symptoms Class. In fact, children in Class 3 were rated almost as low on defiance as children in the Low Symptoms Class but were rated with higher irritability. Consequently, in keeping with previous research, we labeled this the Irritable Class. There was a high level of arguing in both Defiant and Irritable Classes, consistent with Burke (2012). The Dutch sample had qualitatively lower levels of sullenness and disobedience at school in the defiant class than the US model.

Concurrent Validity

The results of the logistic regression of latent class versus current diagnosis (Table 4) demonstrated that class assignment was significantly associated with endorsement of lifetime ODD during childhood. Being in the Low Symptoms Class was significantly association with not having ODD and ADHD. There was no significant association between being in the Irritable Class with a lifetime DSM-IV ODD diagnosis but there was an association with a lifetime diagnosis of Any Anxiety Disorder. Children in the High Symptoms Class were significantly more likely to have received an ODD diagnosis on interview than children placed in any other class, along with ODD, CD, and ADHD. Children in the Defiant Class, while numerically more likely than children in either the Low Symptoms or the Irritable Classes to receive an ODD diagnosis, were not statistically more likely to have a concurrent ODD diagnosis, or any other diagnosis, than children in the other classes.

Table 4.

Concurrent validity in Vermont Family Study – Logistic regression of classes on diagnosis of ODD.

Class Description Number (prop.) of Sample in Class Number (prop.) Female Number (prop.) with completed DSM interview data Number (prop.) of Class with ODD Odds Ratio ODD Odds Ratio CD Odds Ratio Anxiety Odds Ratio ADHD
Class 1 (Low Symptoms) 140 (.35) 76 (.54) 121 (.86) 17 (.14) .17 (.10 – 30)** .38 (.07 – 2.01) 1.20 (.48 – 2.97) .47 (.29 –.76)*
Class 2 (Defiant) 29 (.07) 8 (.28) 28 (.97) 15 (.54) 1.36 (.65 – 2.89) 1.43 (.48 – 4.24) .23 (.05 – 1.17) 1.35 (.59 – 3.31)
Class 3 (Irritable) 88 (.22) 46 (.52) 77 (.88) 23 (.30) .61 (.33 – 1.12) 0 (0)** 3.89 (1.74 – 8.67)** .59 (.34 – 1.03)
Class 4 (High Symptoms) 142 (.36) 57 (.40) 126 (.89) 97 (.77) 6.91 (4.23 – 11.28)** 3.32 (1.24 – 8.92)* .44 (.21 – .91) 2.96 (1.77 – 4.96)**
 Total 399 187 (.47) 352 (.88) 152 (.43)
Open in a new tab
*

p<0.05

**

p < .001;

CI = Confidence Interval; prop. = proportion; ODD = diagnosis of oppositional defiant disorder, CD = diagnosis of conduct disorder; ADHD = diagnosis of Attention Deficit/Hyperactivity Disorder

Adult Outcomes (Predictive Validity)

Results from the logistic regression analyses of childhood latent class versus adult outcomes are shown in Tables 5a and 5b. In all cases Class 1 (the Low Symptoms Class) was used as the reference class. For criminal behavior, membership in either the Defiant Class or the High Symptoms Class at Time 1 was associated with endorsement of delinquent behavior at Time 6. Thus, children who were characterized by their mothers as being the most defiant were also the most likely to self-report violent criminal behavior in adulthood. Conversely, children in the Irritable Class were no more likely than the Low Symptoms Class to endorse violent criminal behavior in adulthood. In addition, as expected, sex was significantly associated with criminal behavior, with males being more likely to endorse the delinquency items than females. Considering criminal behavior as a pseudocontinuous measure by adding all status violations together and entering this measure into a general linear model yielded similar results, with the only contrast reaching significance (p <.05) between the Defiant Class (mean = .18, SD = .54) and the Low Symptoms Class (mean = 0.08, SD = .36)

Table 5a.

Predictive validity in Zuid-Holland Study: Logistic regression results for any endorsement of violence as dependent variable

Variable Any Violence N (prop) B S.E. Wald Df Sig. Odds Ratio 95% C.I. for Odds Ratio
Lower Upper
Reference 40 (.06) - - 6.862 3 .076 - - -
Class 2 (Defiant) 29 (.13) .580 .276 4.427 1 .035* 1.786 1.041 3.065
Class 3 (Irritable) 21 (.09) .449 .287 2.447 1 .118 1.567 .893 2.749
Class 4 (High Symptoms) 24 (.10) .567 .263 4.648 1 .031* 1.763 1.053 2.951
Sex (lower = male) −1.596 .243 43.274 1 .000* .203 .126 .326
Constant −2.131 .176 145.922 1 .000 .119
Open in a new tab
*

Statistically significant at p < .05

CI = Confidence Interval

Table 5b.

Predictive validity in Zuid-Holland Study: Logistic regression results for any endorsement of mood disorder as dependent variable

Variable Any Mood Disorder N (prop) B S.E. Wald Df Sig. Odds Ratio 95% C.I. for Odds Ratio
Lower Upper
Reference 29 (.04) - - 9.821 3 .020 - - -
Class 2 (Defiant) 13 (.06) .231 .345 .447 1 .504 1.260 .640 2.478
Class 3 (Irritable) 20 (.09) .908 .306 8.785 1 .003* 2.479 1.360 4.518
Class 4 (High Symptoms) 13 (.05) .641 .352 3.327 1 .068 1.899 .953 3.783
Sex (lower = male) 1.576 .315 25.071 1 .000* 4.834 2.609 8.957
Constant −4.283 .331 167.448 1 .000 .014
Open in a new tab
*

Statistically significant at p < .05;

CI = Confidence Interval

For mood disorders, the results were quite different. Being in the Irritable Class in childhood increased the odds of endorsing any mood disorder on interview 14-years later as an adult. In contrast, adults who had been children in the classes characterized by defiance (the Defiance or the High Symptoms Classes) were no more likely to endorse a mood disorder than those in the reference class. Using the High Symptoms Class as the reference class, however, demonstrated that there was only a marginally higher odds of an adult mood disorder in children from the Irritable Class as compared to the High Symptoms Class [OR = 1.97 (CI = .945,4.10), p=.07] in As expected, females were more likely than males to endorse ever having had a mood disorder.

DISCUSSION

Consistent with Burke (2012) and Kuny et al (2012), these findings demonstrate that it is possible to identify discrete classes of oppositional behavior. Consistent with the findings using variable-centered approaches (Stringaris & Goodman, 2009), different dimensions of ODD are predictive of certain types of psychopathology with defiance most associated with disruptive behavior and criminality whereas irritability is most associated with internalizing disorders. For example, the criminality measure was associated with both the Defiant Class and the High Symptom Class. This may mean that the “defiant” component of oppositionality is as strong of a predictor of adult violence as the combination of defiance and irritability. In fact, the odds ratios (OR) of adult criminality (Table 5a) were almost identical for the children in the Defiant Class (OR = 1.79) and the High Symptoms Class (OR = 1.76), despite the fact that children in the Defiant Class were no more likely that children in the Low Symptoms Class to receive a concurrent ODD diagnosis in the American sample. Thus, these children may not have been identified as having a disorder in childhood, yet may grow up to be at a risk for criminality.

Another notable result was the finding that only individuals classified into the Irritable Class were associated with a concurrent anxiety disorder and a mood disorder in adulthood. Interestingly the individuals in the High Symptoms Class (who were high in both irritability and defiance) were not. This result may suggest that the presence of some defiance may be actually protective for mood disorder risk. One possible explanation may be that having one’s own way (i.e., being “willful”) is protective in that it allows for the externalizing of distress. Overall, the differences between classes of oppositionality suggest that they provide unique information about psychopathology risk. Thus, a single approach to treatment may not be appropriate for all children with symptoms of the entity labeled ODD, and this might be separable, yet, from the entity of chronic, non-episodic irritability with temper outbursts now described in the DSM-5 as Disruptive Mood Dysregulation Disorder (DMDD). Because we did not have data on DMDD in this sample, we cannot speak to how many of the children in these classes would have met this new diagnosis.

A strength of this work is the replication in two epidemiological samples from two societies (US and Dutch) of very similar latent classes of oppositional behavior. Although we fit the US clinical sample to a US epidemiological sample, these classes were similar to the classes observed in a clinical sample by Burke (2012) and in 3 separate Dutch samples by Kuny et al (2012) suggesting that the findings are reliable across samples, across measures used (CBCL, Connors’ forms, DSM symptoms), and across societies (US, Dutch, UK). Cross-cultural work using a variable-centered approach has demonstrated similar dimensions of oppositionality in a Brazilian sample (Krieger et al., 2013).

The DSM-IV diagnostic criteria consist of eight items; however, the CBCL ODP Subscale does not include all of the criteria and thus is not a strict measure of DSM-IV ODD. Although the scale used for these analyses assesses oppositional defiant behavior rather than ODD specifically, the CBCL DSM Subscale of ODP does predict ODD diagnosis (Ebesutani et al., 2010). Another limitation is the use of a single informant. This may be problematic because it is unclear whether the results would be different if fathers, teachers, other caregivers, or the children themselves provided information. This is especially important in the cases where defiance at home and at school were rated as different by the mothers. In those cases, having teacher reports of child behaviors may have helped inform whether patterns of behavior related to oppositionality and irritability occurred in multiple domains. Moreover, the adult outcome variables in this study are based on self-report. A multi-informant approach to the assessment of these factors (for example using legal records to verify and assess violence) would likely add methodological rigor, but was not available.

Conclusions

The finding of ODD latent classes are very stable across the VFS and in two Dutch samples [including the work from Kuny et al (2013)], lending credence to the idea that latent class differences do exist. These findings are also complementary to studies which have shown factors of items that cluster in these domains (Stringaris & Goodman, 2009). Moreover, the ability to test both concurrent and predictive validity of the empirically-derived classes is a strength. Overall, these findings indicate that children who present with oppositional behavior represent a heterogeneous group who can be further subtyped into classes of behavior which are related with different adult psychopathologic outcomes. These findings support the change in the DSM-5 that there are separable domains of oppositionality that should be considered in childhood.

Key points.

  • There is separation of irritability and defiant dimensions of oppositionality using a person-centered approach which complements findings that separate these domains using other approaches

  • Children with predominant irritability were unlikely to be diagnosed with ODD, but were at increased odds of adult mood disorders

  • Children with predominant oppositionality were also rarely diagnosed with concurrent ODD but were more likely to have violence in adulthood

  • Diagnosis of ODD in childhood is most common when symptoms from both irritable and defiant domains are present, but elevations in one or the other domain increase risk for adult psychopathology

Acknowledgments

Dr. Althoff receives grant or research support from the National Institute of Mental Health and the Klingenstein Third Generation Foundation. Dr. Verhulst is director at the Department of Child and Adolescent Psychiatry, Erasmus University Medical Center-Sophia Children’s Hospital, which publishes the Dutch translations of the Achenbach System of Empirically Based Assessment and from which he receives remuneration. Dr. Hudziak has received funding from the National Institute of Mental Health and the National Institute of Diabetes and Digestive and Kidney Disease. His primary appointment is with the University of Vermont. He has additional appointments with Erasmus University in Rotterdam, Dartmouth Medical School in Hanover, New Hampshire, and Avera Institute of Human Behavioral Genetics in Sioux Falls, South Dakota. Dr. Kuny-Slock and Mr. van der Ende report no competing interests.

The authors would like to thank Thomas Achenbach, PhD for his generosity in allowing the use of his national probability sample for this work. This work was partially supported by National Institute of Mental Health Grant K08MH082116 (R. Althoff, PI). The NIMH had no role in study design; collection, analysis or interpretation of data; in writing of the report; or in the decision to submit the paper for publication.

Footnotes

Conflict of interest statement: See acknowledgments.

References

  1. Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 Profile. Burlington, VT: University of Vermont Department of Psychiatry; 1991. [Google Scholar]
  2. Achenbach TM, Dumenci L, Rescorla LA. Ten-Year Comparisons of Problems and Competencies for National Samples of Youth: Self, Parent, and Teacher Reports. Journal of Emotional and Behavioral Disorders. 2002;10(4):194–203. [Google Scholar]
  3. Achenbach TM, Rescorla LA. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families; 2001. [Google Scholar]
  4. Aebi M, Muller UC, Asherson P, Banaschewski T, Buitelaar J, Ebstein R, Eisenberg J, Gill M, Manor I, Miranda A, Oades RD, Roeyers H, Rothenberger A, Sergeant J, Sonuga-Barke E, Thompson M, Taylor E, Faraone SV, Steinhausen HC. Predictability of oppositional defiant disorder and symptom dimensions in children and adolescents with ADHD combined type. Psychol Med. 2010;40(12):2089–2100. doi: 10.1017/S0033291710000590. [DOI] [PubMed] [Google Scholar]
  5. Aebi M, Plattner B, Metzke CW, Bessler C, Steinhausen HC. Parent- and self-reported dimensions of oppositionality in youth: construct validity, concurrent validity, and the prediction of criminal outcomes in adulthood. J Child Psychol Psychiatry. 2013;54(9):941–949. doi: 10.1111/jcpp.12039. [DOI] [PubMed] [Google Scholar]
  6. Althoff RR, Copeland WE, Stanger C, Derks EM, Todd RD, Neuman RJ, Van Beijsterveldt T, Boomsma DI, Hudziak JJ. The latent class structure of ADHD is stable across informants. Twin Research and Human Genetics. 2006;9(4):507–522. doi: 10.1375/183242706778025008. [DOI] [PubMed] [Google Scholar]
  7. Althoff RR, Rettew DC, Boomsma DI, Hudziak JJ. Latent class analysis of the Child Behavior Checklist Obsessive-Compulsive Scale. Compr Psychiatry. 2009;50(6):584–592. doi: 10.1016/j.comppsych.2009.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Althoff RR, Rettew DC, Faraone SV, Boomsma DI, Hudziak JJ. Latent class analysis shows strong heritability of the child behavior checklist-juvenile bipolar phenotype. Biol Psychiatry. 2006;60(9):903–911. doi: 10.1016/j.biopsych.2006.02.025. [DOI] [PubMed] [Google Scholar]
  9. Althoff RR, Verhulst FC, Rettew DC, Hudziak JJ, van der Ende J. Adult outcomes of childhood dysregulation: a 14-year follow-up study. J Am Acad Child Adolesc Psychiatry. 2010;49(11):1105–1116. doi: 10.1016/j.jaac.2010.08.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Andrews G, Peters L. The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol. 1998;33(2):80–88. doi: 10.1007/s001270050026. [DOI] [PubMed] [Google Scholar]
  11. Boomsma DI. Twin registers in Europe: An overview. Twin Res. 1998;1(1):34–51. doi: 10.1375/136905298320566465. [DOI] [PubMed] [Google Scholar]
  12. Burke JD. An affective dimension within oppositional defiant disorder symptoms among boys: personality and psychopathology outcomes into early adulthood. J Child Psychol Psychiatry. 2012;53(11):1176–1183. doi: 10.1111/j.1469-7610.2012.02598.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Burke JD, Hipwell AE, Loeber R. Dimensions of oppositional defiant disorder as predictors of depression and conduct disorder in preadolescent girls. J Am Acad Child Adolesc Psychiatry. 2010;49(5):484–492. doi: 10.1097/00004583-201005000-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Burke JD, Rowe R, Boylan K. Functional outcomes of child and adolescent oppositional defiant disorder symptoms in young adult men. J Child Psychol Psychiatry. 2013 doi: 10.1111/jcpp.12150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Conners CK. Conners’ Rating Scales-Revised. New York and Toronto: Multi-Health Systems, Inc; 2001. [Google Scholar]
  16. Copeland WE, Shanahan L, Costello EJ, Angold A. Childhood and adolescent psychiatric disorders as predictors of young adult disorders. Arch Gen Psychiatry. 2009;66(7):764–772. doi: 10.1001/archgenpsychiatry.2009.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dempster A, Laird N, Rubin D. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B. 1977;39:1–38. [Google Scholar]
  18. Dougherty LR, Tolep MR, Smith VC, Rose S. Early exposure to parental depression and parenting: associations with young offspring’s stress physiology and oppositional behavior. J Abnorm Child Psychol. 2013;41(8):1299–1310. doi: 10.1007/s10802-013-9763-7. [DOI] [PubMed] [Google Scholar]
  19. Ebesutani C, Bernstein A, Nakamura BJ, Chorpita BF, Higa-McMillan CK, Weisz JR. Concurrent Validity of the Child Behavior Checklist DSM-Oriented Scales: Correspondence with DSM Diagnoses and Comparison to Syndrome Scales. J Psychopathol Behav Assess. 2010;32(3):373–384. doi: 10.1007/s10862-009-9174-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Elliott DS, Huizinga D. Improving self-reported measures of delinquency. In: Klein MW, editor. Cross-national research in self-reported crime and delinquency. Dordrecht [etc.]: Kluwer Academic Publishers; 1989. pp. 155–186. [Google Scholar]
  21. Essau CA, Wittchen HU. An overview of the Composite International Diagnostic Interview (CIDI) Int J Meth Psychiatric Res. 1993;3:79–85. [Google Scholar]
  22. Ezpeleta L, Granero R, de la Osa N, Penelo E, Domènech JM. Dimensions of oppositional defiant disorder in 3-year-old preschoolers. Journal of Child Psychology and Psychiatry. 2012:no–no. doi: 10.1111/j.1469-7610.2012.02545.x. [DOI] [PubMed] [Google Scholar]
  23. Hipwell AE, Stepp S, Feng X, Burke J, Battista DR, Loeber R, Keenan K. Impact of oppositional defiant disorder dimensions on the temporal ordering of conduct problems and depression across childhood and adolescence in girls. J Child Psychol Psychiatry. 2011;52(10):1099–1108. doi: 10.1111/j.1469-7610.2011.02448.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hofstra MB, Van Der Ende JAN, Verhulst FC. Adolescents’ self-reported problems as predictors of psychopathology in adulthood: 10-year follow-up study. The British Journal of Psychiatry. 2001;179(3):203–209. doi: 10.1192/bjp.179.3.203. [DOI] [PubMed] [Google Scholar]
  25. Hudziak JJ. DSM-IV Checklist for Childhood Disorders. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families; 1998. [Google Scholar]
  26. Hudziak JJ, Copeland W, Stanger C, Wadsworth M. Screening for DSM-IV externalizing disorders with the Child Behavior Checklist: a receiver-operating characteristic analysis. J Child Psychol Psychiatry. 2004;45(7):1299–1307. doi: 10.1111/j.1469-7610.2004.00314.x. [DOI] [PubMed] [Google Scholar]
  27. Ivanova MY, Dobrean A, Dopfner M, Erol N, Fombonne E, Fonseca AC, Frigerio A, Grietens H, Hannesdottir H, Kanbayashi Y, Lambert M, Achenbach TM, Larsson B, Leung P, Liu X, Minaei A, Mulatu MS, Novik TS, Oh KJ, Roussos A, Sawyer M, Simsek Z, Dumenci L, Steinhausen HC, Metzke CW, Wolanczyk T, Yang HJ, Zilber N, Zukauskiene R, Verhulst FC, Rescorla LA, Almqvist F, Weintraub S, Bilenberg N, Bird H, Chen WJ. Testing the 8-syndrome structure of the child behavior checklist in 30 societies. J Clin Child Adolesc Psychol. 2007;36(3):405–417. doi: 10.1080/15374410701444363. [DOI] [PubMed] [Google Scholar]
  28. Junger-Tas J, Terlouw GJ, Klein MW, editors. Delinquent behavior among young people in the Western world: first results of the International Self-Report Delinquency Study. Amsterdam: Kugler; 1994. [Google Scholar]
  29. Kolko DJ, Pardini DA. ODD dimensions, ADHD, and callous-unemotional traits as predictors of treatment response in children with disruptive behavior disorders. J Abnorm Psychol. 2010;119(4):713–725. doi: 10.1037/a0020910. [DOI] [PubMed] [Google Scholar]
  30. Krieger FV, Polanczyk VG, Robert G, Rohde LA, Graeff-Martins AS, Salum G, Gadelha A, Pan P, Stahl D, Stringaris A. Dimensions of oppositionality in a Brazilian community sample: testing the DSM-5 proposal and etiological links. J Am Acad Child Adolesc Psychiatry. 2013;52(4):389–400. e381. doi: 10.1016/j.jaac.2013.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kuny AV, Althoff RR, Copeland W, Bartels M, Van Beijsterveldt CE, Baer J, Hudziak JJ. Separating the domains of oppositional behavior: comparing latent models of the conners’ oppositional subscale. J Am Acad Child Adolesc Psychiatry. 2013;52(2):172–183. e178. doi: 10.1016/j.jaac.2012.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Loeber R, Burke JD, Lahey BB, Winters A, Zera M. Oppositional defiant and conduct disorder: a review of the past 10 years, part I. J Am Acad Child Adolesc Psychiatry. 2000;39(12):1468–1484. doi: 10.1097/00004583-200012000-00007. [DOI] [PubMed] [Google Scholar]
  33. Loeber R, Burke JD, Pardini DA. Development and etiology of disruptive and delinquent behavior. Annu Rev Clin Psychol. 2009;5:291–310. doi: 10.1146/annurev.clinpsy.032408.153631. [DOI] [PubMed] [Google Scholar]
  34. Mandy W, Roughan L, Skuse D. Three Dimensions of Oppositionality in Autism Spectrum Disorder. J Abnorm Child Psychol. 2013 doi: 10.1007/s10802-013-9778-0. [DOI] [PubMed] [Google Scholar]
  35. Muthén LK, Muthén BO. Mplus User’s Guide. 5. Los Angeles, CA: Muthén & Muthén; 2007. [Google Scholar]
  36. Nestadt G, Addington A, Samuels J, Liang KY, Bienvenu OJ, Riddle M, Grados M, Hoehn-Saric R, Cullen B. The identification of OCD-related subgroups based on comorbidity. Biol Psychiatry. 2003;53(10):914–920. doi: 10.1016/s0006-3223(02)01677-3. [DOI] [PubMed] [Google Scholar]
  37. Neuman RJ, Heath A, Reich W, Bucholz KK, Madden PAF, Sun L, Todd RD, Hudziak JJ. Latent class analysis of ADHD and comorbid symptoms in a population sample of adolescent female twins. J Child Psychol Psychiatry. 2001;42(7):933–942. doi: 10.1111/1469-7610.00789. [DOI] [PubMed] [Google Scholar]
  38. Rindskopf D. Heavy alcohol use in the “fighting back” survey sample: Separating individual and community level influences using multilevel latent class analysis. Journal of Drug Issue Special Issue: The “Fighting Back” program. 2006;36:441–462. [Google Scholar]
  39. Rowe R, Costello EJ, Angold A, Copeland WE, Maughan B. Developmental pathways in oppositional defiant disorder and conduct disorder. J Abnorm Psychol. 2010;119(4):726–738. doi: 10.1037/a0020798. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Steiner H, Saxena K, Chang K. Psychopharmacologic strategies for the treatment of aggression in juveniles. CNS Spectr. 2003;8(4):298–308. doi: 10.1017/s1092852900018514. [DOI] [PubMed] [Google Scholar]
  41. Stringaris A, Cohen P, Pine DS, Leibenluft E. Adult outcomes of youth irritability: a 20-year prospective community-based study. Am J Psychiatry. 2009;166(9):1048–1054. doi: 10.1176/appi.ajp.2009.08121849. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Stringaris A, Goodman R. Longitudinal outcome of youth oppositionality: irritable, headstrong, and hurtful behaviors have distinctive predictions. J Am Acad Child Adolesc Psychiatry. 2009;48(4):404–412. doi: 10.1097/CHI.0b013e3181984f30. [DOI] [PubMed] [Google Scholar]
  43. Stringaris A, Zavos H, Leibenluft E, Maughan B, Eley TC. Adolescent irritability: phenotypic associations and genetic links with depressed mood. Am J Psychiatry. 2012;169(1):47–54. doi: 10.1176/appi.ajp.2011.10101549. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Todd RD, Lobos EA, Sun LW, Neuman RJ. Mutational analysis of the nicotinic acetylcholine receptor alpha 4 subunit gene in attention deficit/hyperactivity disorder: evidence for association of an intronic polymorphism with attention problems. Mol Psychiatry. 2003;8(1):103–108. doi: 10.1038/sj.mp.4001257. [DOI] [PubMed] [Google Scholar]
  45. Verhulst FC, Akkerhuis GW, Althaus M. Mental health in Dutch children: (I). A cross-cultural comparison. Acta Psychiatr Scand Suppl. 1985;323:1–108. doi: 10.1111/j.1600-0447.1985.tb10512.x. [DOI] [PubMed] [Google Scholar]
  46. Verhulst FC, van der Ende J, Koot HM. Handleiding voor de CBCL/4-18. Rotterdam: Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam/Erasmus Universiteit Rotterdam, Afdeling Kinder-en jeugdpsychiatrie; 1996. [Google Scholar]
  47. Vermunt JK, Magidson J. Latent GOLD User’s Guide. Belmont, MA: Statistical Innovations Inc; 2000. [Google Scholar]
  48. Wakschlag LS, Henry DB, Tolan PH, Carter AS, Burns JL, Briggs-Gowan MJ. Putting theory to the test: modeling a multidimensional, developmentally-based approach to preschool disruptive behavior. J Am Acad Child Adolesc Psychiatry. 2012;51(6):593–604. e594. doi: 10.1016/j.jaac.2012.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Whelan YM, Stringaris A, Maughan B, Barker ED. Developmental continuity of oppositional defiant disorder subdimensions at ages 8, 10, and 13 years and their distinct psychiatric outcomes at age 16 years. J Am Acad Child Adolesc Psychiatry. 2013;52(9):961–969. doi: 10.1016/j.jaac.2013.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wittchen HU, Kessler RC, Ustun TB. Properties of the Composite International Diagnostic Interview (CIDI) for measuring mental health outcome. In: Thornicroft G, Tansella M, editors. Mental Health Outcome Measures, Second Edition. London: Gaskell; 2001. pp. 212–227. [Google Scholar]

RESOURCES