Skip to main content

Advertisement

SpringerLink
Log in

Denoting treatment outcome in child and adolescent psychiatry: a comparison of continuous and categorical outcomes

  • Original Contribution
  • Published:
European Child & Adolescent Psychiatry Aims and scope Submit manuscript

Abstract

Various approaches have been proposed to denote treatment outcome, such as the effect size of the pre-to-posttest change, percentage improvement, statistically reliable change, and clinical significant change. The aim of the study is to compare these approaches and evaluate their aptitude to differentiate among child and adolescent mental healthcare providers regarding their treatment outcome. Comparing outcomes according to continuous and categorical outcome indicators using real-life data of seven mental healthcare providers, three using the Child Behavior Checklist and four using the Strengths and Difficulties Questionnaire as primary outcome measure. Within each dataset consistent differences were found between providers and the various methods led to comparable rankings of providers. Statistical considerations designate continuous outcomes as the optimal choice. Change scores have more statistical power and allow for a ranking of providers at first glance. Expressing providers’ performance in proportions of recovered, changed, unchanged, or deteriorated patients has supplementary value, as it denotes outcome in a manner more easily interpreted and appreciated by clinicians, managerial staff, and, last but not least, by patients or their parents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Achenbach TM (1991) Manual for the child behavior checklist 4–18 and 1991 profiles. Department of Psychiatry, University of Vermont, Burlington

    Google Scholar 

  2. American Psychiatric Association (1994) Diagnostic and statistical manual of mental disorders IV. Author, Washington, DC

    Google Scholar 

  3. Bauer S, Lambert MJ, Nielsen SL (2004) Clinical significance methods: a comparison of statistical techniques. J Pers Assess 82:60–70

    Article  PubMed  Google Scholar 

  4. Berwick DM, Nolan TW, Whittington J (2008) The triple aim: care, health, and cost. Health Aff 27:759–769

    Article  Google Scholar 

  5. Bickman L, Kelley SD, Breda C, de Andrade AR, Riemer M (2011) Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatr Serv 62:1423–1429

    Article  PubMed  Google Scholar 

  6. Carlier IV, Meuldijk D, van Vliet IM, van Fenema E, van der Wee NJ, Zitman FG (2012) Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. J Eval Clin Pract 18:104–110

    Article  PubMed  Google Scholar 

  7. Cohen J (1988) Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, Hillsdale

    Google Scholar 

  8. de Beurs E, den Hollander-Gijsman ME, van Rood YR, van der Wee NJ, Giltay EJ, van Noorden MS, van der Lem R, van Fenema E, Zitman FG (2011) Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother 18:1–12

    Article  PubMed  Google Scholar 

  9. Fedorov V, Mannino F, Zhang R (2009) Consequences of dichotomization. Pharm Stat 8:50–61

    Article  PubMed  Google Scholar 

  10. Fleiss JL (1973) Statistical methods for rates and proportions. Wiley, New York

    Google Scholar 

  11. Goodman R (2001) Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatr 40:1337–1345

    Article  CAS  Google Scholar 

  12. Goodman R, Scott S (1999) Comparing the strengths and difficulties questionnaire and the child behavior checklist: is small beautiful? J Abnorm Child Psychol 27:17–24

    Article  CAS  PubMed  Google Scholar 

  13. Hall CL, Moldavsky M, Baldwin L, Marriot M, Newell K, Taylor J, Hollis C (2013) The use of routine outcome measures in two child and adolescent mental health services: a completed audit cycle. BMC Psychiatr 13:270

    Article  Google Scholar 

  14. Hermann RC, Rollins CK, Chan JA (2007) Risk-adjusting outcomes of mental health and substance-related care: a review of the literature. Harv Rev Psychiatr 15:52–69

    Article  Google Scholar 

  15. Hiller W, Schindler AC, Lambert MJ (2012) Defining response and remission in psychotherapy research: a comparison of the RCI and the method of percent improvement. Psychother Res 22:1–11

    Article  PubMed  Google Scholar 

  16. Hudziak JJ, Achenbach TM, Althoff RR, Pine DS (2007) A dimensional approach to developmental psychopathology. Int J Methods Psychiatr Res 16:S16–S23

    Article  PubMed  Google Scholar 

  17. Jacobson NS, Follette WC, Revenstorf D (1984) Psychotherapy outcome research: methods for reporting variability and evaluating clinical significance. Behav Ther 15:336–352

    Article  Google Scholar 

  18. Jacobson NS, Truax P (1991) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consul Clin Psychol 59:12–19

    Article  CAS  Google Scholar 

  19. Jaeschke R, Singer J, Guyatt GH (1989) Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials 10(4):407–415

    Article  CAS  PubMed  Google Scholar 

  20. Kelley SD, Bickman L (2009) Beyond routine outcome monitoring: measurement feedback systems (MFS) in child and adolescent clinical practice. Cur Opin Psychiatr 22:363–368

    Article  Google Scholar 

  21. Klugh HE (2006) Normalized T Scores. In: Kotz S, Read CB, Balakrishnan N, Vidakovic B (eds) Encyclopedia of statistical sciences, 2nd edn. Wiley, New York

    Google Scholar 

  22. Knaup C, Koesters M, Schoefer D, Becker T, Puschner B (2009) Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis. Br J Psychiatry 195:15–22

    Article  PubMed  Google Scholar 

  23. Kraemer HC, Morgan GA, Leech NL, Gliner JA, Vaske JJ, Harmon RJ (2003) Measures of clinical significance. J Am Ac Child Adolesc Psychiatr 42:1524–1529

    Article  Google Scholar 

  24. Lambert M (2007) Presidential address: what we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psych Res 17:1–14

    Google Scholar 

  25. Markon KE, Chmielewski M, Miller CJ (2011) The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psych Bul 137:856–879

    Article  Google Scholar 

  26. McGlinchey JB, Atkins DC, Jacobson NS (2002) Clinical significance methods: which one to use and how useful are they? Behav Ther 33:529–550

    Article  Google Scholar 

  27. Muris P, Meesters C, van den Berg F (2003) The strengths and difficulties questionnaire (SDQ). Eur Child Adolesc Psychiatr 12:1–8

    Article  Google Scholar 

  28. Norman GR, Sloan JA, Wyrwich KW (2003) Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 41:582–592

    PubMed  Google Scholar 

  29. Porter ME, Teisberg EO (2006) Redefining healthcare: creating value-based competition on results. Harvard Business Press, Cambridge

    Google Scholar 

  30. Russel M (2000) Summarizing change in test scores: shortcomings of three common methods. Practical Assessment, Research & Evaluation 7

  31. Sloan JA, Cella D, Hays RD (2005) Clinical significance of patient-reported questionnaire data: another step toward consensus. J Clin Epidem 58:1217–1219

    Article  Google Scholar 

  32. van Widenfelt B, Goedhart A, Treffers P, Goodman R (2003) Dutch version of the strengths and difficulties questionnaire (SDQ). Eur Child Adolesc Psychiatr 12:281–289

    Article  Google Scholar 

  33. Verhulst FC, Van der Ende J, Koot HM (1996) Handleiding voor de CBCL/4-18 [Dutch manual for the CBCL/4-18]. Department of Child and Adolescent Psychiatry, Erasmus University, Sophia Children’s Hospital, Rotterdam

  34. Walton K, Ormel J, Krueger R (2011) The dimensional nature of externalizing behaviors in adolescence: evidence from a direct comparison of categorical, dimensional, and hybrid models. J Abnorm Child Psychol 39:553–561

    Article  PubMed  Google Scholar 

  35. Wilder J (1965) Pitfalls in the methodology of the law of initial value. Am J Psychother 19:577–584

    CAS  PubMed  Google Scholar 

  36. Wise EA (2004) Methods for analyzing psychotherapy outcomes: a review of clinical significance, reliable change, and recommendations for future directions. J Pers Assess 82:50–59

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors are grateful to the following mental health providers for allowing us to use their outcome data: De Viersprong, Mentaal Beter, Mutsaersstichting, OCNR, Praktijk Buitenpost, Yorneo, and Yulius.

Conflict of interest

No conflicts declared.

Author information

Authors and Affiliations

  1. Stichting Benchmark GGZ, Rembrandtlaan 46, 3723 BK, Bilthoven, The Netherlands

    Edwin de Beurs & Marko Barendregt

  2. Department of Clinical Psychology, Leiden University, Wassenaarseweg 52, 2333 AK, Leiden, The Netherlands

    Bente Rogmans

  3. Yulius Academy, Mathenesserlaan 202, 3014 HH, Rotterdam, The Netherlands

    Sylvana Robbers

  4. De Viersprong, De Beeklaan 2, 4661 EP, Halsteren, The Netherlands

    Marieke van Geffen

  5. PIONN, Papenvoort 21, 9447 TT, Papenvoort, The Netherlands

    Marleen van Aggelen-Gerrits

  6. Mentaal Beter, Soestdijkerweg 17, 3734 MG, Den Dolder, The Netherlands

    Huub Houben

Authors
  1. Edwin de Beurs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marko Barendregt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bente Rogmans
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sylvana Robbers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marieke van Geffen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marleen van Aggelen-Gerrits
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Huub Houben
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edwin de Beurs.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Beurs, E., Barendregt, M., Rogmans, B. et al. Denoting treatment outcome in child and adolescent psychiatry: a comparison of continuous and categorical outcomes. Eur Child Adolesc Psychiatry 24, 553–563 (2015). https://doi.org/10.1007/s00787-014-0609-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00787-014-0609-9

Keywords

Search

Navigation