FAI 25/26-1818

Research paper OFFICIAL For Internal Use Only

Contents

• Evaluation of CANS, ABAS-3 and LSP-16 outcome measures ………………………………………….. 1
  • Contents …………………………………………………………………………………………………. 2
  • Summary ………………………………………………………………………………………………… 2 Care and Needs Scale ……………………………………………………………………………….. 3 Adaptive Behavior Assessment System, 3rd Edition ………………………………………. 3 Abbreviated Life Skills Profile (LSP-16) ………………………………………………….. 4 Summary of outcome measure features ……………………………………………….. 6 References ………………………………………………………………………………… 9

Summary

The document was released under the Freedom of Information Act 1982 by the National Disability Insurance Agency.

This paper examines the scope, psychometric properties and other features of three commonly used outcome measures: Care and Needs Scale (CANS), Adaptive Behavior Assessment System, 3rd Edition (ABAS-3) and Abbreviated Life Skills Profile (LSP-16).

The outcome measures vary from narrow to general in scope. CANS is intended to assess support needs for people over 16 years with moderate to severe traumatic brain injury. LSP-16 is designed for adults with severe or chronic mental health conditions. ABAS-3 is more general and developers suggest it can be used to assess adaptive behaviours for anyone under 89 years.

None of the three outcome measures are intended to be a standalone tool. It is intended that all three are used in combination with other measures, assessments and information gathering methods to generate a fuller picture of a person’s functional capacity or support needs.

The source of the information used to completed the assessments varies. ABAS-3 can be completed by parents, teachers, co-workers, friends or clinicians familiar with the client and it is recommended that information is collected from multiple sources. LSP-16 is usually completed by a clinician but preference should be given to the treating professional or support person with the greatest understanding of the client’s situation. CANS is completed by a clinician but familiarity may be gained through an informal interview with the client or their carer/proxy, or by sufficiently detailed medical records.

Results are further summarised in 6. Summary of outcome measure features.

FOI/25/26-1818

OFFICIAL For Internal Use Only

3. Care and Needs Scale

The CANS was developed to assess support needs for people over 16 years with moderate to severe traumatic brain injury (TBI) (Honan et al, 2019; Tate, 2017; Soo et al, 2007). A version for younger people (PCANS) was also developed (Tate et al, 2014; Soo et al, 2010). CANS can be completed in an interview format with the client or proxy or by a clinician with sufficient knowledge of the client (Tate, 2017). The manual also notes:

the CANS can be completed on the basis of information derived from the patient’s medical record, scales of disability and so forth. In situations where the clinician has knowledge of the patient/client and direct interview is not required, the CANS will only take a few minutes to complete. Interview format with an informant generally takes somewhat longer (10–15 mins).

Research paper

OFFICIAL For Internal Use Only

forms reliability. There is evidence of excellent content, construct and criterion validity (Hayden-Evans et al, 2022; Harrison & Oakland, 2015).

Validity studies targeted at specific populations were conducted for autism, intellectual disability, and ADHD. In addition, validity studies were conducted for the second edition (ABAS-II) for people with:

developmental delay, low birth weight, perinatal respiratory distress, chromosomal abnormalities, fetal alcohol syndrome and prenatal drug exposure, Down syndrome, motor and physical disorders, expressive and receptive language disorders, behavioural and emotional issues, learning disabilities, and hearing impairments; adults with Alzheimer’s and unspecified neuro-psychological disorders (Harrison & Oakland, 2015, p.127).

The developers argue that ABAS-II is sufficiently similar to ABAS-3 for the previous version’s evidence to stand in favour of the current version (Harrison & Oakland, 2015). However, there are some notable differences. For example, ABAS-3 scores are generally higher than ABAS-II scores (von Buttlar et al, 2021; Harrison & Oakland, 2015).

Some limitations were described in the literature. Despite evidence of good psychometric properties, Hayden-Evans et al (2022) note that ABAS-3 does not have very good coverage against the International Classification of Functioning, Disability and Health (ICF) codes deemed most relevant to children with autism. Further, while efforts were made to ensure ABAS-3 was comprehensive, it should not be relied on as the sole instrument of assessment.

Clinicians should also look to other data such as “information derived from concurrent or former assessments; detailed interviews and history taking; developmental, school, or work records; and direct observations” (Harrison & Oakland, 2015, p.7).

5. Abbreviated Life Skills Profile

lsp-16 is a measure of community functioning and disability for people with severe or chronic mental health conditions (Little, 2013; Kightley et al, 2010; Pirkis et al, 2005a; Rosen et al, 2001). It was developed for Australian public mental health services to reduce the rating burden on clinicians (NMHIDEAP, 2013; Little, 2013; Pirkis et al, 2005a). As part of the National Outcome Casemix Collection (NOCC), LSP-16 is now required to be used at certain points in the treatment cycle for adults receiving specialised public sector mental health services across Australia (AMHOCN, 2021a; Little, 2013; Rosen et al, 2001).

it is a shortened form of the 39 item life skills profile (lsp-39). rosen et al (1989) developed the original lsp-39 to assess the daily functioning of people with schizophrenia and it has since been applied generally for people with mental health or psychiatric conditions (Burgesset al, 2017; Deady et al, 2005; Pirkis et al, 2005a). The developers note that only a few of the items in the Communication subscale of LSP-39 related directly to features specific to schizophrenia (Rosen et al, 1989). The communication subscale was removed in the development of LSP-16 (Deady et al, 2005; Rosen et al, 2001).

Evaluation of outcome measures

page 4 of 12 page 4 of 100

Research Paper

OFFICIAL For Internal Use Only

A few studies have investigated the psychometric properties of LSP-16. There is equivocal evidence of concurrent and predictive validity. It was shown to correlate with Health of the Nation Outcome Scale and LSP-39 but not with the Behaviour and Symptom Identification Scale (Burgess et al, 2017). There is some evidence that LSP-16 can predict clinical outcomes such as hospital admission and length of stay, though other studies were not able to find significant correlations (Parker et al, 2020; Burgess et al, 2017; Deady, 2009). There is evidence of poor construct validity (Little, 2013). Studies have found moderate to good interrater reliability and testretest reliability (Burgess et al, 2017). Some studies suggest potential problems for LSP-16’s sensitivity to change but no study has investigated this directly (Sammels et al, 2022; NMHIDEAP, 2013).

More research has been conducted on the psychometric properties of LSP-39. The longer version has been shown to be a valid and reliable measure for people with schizophrenia and severe mental health issues. There is evidence that LSP-39 has moderately good content, construct, concurrent and predictive validity, adequate interrater reliability, high testretest reliability and good sensitivity to change (Burgess et al, 2017; Deady, 2009; Pirkis et al, 2005a).

Some argue that evidence for LSP-39 can be used to support the validity and reliability of LSP16 as all 16 items of the abbreviated form are included in the longer version (Pirkis et al, 2005a; Rosen et al, 2001) And LSP-16 has been shown to correlate with LSP-39 (Burgess et al, 2017; Rosen et al, 2001). However, there are some important differences between the two forms. For example, LSP-39 is a strengthsbased scale with higher scores indicating greater functioning in a particular task, whereas LSP-16 is an impairment based scale with higher scores indicating greater impairment (Pirkis et al, 2005a; Rosen et al, 2001).

Several limitations of LSP-16 have been identified. A review of the NOOC in 2013 recommended removing the LSP-16 from the collection due to its reported limitations. Despite the measure being mandatory, the 3month period between reviews meant that it was not administered to most service users who are in community rehabilitation settings for less than 3 months. While its use in capturing some information around daily living skills in adults was seen as useful, it was found to be inappropriate for children and adolescents, older people and those in a forensic setting. In addition:

Issues were noted in relation to particular items, including domains that are not captured, the glossary and the language of the measure. Participants consistently raised concerns regarding items 10, 11 and 16 which they thought required clarification in the glossary. Some participants suggested that the tool does not capture fluctuations in functioning between reviews, which they thought was of particular clinical relevance. The language was felt to be outdated, not strengthsbased and not supporting the recovery agenda… Participants suggested that there were more useful types of information to collect, including capturing aspects of social inclusion (NMHIDEAP, 2013, p.130).

Evaluation of outcome measures Page 5 of 12 Page 5 of 100

Summary of outcome features

The following table summarises key information about each assessment tool.

Tool	Population	Psychometric properties	Benefits and limitations	Used as a standalone tool	Rater
ABS-3	Originally designed for people with intellectual and developmental disabilities. Developers now suggest it can be used in anyone under 9 years (Darcy et al., 2015). May not be the most appropriate instrument for children younger than 6 (Wearne et al., 2020; Honan et al., 2019)
Evidence of adequate to excellent validity and internal consistency. Standard error of measurement, test–retest reliability, interrater reliability, cross forms equivalence and alternative-forms reliability (Tamm et al.; 2022; Hayden‐Evans et al., 2022; von Buttlar et al., Hansen et al., 2019; Harrison & Oakland, 2015).
Can incorporate multiple sources of information (‟Harrison & Oakland, 2015;”) May be particularly useful in assessing activity and performance in ADHD and early neurodevelopmental conditions (Darcy et al., 2022).
Standardised on a large sample. However, it is an entirely US based sample that skews to higher socioeconomic status and educational achievement.
M ay require adjustment to achieve across cultural validity (Proctorpak & Kireenko, 2020; Emmam et al., 2020).
Should be used as one in a battery of measures when different information gathering tools are needed (Darcy et al., 2022; Harrison & Oakland, 2015).
Can be used as a self report tool or by parents teachers clinicians or anyone familiar with the client
CANS	Adults over 16 years with moderate–severe traumatic brain injury (Hunter, Tazei, Holt Jr., 2019; Toriello Tate, 2021)
Excellent interrater and test-retest reliability and convergent validity criterion evidence (Holt Jr. et al.; 2018; Hunter et al., 2019; Toriello Tate, 2021).
Provides information on support needs (‟Toriello Tate,” 2021) Recommended as a need assessment tool if functionality cannot be assessed through other means such as clinical interview Client should also have opportunity for input into results
Can be completed by clinician with thorough knowledge of the patient including format which is an interactive prompt.

FOI 25/26-1818 Research paper OFFICIAL For Internal Use Only

The Population Psychometric properties Benefits and limitations Used as a standalone Rater tool LSP-16 Adults with severe or chronic mental health conditions (Rosen et al., 1999; Rosen et al., 2001). May not be appropriate for older people (>65 years), children/adolescents or patients in forensic setting (AMHOCN, 2021a–b; MNHDEAP, 2013). Relies on clinical subjective judgment which can affect reliability. Training required for rater to achieve high interrater reliability. A brief and generally considered easy to use and understand within minimal training (AMHOCN, 2021a–b; Rosen et al., 1999) Due to limitations, AMHOCN (2021a) suggest LSP-16 should only be used as part of more comprehensive assessment. (Hunter, 2021; Honan et al., 2019); medical records (Tate, 2017); Suitable for use by clinicians from different disciplines (Soo et al. , 2007). Rater must be familiar with the client’s sample’s elicits (Sammels et al. 2004 ; Kightley et al. 2010 ; Eagar et al. 2008 ) Some equivocal evidence for concurrent and predictive validity (Parker et al., 2020; Burgess & Deadl 2017; Deady, 2009). single study looking at construct validity found poor performance (Little , 2013). Moderate to good internal consistency and retest reliabilit y(Burgess et al., 2017). equivocality in how it is administered - can be done independently or alongside other tools (AMHOCN, 2021 a–b; MNHDEAP, 2013 ). Parker et al.(2015) suggest that LSP-16 is useful in predicting clinical outcomes (Parker et al. 2020; Burgess & Deadly, 2009) Parker et al. (2015); Little, 2013; Burgess et al. 2005; Eager pirks et al , 2005a: Egar et al .2000); Suitable as an adjunct tool . A number of state carers and family members can administer the test due to its ease of use (Burges s et al ., 2017 ; D eadyl, 2009 ) The assessment may also be conducted by allied health professionals such as: disability support workers psychologists psychiatrists occupational therapists social worker

Outcome Measures

The National Disability Insurance Agency

FOI/25/26-1818 Research paper OFFICIAL For Internal Use Only

Tool	Population	Psychometric properties	Benefits and limitations	Used as a standalone tool	Rater
(Burgess et al., 2017; Purkiss et al., 2003a; Rosen et al., 2001).	lost in LSP-16 which may be due language that is out dated and not strengths based (Burgess et al., 2017; NMHIDEAP, 2013) .	Minimal evidence of validity and reliability in adolescents with early onset schizophrenia (Puig et al., 2013).

Page 8 of 12 OFFICIAL Page 8 of 100

References

Australian Mental Health Outcomes and Classification Network.

• National Outcomes and Casemix Collection (NOCC) basic training manual: adult services ” Edition [PDF] Australian Mental Health Outcomes and Classification Network.
• National Outcomes and Casemix Collection (NOCC) basic training manual: child and adolescent services [PDF] Australian Mental Health Outcomes and Classification Network.
• Rater and clinical utility training manual: Adult Rev ed Sydney: AMHOCN [PDF] Australian Mental Health Outcomes and Classification Network.
• Rater and clinical utility training manual: Child and Adolescent Rev Ed Sydney: AMHOCN [PDF] P.M., Burgess; M.G, Harris ; T Coombs & J.E Pirkis(2017). A systematic review clinician-rated instruments to assess adults’ levels functioning in specialised public sector mental health services The Australian New Zealand journal psychiatry514338-354 https://doi.org/10 . p.P Burgesses P ,Pirkis J &Coombes T 2015 Routine outcome measurement Australia International Review Psychiatry.https// doi org /10pM.BurgessT.CoombsA ClarkeR DicksonJ& Pirkis J 2012 Achievements mental Australian Mental Health outcomes classification network AMHOCNInternational Journal of Mental Health Systems614 https //doiorg/ dArcy E Wallace K Chamberlain A EvansK Milbourn B Bolte S Whitehouse AJ GirdlerS (2022) Content validation common measures function young children against the international classifying Functioning Disability Health Code Core Sets relevant neurodevelopmental conditions Autism :the international journal research practice ”928“939.

OFFICIAL

For Internal Use Only

• Deady, M. (2009). A review of screening, assessmentandoutcome measures for drug**alcohol settings*. Drug And Alcohol And Mental Health Information Management Project. Sydney : NSW Department Of Health.https://www.drugsandalcohol.ie/18266/1/NADA A Review%2C Assessment_and_Outcome_Measures_for_Drug_and_Alcohol_Settings.pdf

Eagar K., Buckingham B., Coombs T., Trauer T., Graham C, Eagar L & Callaly T.(2000). Outcome Measurement in Adult Area Mental Health Services: Implementation Resource Manual.DepartmentOf HumanServices Victoria.https://www.vgls.vic.gov.au/client/en AU/search/asset/1160101/0 Emam,M.M..Al-Sulaimani,H.Omara,E.& Al-Nabhany,R. ( 2019).Assessment of adaptive behaviourin children with intellectual disability In Oman an examinationof ABAS-3 factor structure and validationIn the Arab context.International journal Of developmental disabilities , 6*(4),* 37–32. [http(s)://doi.org/10 .org /10 .*80/20473869.2019.587939 ](https://doi.org/10.1080/20473869.2019.1587939 Hansen,L,(2019)A Concurrent validity studythe Missouri Adaptive Ability ScaleandTheAdaptive Behavior Assessment System,Third Edition—Teacher FormMurray State Theses And Dissertations. https: //digitalcommons.murraystate.edu/etd/149 Harrison,P.L.,& Oakland,T.(2015).ABAS - Torrance Western Psychological Services.Hayden-Evans,M.Milbourn,B.D’Arcy,E.Chamberlain,A.Afsharnejad.B.Evans,K… & Girdler,S ( 2022 ) An evaluation of overall utility measuresof functioning suitable for school-aged children on the autism spectrum A scoping review.International Journal Of Environmental Research and Public Health , 1(21), 1114.Honan,C .A McDonald S Tate R Ownsworth T.Togher L Fleming J Anderson V Morgan A Catroppa C Douglas J Francis H WearneT.SigmundsdottirL PonsfordJ(2019Outcome instruments in moderate-to-severe adult traumatic brain injury recommendationsfor usein psychosocial research Neuropsychological rehabilitation,2(6)896–-916.[http(s)://doi.org /org/10 .80/09602011.2017.339616 ] Hunter, S,(2021Occupational Therapy Australia response to question takenon notice at public hearing Melbourne April OccupationalTherapyAustralia. [https://everyaustraliancounts.com.au/wp-content/uploads/QoN_03_IA.pdf](https://everyaustraliancounts.com.au/wp-content/uploads/QoN _IA.pdf) Kightley,M.,Einfeld,S.& Hancock,N.( 2010 ) Routine outcome measurement mental healthfeasibility for examining effectiveness of an NGO.Australasian psychiatry : bulletin Royal Australian New Zealand College Psychiatrists , 1(2), 17 –**1***9. [http: //doi org/10 .3109/10398560903473660Leifker,F.R Patterson,T.L Heaton,R.K Harvey,P.D(2011).Validating measures real-worldoutcome results VALERO expert survey RAND panel.Schizophreniabulletin,3(2)334–-343.[http(s)://doi.org /org/10 .*1093/schbul/spb044 ]

Outcome Measures PageofPageOf100

OFFICIAL

For Internal Use Only

Little J. (2013). Multilevel confirmatory ordinal factor analysis of the Life Skills Profile-16. Psychological assessment, 25(3, 810-825).* https://doi.org/10.1037/a0032574 National Mental Health Information Development Expert Advisory Panel. (2013). Mental Health National Outcomes and Casemix Collection: NOCC Strategic Directions 2014 – 2024.* Commonwealth of Australia,* Canberra*. https://www.amhocn.org/training-and-service-development/special-projects/nocc-strategic-directions-2014-2024 Parker, S., Arnautovska, U., Siskind, D., Dark, F., McKeon, G., Korman, N., & Harris, M.(2020). Community-care unit model of residential mental health rehabilitation services in Queensland,Australia: Predicting outcomes of consumers 1-year post discharge.Epidemiologyand Psychiatric Sciences,29 E109.[https://doi.org/10.1017/S2045796020000207]* pirkis,J.Burgess,P.Kirk.P.Dodson,S.&Coombs,T..(2005a).Reviewof standardised measures usedin theNationalOutcomesCasemixCollection(NOCC).New South Wales InstituteOf Psychiatryhttp://hdl.handle.net/10536/DRO/DU:30073558prikis.J,BurgessP,CoombsTClarkeAJonesEllisd&DicksonR.(2005b).Routine measurementofof outcomesth Australia’s public sectormentalhealthservices.Australia and New Zealand Health policy, 2(1, 8. https://doi.org/10.1186/1743-8462-2-8 Prokopiak, A., & Kirenko J.(2020). ABAS - 3 – an instrument for assessing adaptive skills in people withan intellectual disability.Hrvatska revija za rehabilitacijska istraživanja,[56],** (**)2***), [1][54]-168.[[https://doi.org/10.31299/hrri.56.2.9]PuigO,Penadés,R.Baeza,I.De la Serna,E.Sánchez-Gistau,V.Lázaro,L..Bernardo,M.&Castro-Fornieles,J.(2013 Assessment of real-world daily-living skillsin early-onset schizophrenia trough the Life Skills Profile scale.Schizophrenia research,, **]145(1-[3),[95–100]. [https://doi.org/10.1016/j.schres.2012.12.026]Rosen,A.Hadzi-Pavlovic,D,&Parker,G.(1989).The life skills profile: a measureassessing function anddisabilityinschizophrenia. Schizophreniabulletin,* 15(, 325-—37*. https://doi.org/10.1093/schbul/15.2.325 Rosen, A., Trauer,T,HadzipavlovicD& Parker G.(2001). Developmentofa brief formofoftheLifeSkillsProfilethelSP-20.The Australianand New Zealand journal ofpsychiatry,[3][5]()5***), [677]-683.[[https://doi.org/10.1080/0004867010060518]Sammells,E.LoganA.&SheppardL.(2023 Participant OutcomesFacilitator ExperiencesFollowingacommunityLiving SkillsProgramforAdultMentalHealth Consumers.Community mental healthjournal**]59*(3**) , 428-–438*https://doi.org/10.1007/s10597-022-01020-x)So,C.Tate,R.L.Anderson,V,&Waugh,M.-C.* ( 2010 ). Assessing care and support needs for children with acquired brain injury: Normative data fo the Paediatric Care Needs Scale(PCANS).* Brain Impairment,* 11(**, [183]-[196]. [https://doi.org/10.1375/brim.11.2.183*

Outcome Measures Page ofPageof[1][12]

OFFICIAL For Internal Use Only

Soo, C.; Tate, R; Hopman, K; Forman M et al. (2007) Reliability of care needs scale in assessing support after TBI The Journal head trauma rehabilition vol-22 no -5 pp= pgs 288 to 295 doi = Tate RL (2017). Manual Care Needs Scale (CANS). Unpublished manuscript John Walsh Centre for Rehabilitation Research University Sydney Updated version 2 [link] text continues below