3 May 2021
Committee Secretary Joint Standing Committee on the National Disability Insurance Scheme Department of the Senate PO Box 6100 Parliament House CANBERRA ACT 2600 AUSTRALIA
Dear Committee Secretary,
Re: Joint Standing Committee on the National Disability Insurance Scheme inquiry into Independent Assessments
At the request of Senator Jordan Steele-John we provide below some more information about our concerns with the proposed toolkit for the Independent Assessment Process. Of the measures in the toolkit, we believe the Vineland Adaptive Behavior Scale is one of if not the most psychometrically robust, meaning of the highest quality, and yet we still have grave concerns of its intended purpose in the Independent Assessment Process. These concerns are discussed below.
1. The Vineland is not designed to be used as a self-report measure
In 1935, Edgar A. Doll developed the Vineland Social Maturity Scale (VSMS) to aid the diagnosis of intellectual disability (ID) through formal assessment of adaptive behaviour (1). In 1984, the publishers of the Vineland selected Sara S. Sparrow to revise Doll’s measure into what today is known as the Vineland Adaptive Behavior Scale (VABS). Both Sparrow and Doll agreed that the Vineland should be an informant report measure, meaning it is designed to be completed by someone who cares for a person with intellectual disability. The VABS was designed as an informant report because it is expected that intellectual impairment can impede a person’s ability to either accurately report their functional capacity or to perform a task on request to enable the observer to assess function directly (2). The Vineland is, therefore, not designed to be completed by the person with disability (self-report).
Sparrow was a trained speech therapist, clinical psychologist and neuropsychologist and strongly felt that the best way to administer the VABS is through clinical interview with an informant. Although self- administered parent and teacher-informant versions are available. Sparrow still advocates that the clinical interview method is the gold standard as it allows for the use of experienced clinical judgement. This method requires extensive training to learn the detailed rules for administering and scoring the items. The psychometric properties of the measure are dependent on the measure being administered correctly. Any deviation from the rules for administration or scoring will compromise the validity and reliability of the measure. Administering the Vineland directly to a person with disability would require changes to the wording of the items or the interview questions, which changes the meaning of the items and the validity of the results. To put it simply, administering the Vineland as a self-report is to use the measure incorrectly and will likely result in incorrect results.
2. The Vineland is not culturally appropriate for use in Aboriginal and Torres Strait Islander communities
Both Doll and Sparrow’s versions of the Vineland define adaptive behaviour as “the performance of daily activities required for personal and social sufficiency” (Sparrow, 2016, page 10). One of the key principles inherent in this definition is that adaptive behaviour is evaluated in a social context, and adaptive competence is relative to the expectations and standards of the social environment. Culture heavily influences a social environment. Doll’s original Vineland was designed “for general use in ordinary urban and rural United States situations, within the usual limits of social economic status” (3). Put simply it was designed for the average American.
As we have noted in our original submission to the Committee on the 29th March 2021 and in our presentation to the Committee on the 27th April 2021, we have first-hand experience to confirm that the VABS does not fit the social context of remote Aboriginal and Torres Strait Islander communities.
In 2020 and 2021 we administered the VABS to 70 families living in remote Aboriginal communities in the Fitzroy Valley and found the measure is incredibly difficult to administer as it does not fit with the social expectations and standards of these communities. Consistent with our experience, government schools in the Kimberley region of WA do not use the VABS to assess adaptive function, despite it being the gold standard, as it simply does not work for their students and families.
Academics and clinicians leading the Lililwan Project (4) and the Banksia Hill study (5), which were both studies of the prevalence of fetal alcohol spectrum disorder in predominately Aboriginal populations in WA, also considered the cultural appropriateness of the VABS and both research teams decided not to use the measure because it simply does not measure what it is intended to measure.
Furthermore, the VABS is inappropriate for use in older children and adolescents living in remote communities, who are given more independence and are less likely to be closely supervised by their parents than non-Aboriginal children in urban environments.
The NDIA have stated that staff will receive cultural competency training before administering the assessments, but we argue that even a local Aboriginal person with disability experience would not be able to administer the VABS as it is the measures itself that is inappropriate.
It is imperative that the government invests in the development of cultural appropriate measures for Aboriginal and Torres Strait Islander Communities.
2. The measure was not designed to place a monetary value on the total scores
Australian research has shown that the total economic cost of intellectual disability is $14,720 billion annually (6) and that behaviour problems directly increase the cost of care of a child with intellectual disability (7). These findings mean that a child with mild intellectual disability, but severe behaviour problems may require more funding than a child with profound intellectual disability but no behaviour problems. However, the proposed NDIS Independent Assessment model would provide the reverse, namely more funding to a person with profound intellectual disability but without behavioural problems. The reason for this error is that the model is reliant on results from a tool designed to be an aid for clinical judgement to make financial decisions. This is just one example of many in how the use of the VABS to determine funding will create inequity.
Page 2 of 4
3. The VABS is designed to aid clinic decision not to make the decision for clinicians
Psychological assessments are designed to aid in clinical decision-making. Measures of adaptive behaviour, IQ or even autism spectrum disorder (like the Autism Diagnostic Observation Schedule) do not directly determine a person’s diagnosis or account for a person’s overall ability. Rather they help contribute information to the clinician to allow them to bring the evidence together to make a final decision. The clinical judgement is essential in ensuring an accurate diagnosis as it is able to consider other vital pieces of information and the influence of external factors. For example, a child who was up all night with a tummy bug will score more poorly on an IQ test than usual simply because they have not had enough sleep. Ignoring this external influence and focusing only on the score risks incorrectly diagnosing the child. Similarly, a VABS conducted with a caregiver who only sees a person with a disability in one environment, such as in their home to assist with self-care, will struggle to accurately report on how that person performs in a social environment or with their finances. A clinician with disability experience will consider these external factors when interpreting total scores to ensure a report provides an accurate reflection of a person’s ability and acknowledges where more evidence is required.
The proposed Independent Assessment process will minimise reliance on clinical judgement as it will only rely on a total score and not a clinician’s interpretation of the validity and reliability of the score. The process will also place what little clinical judgement is still present in the hands of inexperienced professionals. While the model intends to use allied health professionals, measures like the VABS can only be completed by one professional per person with a disability. This means that a physiotherapist, for example, will be asked to assess not only the physical but also the cognitive, social and behavioural functions of an individual despite only being qualified to assess physical function. The physiotherapists clinical training, experience and judgement is of little relevance when interpreting information about the cognitive, social and behavioural needs.
In sum, the diagnostic measures, like the VABS, do not determine the care or treatment needs of an individual. Rather the needs are determined through a clinician using their judgement to piece information from multiple sources together, including their own consultation and observations. Experienced clinical judgement is essential to the use of these tools and must be included when deciding the needs of an individual.
Kindly,
Ms Emily Carter
CEO MWRC
Dr Lauren Rice
Research Fellow,
University of Sydney
Prof Elizabeth Elliott
Professor of Paediatrics,
University of Sydney
Ms Sue Thomas
Strategic Priority Lead MWRC
Emeritus Prof Stewart Einfeld
Child and Adolescent Psychiatrist,
University of Sydney
Prof Carol Bower
Telethon Kids Institute
Page 3 of 4
References
- Doll EA. Vineland Social Maturity Scale. 1935:30.
- Sparrow SS, Saulnier CA, Cicchetti DV. Vineland-3. 2016:350.
- Doll EA. The Measurement of Social Competence; A Manual for the Vineland Social Maturity Scale, by Edgar A. Doll. 1953:664.
- Fitzpatrick JP, Elliott EJ, Latimer J et al. The Lililwan Project: study protocol for a population-based active case ascertainment study of the prevalence of fetal alcohol spectrum disorders (FASD) in …. BMJ open. 2012
- Passmore HM, Giglia R, Watkins RE et al. Study protocol for screening and diagnosis of fetal alcohol spectrum disorders (FASD) among young people sentenced to detention in Western Australia. BMJ open. 2016;6:e012184.
- Doran CM, Einfeld SL, Madden RH et al. How much does intellectual disability really cost? First estimates for Australia. Journal of Intellectual and Developmental Disability. 2012;37:42-49.
- Einfeld SL, Ellis LA, Doran CM et al. Behavior problems increase costs of care of children with intellectual disabilities. Journal of Mental Health Research in Intellectual Disabilities. 2010;3:202-209.
Page 4 of 4