NDIA Accountability Project

Concerns regarding independent assessments and their impact on NDIS planning

‹ PrevPage 1 of 28 · Source p. 1Next ›

Independent Assessments Submission 1

Submission to the Joint Standing Committee on the

National Disability Insurance Scheme

Inquiry into Independent Assessments

2021

Enable Plus 1 February 2021

ENABLE+ NDIS, Disability and Mental Health Assessment, Planning and Support

NDIS Provider No. 4-433C-311

www.enableplus.com.au ABN: 85 828 612 457

Dr Larry Cashion Consultant Psychologist

Philomena Holdforth Consultant Psychologist

Enable Plus Submission – NDIS Independent Assessments

Table of Contents

Executive Summary ……………………………………………………………………………………………… 2

  1. Introduction …………………………………………………………………………………………………….. 3
  2. Responses to the Terms of Reference ……………………………………………………………….. 4 2.1 a. the development, modelling, reasons and justifications for the introduction of independent assessments into the NDIS …………………………………………………………… 4 2.1.1 Development …………………………………………………………………………………………… 4 2.1.2 Modelling and methodology …………………………………………………………………….. 11 2.2 c. the human and financial resources needed to effectively implement independent assessments ……………………………………………………………………………………………….. 14 2.3 d. the independence, qualifications, training, expertise and quality assurance of assessors……………………………………………………………………………………………………. 14 2.3.1 Independence ……………………………………………………………………………………….. 14 2.3.2 Qualifications, training, and expertise ………………………………………………………… 15 2.3.3 Quality assurance ………………………………………………………………………………….. 16 2.4 e. the appropriateness of the assessment tools selected for use in independent assessments to determine plan funding …………………………………………………………… 16 2.4.1 Quality of assessment …………………………………………………………………………….. 16 2.4.3 Appropriateness of scales ……………………………………………………………………….. 18 2.5 f. the implications of independent assessments for access to and eligibility for the NDIS ………………………………………………………………………………………………….. 23 2.6 g. the implications of independent assessments for NDIS planning, including decisions related to funding reasonable and necessary supports ………………………………………. 23 2.7 h. the circumstances in which a person may not be required to complete an independent assessment ……………………………………………………………………… 24 2.8 i. opportunities to review or challenge the outcomes of independent assessments …. 24 2.9 j. the appropriateness of independent assessments for particular cohorts of people with disability, including Aboriginal and Torres Strait Islander peoples, people from regional, rural and remote areas, and people from culturally and linguistically diverse backgrounds ……………………………………………………………………………………. 24 2.10 k. the appropriateness of independent assessments for people with particular disability types, including psychosocial disability ……………………………………………………………. 25 2.11 l. any other related matters ……………………………………………………………………… 26
  3. Summary and conclusions ……………………………………………………………………… 26

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

Executive Summary

The National Disability Insurance Scheme’s (NDIS) objects and principles are based on the concepts of choice, control, independence, and opportunity for Australians living with a disability. We submit to the Committee that the proposed Independent Assessment implementation by the NDIS is inconsistent with the objects and principles enshrined in NDIS legislation and the intention of the Parliament of Australia. The main contentions in our submission are:

  • There is insufficient evidence to support the need for the Independent Assessment proposal for NDIS applicants and Participants.
  • There is a sufficient number of incorrect statements and unsupported interpretations in NDIS reports associated with Independent Assessments to raise questions over the validity of conclusions in those documents.
  • In a sufficient number of cases, what has been reported as evidence in the NDIS documents is not consistent with accepted practice standards in reviewing literature or conducting research. This is especially relevant when it has the potential to affect several hundred thousand Australians with disabilities, their families, and the community.
  • There are multiple instances of NDIS decision-making in the reports that are contrary to the criteria set out in the Independent Assessments framework.
  • Many of the issues of concern reported by the NDIS as problematic in the current system of functional assessment either remain present in the Independent Assessment proposal or are likely to be exacerbated by its introduction.
  • The Independent Assessment proposal removes the rights of people with a disability to exercise choice and control in decision-making.

We submit that there remains too much uncertainty in how the implementation of Independent Assessments will affect the balance and transparency of NDIS determinations, potentially perpetuating current concerns about fairness and accountability in the agency’s processes.

P a g e | 2

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

1. Introduction

Enable Plus is an NDIS Provider that offers accredited specialist behavioural support, psychological assessment and intervention, and specialist support coordination to NDIS Participants. We have been registered NDIS Providers since its inception in Tasmania in 2014 and are currently practicing in Victoria. We were also working in Cairns during the NDIS roll out in Far North Queensland. Prior to the merging of the NDIS and the Helping Children With Autism programs, we were an accredited for FaCHSIA Early Intervention provision.

It is our overall submission that the introduction of Independent Assessments is inconsistent with the principles of the National Disability Insurance Scheme Act 2013 (Cth) (hereafter ‘NDIS Act’) and subordinate legislation that enshrines the construct of Participant ‘choice and control’ throughout.^1,2,3,4,5,6,7 Given the social and cultural disadvantage of most NDIS Participants, connection with trusted providers is one of the few aspects of their lives they are likely to have control over. Compulsory Independent Assessments for Participants who already have treating clinicians and supports will have that choice and control removed from decisions about which professionals provide appropriate assessment. We submit that NDIS

^1 NDIS Act s 3(1)(e): enable people with disability to exercise choice and control in the pursuit of their goals and the planning and delivery of their supports (underline added). ^2 NDIS Act s 3(1)(a): in conjunction with other laws, give effect to Australia’s obligations under the Convention on the Rights of Persons with Disabilities. ^3 NDIS Act s 4(4): People with disability should be supported to exercise choice, including in relation to taking reasonable risks, in the pursuit of their goals and the planning and delivery of their supports (underline added). ^4 NDIS Act s 4(8): People with disability have the same right as other members of Australian society to be able to determine their own best interests, including the right to exercise choice and control, and to engage as equal partners in decisions that will affect their lives, to the full extent of their capacity (underline added). ^5 NDIS Act s 4(13): The role of advocacy in representing the interests of people with disability is to be acknowledged and respected, recognising that advocacy supports people with disability by… (b) promoting choice and control in the pursuit of their goals and the planning and delivery of their supports (underline added). ^6 NDIS Act s 5(13): It is the intention of the Parliament that, if this Act requires or permits an act or thing to be done by or in relation to a person with disability by another person, the act or thing is to be done, so far as practicable, in accordance with both the general principles set out in section 4 and the following principles … (a) people with disability should be involved in decision making processes that affect them, and where possible make decisions for themselves; … (c) the judgements and decisions that people with disability would have made for themselves should be taken into account; … (e) the supportive relationships, friendships and connections with others of people with disability should be recognised (underline added). ^7 See also NDIS Act s 9, s 17A, s 31, s 118(1)(a)(ii), s 144; National Disability Insurance Scheme (Quality Indicators) Guidelines 2018 (Cth) s 6, s 9, s 57; National Disability Insurance Scheme (Provider Registration and Practice Standards) Rules 2018 (Cth) sch 1 s 3, s 6.

P a g e | 3

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

Participants are entitled to their legislated and socially constructed human rights being upheld, rather than treated as a quantum of functional capacity.

2. Responses to the Terms of Reference

2.1 a. the development, modelling, reasons and justifications for the introduction of independent assessments into the NDIS

2.1.1 Development

The foundations of the development of the Independent Assessment review included:

The suite of assessment tools should be diagnosis neutral (i.e. it should be possible to use the suite of assessment tools across all disabilities) (sic).^8

The stated philosophy of the NDIS is to support people with disabilities through individually tailored programs. It is contradictory to this philosophy that a generic assessment of functional disability can provide a holistic approach in understanding individual needs. Furthermore, the NDIS has reported that it commenced reviewing generic (‘disability- neutral’) assessments in 2015,^9 prior to the Productivity Commission review of 2017^10 and the ‘Tune Review’ of 2019.^11 However, the NDIS relies on these latter two documents as bases for contending Independent Assessments are required at this time. As such, the statement by the NDIS that Independent Assessments are being introduced “to improve equity and consistency in decision making in response to the Tune Review” is not supported.^12 Regardless of whether the Tune Review recommended Independent Assessments, the evidence strongly suggests they are not “a response”, per se, to that that review. This statement could be considered misleading on the face of it.

A further foundation of the development of the Independent Assessment review was that any assessment should consider:

What is the best a person can do at a given time in a given place with and without assistance? (sic)^13

^8 NDIS. (2020). Independent assessment: Selection of assessment tools. Canberra, ACT: Author, p. 8. ^9 NDIS. (2020). Independent assessment framework. Canberra, ACT: Author, p. 3. ^10 Productivity Commission. (2017). National Disability Insurance Scheme (NDIS) Costs (Study Report). Canberra, ACT: Author. ^11 Tune, D. (2019). Review of the National Disability Insurance Scheme Act 2013: Removing red tape and implementing the NDIS Participant Service Guarantee. Canberra, ACT: Department of Social Services. ^12 NDIS. (2020). Independent assessments: Pilot learnings and ongoing evaluation plan. Canberra, ACT: Author. ^13 Above n 8.

P a g e | 4

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

This criterion is confusing and open to a range of interpretations by assessors and planners, some of which contradict the principles of the NDIS Act. It raises questions left unanswered in the framework, including:

  • If a person has one ‘good’ day in seven days, is the ‘good’ day the benchmark set by the framework?
  • If a person is unlikely to have improved functionality with assistance, but would experience better quality of life, is there scope to include such information in the Independent Assessment?
  • People with cognitive challenges often do not comprehend questions being asked when anxious and can tend to respond to demand characteristics. Is this taken into consideration when a self-reported ‘best’ is not an objective or realistic portrayal of functioning?
  • If a professional assessment by a qualified medical or allied health practitioner reports significantly lower functioning than during Independent Assessment, will practitioner information be disregarded on the basis that the Participant reports his or her ‘best’ is significantly greater?

The inconsistency frequently reported in current NDIS planning decisions will, we submit, be promoted by statements that are unclear and open to interpretation by assessors, planners, and delegates.

The foundations of the framework continue with:

Assessment tools should be questionnaire based rather than performance based (to avoid reflecting assessment performance in an unfamiliar or unnatural setting; to avoid reflecting assessment performance on an atypical day - either ‘good’ or ‘bad’ day; to avoid the challenges of performing on-demand; to avoid challenges of performing in front of an unfamiliar assessor)(sic).^14

Questionnaires suffer from the same problems listed in the above quote that the NDIS has associated with performance-based assessments. Despite the assertion that the proposed approach will minimise these issues, there is nothing in the framework to justify such a statement, other than unsupported opinion. It is well established in peer-reviewed literature and authoritative texts that:

  • Questionnaire-based assessments conducted in an unfamiliar or unnatural setting have the potential to lead to affective and cognitive disturbances that deviate from typical

^14 Ibid.

P a g e | 5

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

presentations. This tends to result in response biases.^15,16,17 There is nothing in the framework document to guarantee Independent Assessments will not be administered in unfamiliar or unnatural settings in any case.

  • Answering a questionnaire does not mitigate changes to responding on an ‘atypical’ day any more than any other assessment types. Indeed, first-hand observations of functionality are significantly more difficult to ‘fake’ than self-reports where a skilled practitioner conducts an assessment.
  • As noted above, the idea that a generic questionnaire about functionality avoids responding to a ‘good’ day conflicts with the statement that NDIS Independent Assessments are based on ‘best’ performance.
  • Demand characteristics are inherent in research and assessments where cues about the purpose of the task are available (or inferred) by a Participant.^18,19 The nature of the proposed Independent Assessments is well known and of great importance to the responders. Social desirability bias using questionnaires and structured interviews can lead to significantly higher ratings of functioning than found using objective measures, including when a third-party is the respondent.^20,21,22
  • Significantly inflated ratings of functional capacity have also been reported for interviews facilitated by professionals compared to self-administered questionnaires.^23,24 Questionnaires are particularly problematic for people with disabilities that affect

^15 Seidenberg, M., Taylor, M. A., & Haltiner, A. (1994). Personality and self-report of cognitive functioning. Archives of Clinical Neuropsychology, 9, 353–361. ^16 Murdoch, M. et al. (2014). Impact of different privacy conditions and incentives on survey response rate, participant representativeness, and disclosure of sensitive information: A randomized controlled trial. BMC Medical Research Methodology, 14, Article 90. ^17 Harber, K. D., Stafford, R., Kennedy, K. A. (2010). The positive feedback bias as a response to self-image threat. British Journal of Social Psychology, 49, 207–218. ^18 Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783. ^19 Sharpe, D. & Whelton, W. J. (2016). Frightened by an old scarecrow: The remarkable resilience of demand characteristics. Review of General Psychology, 20(4), 349–368. ^20 Van de Mortel, T. F. (2008). Faking it: Social desirability response bias in self-report research. Australian Journal of Advanced Nursing, 25(4), 40–48. ^21 Logan, D. E., Claar, R. L., & Scharff, L. (2008). Social desirability response bias and self-report of psychological distress in pediatric chronic pain patients. Pain, 136(3), 366–372. ^22 Fayers, P. M. & Machin, D. (2016). Quality of life: The assessment, analysis and reporting of patient-reported outcomes (3rd ed.). Chichester, UK: John Wiley & Sons. ^23 Cook, D. J. et al. (1993). Interviewer versus self administered questionnaires in developing a quality of life instrument for asthma. Journal of Clinical Epidemiology, 46, 529–534. ^24 Fayers & Machin (2016) n 22.

P a g e | 6

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

independent reading ability.^25 Reading scale items aloud to responders can affect the way participants answer questions.

  • Anxiety or fear about the context of assessment and how it will impact on supports might also result in negative information bias.^26
  • Potential NDIS Participants will be subject to ‘performing in front of an unfamiliar assessor’ as part of the Independent Assessment proposal. Logically, if there were any concerns about lack of familiarity with assessors as a general principle, as noted in the previous quote, Independent Assessment would not be proposed. Performance in an interview session may also subject to the Hawthorne Effect, where a person might modify his or her behaviour simply because the person is being observed or some other associated environmental factor.^27

The 2011 Productivity Commission stated that Independent Assessment would reduce the potential for ‘sympathy’ bias by professionals through:

The supports to which an individual would be entitled should be determined by an independent, forward-looking assessment process by the NDIA, rather than people’s current service use.^28

Significantly, the section of the Productivity Commission report titled Maintaining professional objectivity contains no references to peer-reviewed research to support the contention that a potential Participant’s current service providers might have any form of bias. Later in that report a single reference to a brief report from Guscia et al. (2006)^29 is cited as indicating susceptibility to manipulation as affecting validity and reliability of assessments conducted by professionals.^30 However, the respondent sample in the Guscia et al. study comprised 12 support workers and not registered health and allied health practitioners, all from the same setting. The 29 participants who were rated for service needs were already in residential care,^31 unlike most applicants to the NDIS. Furthermore, comparisons were made between 2001 and 2003 scores on a single scale – Service Need Assessment Profile (SNAP; Gould, 1998).^32 Interrater reliability for the SNAP has been

^25 Ibid. ^26 This should not be considered synonymous with malingering, which is intentional false responding in order to avoid obligations. ^27 McCambridge, J., Witton, J., & Elbourne, D. R. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267–277. ^28 Productivity Commission. (2011). Disability Care and Support (Report no. 54). Canberra, ACT: Author, p. 65. ^29 Guscia, R., Harries, J., Kirby, N., & Nettelbeck, T. (2006). Rater bias and the measurement of support needs. Journal of Intellectual & Developmental Disability, 31(3), 156–160. ^30 Above n 28, p. 316. ^31 Above n 29, p. 157. ^32 Ibid, p. 157.

P a g e | 7

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

reported to be as low as 0.61, which is psychometrically unreliable for a rating scale.^33 That fact alone provides as convincing an explanation for score differences over time as any perceived bias by raters. As such, we submit that the authors of the Guscia et al. paper and the reference to it by the Productivity Commission overstate the importance, significance, and relevance of this small study of support workers responding on a single questionnaire. We submit that on relevance alone this paper provides limited, if any, support for the concerns of the Productivity Commission regarding potential bias by health and allied health professionals.

The same section of the Productivity Commission report recommends a “forward-looking assessment process”. There is no definition of what a ‘forward-looking’ process is and how it might differ from current practices. We submit that current functional assessment practices are generally conducted in a way that provides conclusions about what supports and interventions are required in the future, as well as under current circumstances. In the case of current service providers, prognoses are based on a range of assessments and include clinical judgement. The proposed Independent Assessment procedure will provide less accurate prognoses because clinical judgement and knowledge of the client are largely removed from the process. We submit that rather than being ‘forward-looking’, the Independent Assessment proposal is firmly rooted in present functioning using the tools and procedures recommended by the NDIS.

In the development of the framework, the NDIS used information from the 2011 Productivity Commission report to state there are two “key causes of potential bias” in the current assessment approach of practitioners who already know the person providing capacity information.^34

The first source of “potential bias” was reported as:

An assessment approach which is perceived to be deficits-based, whereby people feel the need to present themselves at their worst in order to be funded for the supports that they need.^35

Nothing in the proposed Independent Assessment approach ensures potential Participants could not present themselves “at their worst”. As noted above, the proposed approach may increase negatively biased responding in accordance with peer-reviewed research. It is difficult to follow the rationale for this comment by the NDIS, as current service providers often have the opportunity to see a person over the long term, during ‘good’ and ‘bad’ periods, rather than an independent assessor who might speak to or observe the person once.

^33 Verdugo, M. A., Aguayo, V., Arias, V. B., & García-Domínguez, L. (2020). A systematic review of the assessment of support needs in people with intellectual and developmental disabilities. International Journal of Environmental Research & Public Health, 17, Article 9494, p. 11. ^34 Above n 9, p. 7. ^35 Ibid.

P a g e | 8

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

The second source of “potential bias” identified by the NDIS was:

Real or perceived inconsistency and uncertainty around the process of decision making by the NDIS, which may lead to assessors overstating, whether intentionally or not, the need for funding for supports for the people with whom they have developed a professional relationship.^36

We submit that if there is any real or perceived inconsistency around NDIS practices, it falls to the NDIS to ensure that decision-making processes are improved within the organisation. We contend that the above statement is consistent with blame-shifting from the NDIS to the many highly trained and ethical practitioners working with people with disabilities in the period of their lives prior to them making NDIS applications. The only peer-reviewed evidence cited to support the above statement was the same Guscia et al. (2006) paper discussed above. The NDIS stated:

An Australian study by Guscia, Harries, Kirby and Nettelbeck (2006) investigated whether assessment tools obtained different results for different purposes. They found that support measures “may significantly overestimate support needs when raters know they are being used for funding purposes” (Guscia et al 2006, p159). This risk may be amplified if the ‘rater’ or assessor has a professional relationship with the person being assessed. (sic).^37

We submit that the final statement of this paragraph does not accurately reflect the conclusions of Guscia et al. It is misleading to state that the support workers who responded to the scale have a “professional relationship” in the same way as a registered health or allied health professional. Furthermore, there is nothing in the Guscia et al. paper indicating that the risk of overestimating support needs is “amplified” where a professional relationship exists. Once again, the small sample size, lack of reliability of the scale, single residential care setting, responders being support workers and not health or allied health professionals, inflation of the importance of the results, and lack of general applicability of the results render this study of limited value in the current context. We have no doubt that many health and allied health practitioners who work with people with disabilities would be offended by such baseless assertions about their competence and professionalism.

We also submit that the NDIS suggestion that clinicians advocating for clients involved in therapy involves any form of unethical or unprofessional conduct is opinion and no basis is given for it:

Clinicians advocating for their clients is fundamental to any therapist-client partnership.^38

The Australian Concise Oxford Dictionary defines advocacy as:

^36 Ibid. ^37 Above n 9, p. 26. ^38 Above n 9, p. 7.

P a g e | 9

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

  1. verbal support or argument for a cause, policy, etc. 2. the function of an advocate. (p. 22).

The same source defines advocate as:

  1. A person who supports or speaks in favour. 2. A person who pleads for another. 3. A professional pleader in a court of justice. (p. 22)

We respectfully contend that there is nothing attributed to the role of advocate in these definitions that would imply deceptive, biased, or unprofessional conduct. As such, we further assert that neither of the “key causes of potential bias” have been supported by the NDIS in its framework document.

It is also worthwhile noting that most practitioners working with people with disabilities are trained in ethics and professional practice, in addition to AHPRA registered clinicians having to follow mandatory codes of conduct. Most practitioners that work with people with disabilities have chosen this professional pathway. We are trained to provide objective, professional, and ethical reports, even when the information might be inconsistent with the desires of a client, parent, or school. We found it concerning that the 2011 Productivity Commission report stated:

It is clear from the experiences of VCAT appeals on TAC benefit decisions that treating professionals are often placed in an invidious position when asked by their patients to make an assessment that determines the person’s eligibility for benefits.^39

We submit that this statement is emotive, rather than providing any evidence of potential bias in the conduct of a health professional working with a person with a disability. The Productivity Commission report does not state why being requested to provide a professional report is “invidious”. We assert that being asked to write reports about client functioning is a core aspect of working in the broader health field. The same professional practices apply to functional disability reports as they do with neuropsychological, forensic, speech and language, and other reports. Clients do not always agree with the conclusions in those reports, but in our extensive experience as registered psychologists, members and managers of multidisciplinary teams, professional supervisors, university lecturers, and providers of professional development, we have observed the vast majority of practitioners staunchly engaging in ethical practice, regardless of pressure brought to bear by other parties.

Finally on this point, there is no guarantee independent assessors will not demonstrate ‘sympathy bias’ based on Participant presentation. The utilisation of a standard, diagnosis- neutral assessment for current and potential NDIS Participants remains subject to not only human biases, but is also highly dependent on the qualifications, training, and experience of the independent assessor in each case. As stated by Cushman (2013):

^39 Above n 28.

P a g e | 10

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

The study of humans … cannot be an activity made neutral by putatively objective procedures. It cannot be removed from the messy world of interpretations, meanings, and power relations.^40

2.1.2 Modelling and methodology

The modelling used in developing the rationale for Independent Assessments has relied heavily on a primarily economic approach. Several NDIS documents acknowledge that the reported research undertaken by the organisation to support these assessments was predicated on economic modelling.

The NDIS has not released the internal report from its evaluation from the pilot studies. It is unclear why a summary was publicly available,^41 but not the report. As such, readers can only rely on NDIS interpretations of the studies, which fails to provide transparency and limits opportunities for constructive comment.

However, there are major issues affecting the reliability of the studies in the summary report, including:

The first pilot ran from November 2018 to April 2019. Its overarching objective was to demonstrate the potential benefit of independently sourcing standardised functional assessments for NDIS applicants and participants, to improve the consistency, accuracy and reliability of NDIA decisions.^42

A fundamental principle of any research is that it approaches the research question objectively (i.e., hypothesis testing). The NDIS summary overtly stated that the intention of the first pilot study was to confirm a preconceived conclusion. This means that any interpretation of the results must be treated with great caution, given the stated intention of the research.

Further significant informational, methodological and modelling issues included:

  • The summary report contains a large amount of information that has limited value in summarising the outcomes of the pilot studies. Several pages of figures and tables are provided that detract from the summary’s readability.^43
  • The summary report includes conclusions about the need for Independent Assessment based on this extraneous information.^44 These conclusions are independent of the pilot studies and have dubious relevance to the studies themselves.

^40 Cushman, P. (2013). Because the rock will not read the article: A discussion of Jeremy D. Safran’s critique of Irwin Z. Hoffman’s “Doublethinking our way to scientific legitimacy”. Psychoanalytic Dialogues, 23, 211–224 (p. 215). ^41 Above n 12. ^42 Ibid, p. 10. ^43 Ibid, much of pp. 4–9. ^44 Ibid, pp. 7–8.

P a g e | 11

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

  • The frequent digression into commenting on the second pilot study when discussing the first pilot study unnecessarily complicates the summary.
  • Data for the two pilot studies are inconsistently reported in the summary report (e.g., response rates for one pilot study are provided in the text, but not for the other).
  • The study has a serious methodological flaw in that there was no comparison group to contrast the ratings of those completing the Independent Assessment pilot and those who experienced ‘business as usual’.
  • The sample in the first pilot study was reported to have included categories of disability applicable to 64% of the population of NDIS Participants.^45 This means that even in the best case it would be unclear what the implications of Independent Assessments are for 36% of NDIS Participants, amounting to 148,514 people with one or more disabilities.^46
  • The reported sample size of the first pilot study of 513 Participants only refers to the number of assessments and not the number of respondents to the post-assessment survey.^47 In any case, the assessments sample comprised only about 0.1% of the NDIS Participant population, which raises questions about applicability of findings to the whole Participant population.
  • Sample size for the number of Participants who completed all the requirements of the first pilot study was reported as 145.^48 It was also stated that 91% of completers reported being satisfied or very satisfied with the assessment, but there was no breakdown on the two types of responding.^49 There was no report of the questions to Participants in eliciting these responses, which raises questions about what they were actually asked.
  • It was reported that 28% of the first pilot sample reported that the assessor was not sufficiently familiar with the person’s disability. If this finding was applied to the entire NDIS Participant population, that percentage potentially represents about 115,510 people with a disability. Regardless of the satisfaction with the assessment process and the professionalism of the assessor, this figure represents a significant perception that assessors did not have the requisite knowledge and skills about each individual assessed to provide a sense of certainty and security for NDIS Participants.
  • The sample size for the second pilot study was reported as 30 in Table 3.^50 However, it was later stated that the sample was 27 Participants.^51 By any measure this sample,

^45 Ibid, p.10. ^46 Based on data from the NDIS for July – September 2020 reporting 412,539 active NDIS Participants nationally: https://data.ndis.gov.au/explore-data accessed 9 January 2021. ^47 Above n 12, p. 12. ^48 Ibid, p. 13. ^49 Ibid, p. 15. ^50 Ibid, p. 13. ^51 Ibid, p. 15.

P a g e | 12

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

regardless of the reasons for it being so small, is not representative of the NDIS Participant population. The percentages reported for satisfaction in the sample are statistically unreliable and potentially misleading to the lay reader.^52

  • It was further reported:

In the first pilot, for the purposes of the evaluation, an assumed “typical population range” was also set for each assessment tool in order to determine a participant’s likely functional capacity … For the purposes of the first pilot a participant was considered to be in the assumed “normal population range” if Vineland 3 and PEDI-CAT domain scores fall within two standard deviations of the population mean (sic).^53

No rationale for this decision was provided in the summary report. It is generally recognised that a score of less than or equal to two standard deviations (≤ -2 SD) below the mean score on a psychometric assessment indicates a clinically significant deficit. However, the level where functional impairment occurs is not universally agreed upon. Standardised assessments apply various descriptors to performance levels, but the range ≤ -1 SD – > -2 SD is usually not considered ‘average’ or ‘typical’.^54 Multiple peer-reviewed papers in the Journal of the International Neuropsychological Society have recommended using a standard of ≤ -1.5 SD on one measure and ≤ -1 SD on two or more measures as indicating clinically significant impairment.

  • We are further concerned that the summary report appears to extend beyond the scope of the studies in forming the conclusion:

The results indicated that providing these participants with early intervention rather than permanent disability supports would likely have been more appropriate.^55

There is no rationale for this statement other than the assumption a score on one assessment of ≥ -2 SD indicated ‘typical’ performance. We submit that this conclusion is

^52 Ibid, pp. 15–16. ^53 Ibid, p. 14. Note: the terminology ‘normal population range’ is incorrectly applied in this passage. Any introductory statistics text explains the ‘normal distribution’ of scores (sometimes erroneously called the ‘bell curve’) and that in statistical language the word ‘normal’ is not synonymous with ‘typical’. ^54 For example, the Wechsler Adult Intelligence Scale – 4th Edition (WAIS-IV) considers the ‘Average’ range as ± 0.66 SD (< -0.66 SD – ≥ -1.33 SD is the ‘Low Average’ range; < -1.33 SD –

-2 SD is the ‘Borderline’ range); the Wechsler Intelligence Scale for Children – 5th Edition (WISC-V) has the same mean and SD, but uses ‘Very Low’ in preference to ‘Borderline’; the Vineland Adaptive Behaviour Scales – 3rd Edition (Vineland-3) refers to the range -1 SD – > -2 SD as ‘Moderately Low’; Clinical Evaluation of Language Fundamentals Australian and New Zealand – Fifth Edition (CELF-5 A&NZ) considers the ‘Average’ range ± 1 SD (< -1 SD – ≥ -1.50 SD is the ‘Marginal’ or ‘Borderline’ range; < -1.50 SD – > -2 SD is the ‘Low’ range); the Bruininks- Oseretsky Test of Motor Proficiency (BOT-2) considers a score of < 1 SD from the mean to be indicative of clinically significant impairment. ^55 Above n 12, p. 15.

P a g e | 13

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

neither supported nor justified by the data contained in the summary report. We are unaware of the clinical skills and qualifications of the report authors that would allow them to form legitimate conclusions about the need for NDIS support or early intervention. However, we submit that it was appropriate to identify those qualifications and experience where such a definitive clinical judgement was made about “appropriate” intervention.

The summary report’s section titled Ongoing evaluation continues in a manner that suggests Independent Assessments are definitively supported. We submit that the contents of the summary report do not provide a compelling argument in favour of initiating Independent Assessments. Indeed, that section of the summary report clearly indicates that insufficient information is currently available to draw reliable conclusions about the proposed assessments.^56

2.2 c. the human and financial resources needed to effectively implement

2.2.1 Human resources

Given the significant delays experienced in the second pilot study, it is unclear how this will be avoided in the future. The financial impact of Independent Assessments has been calculated using economic modelling, but the ability of the workforce to provide assessments at the level of clinical skill required is not well addressed in available NDIS documentation.

2.2.2 Financial resources

The economic modelling supporting Independent Assessments has been comprehensive. However, these economic models have not factored in the additional costs that are likely to result from appeals related to rejection of applications by potential Participants as a result of these assessments. Similarly, the likelihood of review requests by active Participants were supports downgraded through the Independent Assessment process has not been factored into the models.

Some of the modelling uses assumptions about community costs of establishing eligibility for inclusion in the NDIS in the absence of Independent Assessments. The assumption in some models that Independent Assessments will relieve costs from services by consultant medical and allied health professionals is based on fallacy. The assessment and diagnosis process has to occur prior to any NDIS application in any case, which will often include functional assessment. It is our submission that the opportunity to utilise functional assessments that have already been administered by health professionals has not been genuinely considered in the economic modelling.

2.3 d. the independence, qualifications, training, expertise and quality assurance of assessors

The NDIS framework states in the section titled Rationale for Reviewing NDIS Assessment of Functional Capacity:

^56 Ibid, pp. 17–19.

P a g e | 14

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

Enable Plus Submission – NDIS Independent Assessments

Mainstream or specialist health professionals, who may or may not know the person well, may provide information. NDIS staff who have completed an in-house assessment, either in person or over the phone, may also provide information. Qualifications, skill level, experience, and understanding of functional capacity can also differ greatly between assessors, which in turn impacts the quality and quantity of information provided to NDIA.^57

There is no evidence provided in the NDIS documentation that demonstrates the issues in the above quote would be resolved by Independent Assessment. If the current Independent Assessment proposal was to proceed, issues would include:

  • An independent assessor will not know the person well.
  • Qualifications, skill level, experience, and understanding of functional capacity will differ greatly between independent assessors.
  • Consistency between independent assessors will not be improved in the current Independent Assessment proposal, as the stated intention of the NDIS is to provide minimal training.

2.3.1 Independence

Assessors in the current implementation trial either work for or are contracted to APM. This company is reliant on funding from the NDIS and, given the repeated assertions by the NDIS that it is focussed on seeking support for Independent Assessments, it must be considered that as a business APM will follow the directions and guidance of the NDIS, rather than being genuinely independent. Furthermore, the workers or contractors will be required to follow the directions of APM and are, again, not genuinely independent.

If the NDIS subsequently puts Independent Assessment out to tender, providers will have to compete on financial grounds. Economising on these assessments will likely result in lower remuneration for assessors, meaning that it is unlikely more experienced or qualified professionals will take the opportunity to work for the successful tenderer.

Assessors or organisations with a contract for assessment will be subject to key performance indicators (KPIs). If these KPIs are linked to quotas of any form, this will significantly impact on the genuine independence of the assessments.

2.3.2 Qualifications, training, and expertise

The framework states that generic skills will be required to administer the Independent Assessments.^58 Allied health disciplines approach assessment through differing lenses. While tools might be generic, interpretations made by different disciplines may vary widely. The intention of providing limited training to assessors is likely to result in varying recording

^57 Above n 9, p. 5. ^58 Above n 8, p. 8, p. 38.

P a g e | 15

ENABLE+

www.enableplus.com.au ABN: 85 828 612 457

of subjective assessment items dependent on discipline-specific factors.59 Furthermore, qualifications and training differ between allied health disciplines to the extent that the idea of a generic skill set is of limited value.60

In terms of expertise, we would again submit that it is unlikely highly experienced practitioners will be interested in Independent Assessment employment. As noted, the financial outcomes of working with a successful tenderer are likely to be significantly lower than for private practitioners. Furthermore, the inability to use clinical judgement in assessments would likely be unappealing to experienced clinicians. We submit that it is likely that assessors will come mainly from a pool of less-experienced professionals, compared to the current system where many health and allied health professionals are approached because of expertise and reputation. These professionals in private practice and organisations are genuinely independent, as they are selected through Participant choice and control, rather than allocated by a government agency.

2.3.3 Quality assurance

There is limited information available about quality assurance in the proposed Independent Assessment approach. This area of practice also raises the question of whether an assessor who rates Participants with greater than average impairment than other assessors, for professionally appropriate reasons, will be considered as providing ‘poor quality’ outcomes in terms of the applicable business model.

2.4 e. the appropriateness of the assessment tools selected for use in independent

assessments to determine plan funding

2.4.1 Quality of assessment

Further to quality considerations, the NDIS stated:

The NDIA conducts ongoing evaluation to ensure standards are being maintained, and as part of this process has undertaken an evidence based review of the approach to how functional capacity is assessed. Assessment of functional capacity in the NDIS

59 Mokkink, L. B. et al. (2010). Inter-rater agreement and reliability of the COSMIN (COnsensus- based Standards for the selection of health status Measurement Instruments) Checklist. BMC Medical Research Methodology, 10, Article 82. 60 Registered psychologists complete a minimum of 6 years of academic and clinical training before full registration through AHPRA can be sought; Occupational therapists complete 4-year degree or equivalent, including practicums in clinical settings, before full registration through AHPRA can be sought; Physiotherapists complete a 4-year degree or equivalent, including practicums in clinical settings, before full registration through AHPRA can be sought; Certified Practising Speech Pathologists complete a 4- or 5-year degree or equivalent, including practicums in clinical settings, before full certification through Speech Pathology Australia be granted; Social workers complete a 4-year degree or equivalent, including practicums in various settings, before full membership of the Australian Association of Social Workers can be granted.

P a g e | 16

context is complex. The unique NDIS context has been closely examined and understood in order to ensure a robust framework for assessment.61

We agree with the statement that assessment of functional capacity in the NDIS context is complex. As such, it is our submission that the complexity of disability presentations and the additional challenge of standardising human individuality makes a ‘one size fits all’ generic assessment of functional capacity unlikely to provide fair and reasonable outcomes for NDIS applicants and Participants.

Importantly, evidence of a complete literature review of best practice in assessing functional capacity is not evident in the framework and other NDIS reports. If, as asserted by the NDIS, that context is “unique”, it is difficult to ascertain from the summary information in the documentation whether the framework is compatible with that uniqueness, is “robust”, or even fit for purpose. The insufficiency of the reported “evidence based review (sic)” starkly contrasts with accepted academic and clinical approaches.62 We further submit that the NDIS review of tools used to evaluate capacity is not synonymous with a comprehensive review of “how functional capacity is assessed”.

2.4.2 Decision-making and consensus

The NDIS reported that the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) (Prinsen et al., 2016)63 was utilised in determining the appropriateness of assessment tools for the Independent Assessment approach.64 While acknowledging that a flow chart of the COSMIN process was included in the report, the report text itself does not accurately reflect the Prinsen et al. methodology.

The COSMIN approach of Prinsen et al. (2016) as reportedly utilised by the NDIS does not refer to functionality scales, but is rather a methodological guide to clinical trials. We submit that it cannot be argued that the ‘pilot studies’ by the NDIS to try and support the need for Independent Assessments are clinical trials for the myriad reasons discussed in section 2.1.1 of this submission.

The COSMIN approach was developed by Mokkink et al. (2006) to assess evaluative health related patient-reported outcomes (i.e., health status) associated with quality of life scales, rather than functionality scales.65 With the exception of Prinsen et al. (2016), the broader academic work by this group that has been published in peer-reviewed journals has

61 Above n 9, p. 9. 62 See the discussion of the Vineland-3 below as an example, where a single encyclopedia entry was considered sufficient “high quality evidence” of cross-cultural validity. 63 Prinsen, C. A. C. et al. (2016). How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials, 17(1), Article 449. 64 Above n 8, p. 4, p. 9. 65 Mokkink, L. B. et al. (2006). Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Medical Research Methodology, 6, Article 2.

P a g e | 17

consistently applied the COSMIN methodology to the evaluation of perceived health outcomes.66,67,68,69,70,71,72,73,74

Unlike the methods prescribed in the COSMIN literature, there is no compelling evidence of a consensus of international experts. The number of experts consulted by the NDIS is not disclosed, while Prinsen et al. (2016) reported 481 were invited to participate in their study, although ultimately 95 experts responded. With respect to the experts who were consulted by the NDIS, several appropriate scales for functional assessment under the framework appear not to have been identified. This suggests that a full census of scales likely to be appropriate were not included in the consensus part of the selection process.

Overall, it is unclear why COSMIN methodology was chosen when the selection of assessment tools was neither a report of patient outcomes nor a clinical trial. In any case, there is limited evidence that the selection of scales consistently applied the COSMIN methodology.

2.4.3 Appropriateness of scales

2.4.3.1 Vineland-3

The selection of the Vineland Adaptive Behaviour Scales – 3rd Edition (Vineland-3) is, we submit, reasonable under all circumstances. The Vineland-3 is a well accepted and widely- used assessment of adaptive functioning. However, the Vineland-3 is a semi-structured interview and the skills and experience of the professionals administering the scale and

66 Mokkink, L. B. et al. (2009). Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Quality of Life Research, 18, 313–333. 67 Terwee, C. B., Jansma, E. P., Riphagen, I. I., & de Vet, H. C. W. (2009). Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Quality of Life Research, 18, 1115–1123. 68 Mokkink, L. B. et al. (2010). The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Medical Research Methodology, 10, Article 22. 69 Mokkink, L. B. et al. (2010). Inter-rater agreement and reliability of the COSMIN (COnsensus- based Standards for the selection of health status Measurement Instruments) Checklist. BMC Medical Research Methodology, 10, Article 82. 70 Mokkink, L. B. et al. (2010). The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Quality of Life Research, 19, 539–549. 71 Mokkink, L. B. et al. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient- reported outcomes. Journal of Clinical Epidemiology, 63, 737–745. 72 Mokkink, L. B. et al. (2018). COSMIN Risk of bias checklist for systematic reviews of Patient- Reported Outcome Measures. Quality of Life Research, 27, 1171–1179. 73 Prinsen, C. A. C. et al. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27, 1147–1157. 74 Terwee, C. B. et al. (2018). COSMIN methodology for evaluating the content validity of patient reported outcome measures: a Delphi study. Quality of Life Research, 27, 1159–1170.

P a g e | 18

interpreting responses are potential sources of variability. The element of subjectivity presents a challenge to the stated intention of the NDIS to use objective assessments in Independent Assessments.

A particular issue in the NDIS documentation with regard to the Vineland-3 is the strong reliance on the reference to an online encyclopedia entry by Burger-Caplan, Saulnier, and Sparrow (2018)75 as evidence of validity.76 In reviewing the print edition of Burger-Caplan et al. (in which the text is identical), scale validity is treated cursorily and does not mention the specific types the NDIS states are supported by that brief summary. The encyclopedia entry simply states:

Investigation for validity was conducted based on the content, structure, demographic characteristics, clinical groups, and the relationships with other measures such as Adaptive Behavior Assessment System, Third Edition (ABAS-3), Bayley Scales of Infant Development, Third Edition (Bayley III), and the Vineland II (sic).77

It is our submission that the conclusions regarding Burger-Caplan et al.78 contained in the NDIS report were overly liberal. Indeed, the NDIS conclusion that the Vineland-3 showed good ‘cross-cultural validity’ with ‘quality of evidence’ reported as “high” was based solely on a loose interpretation of the encyclopedia summary.79 We further submit that in any case, the practice of citing a brief encyclopedia summary as evidence lacks the robustness required to form the associated conclusions reached by the NDIS.

Similar issues were also manifested by the inclusion of a test review of the Vineland-3 in the Canadian Journal of School Psychology.80 The authors of that review simply outlined the psychometric properties in the Vineland-3 from the test manual, rather than reporting on independent research. Furthermore, these authors concluded that the Vineland-3 showed:

… highly variable test–retest stability meaning that the subscales appear to be somewhat inconsistent across time … the Vineland-3 is reliant upon other observers

75 Cited as: Burger-Caplan, R., Saulnier, C., & Sparrow, S. (2018). Vineland adaptive behavior scales. In J. Kreutzer, J. DeLuca, & B. Caplan (Eds.), Encyclopedia of clinical neuropsychology (Living Edition) (pp. 1–5). Springer International Publishing. 76 Above n 8, p. 28. 77 Burger-Caplan, R., Saulnier, C., & Sparrow, S. (2018). Vineland Adaptive Behavior Scales. In J. Kreutzer, J. DeLuca, & B. Caplan (Eds.), Encyclopedia of clinical neuropsychology (2nd ed.) (pp. 3597–3601). Cham, Switzerland: Springer International. 78 Above n 8, p. 28. 79 Above n 8, p. 28, Table 1. 80 Pepperdine, C. R. & McCrimmon, A. W. (2018). Test review: Vineland Adaptive Behavior Scales, Third Edition (Vineland-3) by Sparrow, S. S., Cicchetti, D. V., & Saulnier, C. A. Canadian Journal of School Psychology, 33(2), 157–163. (Cited in error in NDIS report as: Pepperdine CR, McCrimmon AW. Test Review: Vineland Adaptive Behavior Scales, (Vineland-3) by Sparrow, SS, Cicchetti, DV, & Saulnier, CA: SAGE Publications Sage CA: Los Angeles, CA; 2018.)

P a g e | 19

rating the individual in question and so is susceptible to responder bias (potentially reflected in the measure’s variable interrater reliability scores).81

Another reference cited in support of the Vineland-3 was an unpublished study by the Autism CRC in Queensland.82 Again, the absence of peer-review and unavailability for public scrutiny is concerning in terms of transparency and robustness of the NDIS review. A search of the Autism CRC web site found only a summary description of the project, which stated the study was funded by the NDIS, that it was primarily to evaluate the PEDI-CAT (ASD), and did not include a completion date.83 However, a poster including some of the data from that project was presented at the International Society for Autism Research 2020 Annual Meeting.84 Contrary to the NDIS report that internal consistency of the Vineland-3 was acceptable,85 Evans et al. (2020) reported that this property was found to be “unacceptable to excellent”, indicating marked variability.

The fact that we consider the scale to have sufficient psychometric properties to be appropriately included on the proposed NDIS assessment list does not detract from the concerns we have about how the decision-making process was conducted and justified.

2.4.3.2 PEDI-CAT (ASD)

The Pediatric Evaluation of Disability Inventory-Computer Adaptive Test for autism spectrum disorders (PEDI-CAT [ASD]) was reported by the NDIS as being “validated for children and young people with Autism Spectrum Disorders (ASD) (sic)”.86,87 The single peer-reviwed, published reference provided by the NDIS to support this statement is a paper by Kramer et al. (2016).88 This study utilised a 39-participant “convenience sample”.89 Kramer et al. also

81 Ibid, p. 162. 82 Cited as: Evans K, Whitehouse A, Chamberlain A, et al. Reliability, Validity and Usability of Assessment of Functioning Tools for Autism Spectrum Disorder and Neurodevelopmental Conditions in the Australian Context (unpublished): Autism CRC; 2020. 83 Autism CRC (undated). Reliability, validity and usability of assessment of functioning tools for autism in the Australian context. Accessed 12/1/2021 at <https://www.autismcrc.com.au/our- programs/early-years/reliability-validity-and-usability-assessment-functioning-tools-autism> 84 Evans, K. et al. (2020, June). Formalising an assessment of functioning process for individuals undergoing an autism diagnostic evaluation and/or service planning in Australia [Poster]. International Society for Autism Research 2020 Annual Meeting. Virtual conference. 85 Above n 8, p. 28. 86 Above n 8, p. 17. 87 In Australia, the acronym ASD represents Autism Spectrum Disorder (not ‘Disorders’, plural), referring to DSM-5 terminology. While the ICD continues to separate Pervasive Developmental Disorders into several diagnoses, this is not the standard set by the federal government. 88 Kramer, J. M., Liljenquist, K., & Coster, W. J. (2016). Validity, reliability and usability of the Pediatric Evaluation of Disability Inventory-Computer Adaptive Test for children and young people with autism. Developmental Medicine & Child Neurology, 58(3), 255–261. 89 Ibid, p. 255; scores on all scales administered in this study were reported for only 36 participants.

P a g e | 20

elected to only assess three of the four domains, omitting the mobility domain “because it examines basic motor skills not typically affected in this population”.90 This rationale is not supported, as motor skills issues have been regularly reported in the ASD literature.E.g.,91,92,93,94

With 10 – 15 items per domain over the three scales considered, this meant that a minimum of 30, although potentially up to 45, PEDI-CAT (ASD) items were administered for each study participant. Common wisdom in research methodology and statistical analysis is that 10 participants per scale item are required as a bare minimum in reliability studies, meaning that caution needs to be taken in generalising the Kramer et al. results from 39 responders when 300 would be the lower limit. We would acknowledge that the reported results of Kramer et al. provided an impressive set of statistics with this very small sample. However, we submit that Kramer et al. significantly overstated the generalisability of their results with their sample of only 39 participants. As such, we further submit that the heavy reliance on this one study with methodological and interpretative challenges does not provide a robust reason for inclusion of this assessment tool in the Independent Assessment proposal by the NDIS.

We acknowledge that the NDIS included a second reference in support of the PEDI-CAT (ASD).95 However, this reference was an internal NDIS report that was not peer-reviewed, was not published in an authoritative journal, and was not released to the public for scrutiny. We submit that this reference does not offer objective evidence to support the inclusion of the PEDI-CAT (ASD) in the Independent Assessment proposal.

Further to the inclusion of the PEDI-CAT (ASD), the NDIS framework and rationale for selection of appropriate tools included that all assessments were diagnosis-neutral. The PEDI-CAT (ASD) was developed specifically for young people with an Autism Spectrum Disorder diagnosis and, therefore, fails to comply with that parameter set by the NDIS.

2.4.3.3 WHODAS-2.0

The World Health Organisation Disability Assessment Schedule – Version 2.0 (WHODAS- 2.0) was developed as a culture-neutral, quantitative functional assessment. There are some

90 Ibid, p. 257. 91 Gillberg, C. & Coleman, M. (2000). The biology of the autistic syndromes. London, UK: Mac Keith. 92 Jansiewicz, E. M. et al. (2006). Motor signs distinguish children with high functioning autism and Asperger’s syndrome from controls. Journal of Autism & Developmental Disorders, 36(5), 613–621. 93 Attwood, T. (2007). Movement and coordination. In T. Atwood, The complete guide to Asperger’s Syndrome (pp. 259–270). London, UK: Jessica Kingsley. 94 Neely, K. A. et al. (2019). Motor memory deficits contribute to motor impairments in Autism Spectrum Disorder. Journal of Autism & Developmental Disorders, 49(7), 2675–2684. 95 Cited as: Chamberlain, A. et al. (2019). Reliability, validity and usability of the PEDI-CAT and PEDI-CAT (ASD) for Autism Spectrum Disorder and neurodevelopmental conditions in the Australian context: Scoping review, ICF linking and pilot feedback (Internal report only).

P a g e | 21

benefits to using the WHODAS-2.0 as it is simple to administer and the items are transparent, requiring little interpretation by participants. However, the WHO and other organisations, including the NDIS, have frequently omitted studies of the WHODAS-2.0 that challenge its utility across populations. For example:

  • The WHODAS-2.0 has been found to classify the functioning of psychiatric patients as significantly less impaired than experienced clinicians (with no funding issues involved to potentially influence results).96
  • Using the standard scoring method, the WHODAS-2.0 correctly classified the functioning of psychiatric patients in only 44% of cases.97
  • While the WHODAS 2.0 was developed as a measure of disability associated with all physical and mental disorders, many items are not relevant for assessing disability related to mental disorders.98
  • According to the American Psychiatric Association, here are no established thresholds for interpreting the global or domain-specific scores in relation to the criterion of clinically significant impairment. Furthermore, evidence is limited for whether any cut-off scores are meaningful across patients with different types of mental disorders, “let alone for patients with any kind of disease or disorder and across all demographic categories”.99
  • Paradoxically, a person with a disability who is working will complete the work scale items and may receive a higher disability score than another who does not or cannot work.100 This issue also applies to items Participants report are ‘not applicable’ to them, resulting in scores suggesting less impairment than objectively found.101

Our submission is that the literature review reported by the NDIS in relation to the inclusion of the WHODAS-2.0 lacked objectivity. This is not a criticism of the scale itself, but highlights that in evaluating the relevant material the NDIS failed to consider the scales using an appropriately balanced and academically rigorous approach.

96 Sedano-Capdevila, A. et al. (2018). WHODAS 2.0 as a measure of severity of illness: Results of a FLDA analysis. Computational and Mathematical Methods in Medicine, 2018, Article 7353624, pp. 5–6. 97 Ibid, p. 5. 98 Konecky, B. et al. (2014). Using the WHODAS 2.0 to assess functional disability associated with DSM-5 mental disorders. American Journal of Psychiatry, 171(8), 818–820. 99 Ibid, p. 818. 100 Ibid. 101 Castro, S., Leite, C. F., Coenen, M., & Buchalla, C. M. (2019). World Health Organization Disability Assessment Schedule 2 (WHODAS 2.0): comentários sobre a necessidade de revisar a WHODAS (‘The World Health Organization Disability Assessment Schedule 2 (WHODAS 2.0): Remarks on the need to revise the WHODAS’). Cadernos De Saúde Pública, 35(7), Article e00000519.

P a g e | 22

2.5 f. the implications of independent assessments for access to and eligibility for the

NDIS

Almost every set of diagnostic criteria in the DSM-5 includes a criterion that the person diagnosed experiences “clinically significant impairment in social, occupational, or other important areas of current functioning”. As such, at least for individuals with a DSM-5 diagnosis, a qualified and experienced health or allied health practitioner (or team of practitioners) has identified clinically significant functional impairment. Any disagreement by an assessor under the Independent Assessment proposal would require at least equivalent qualifications, experience, and understanding of the individual’s presentation across all functional domains to legitimately query the previous clinicians’ conclusions.

We submit that it is highly unlikely assessors under the proposed scheme will meet that high-level skill threshold, especially given that the NDIS has reported it intends to provide limited training to those assessors. The approach by some NDIS Planners in questioning the opinions of experts in disability has already damaged the credibility of the system and we contend that disagreement between highly-skilled, qualified external clinicians and the NDIS assessors is likely to further erode confidence in the agency.102 In any case, regardless of what degree or experience the Planner might have, he or she is not acting in the capacity of a health practitioner in that role.103

2.6 g. the implications of independent assessments for NDIS planning, including

decisions related to funding reasonable and necessary supports

In the current approach, professionals familiar with NDIS Participants have an opportunity to advocate (in a reasonable and unbiased manner) to assist with accessing reasonable and necessary supports. Independent assessors will not be in a position to advocate for the best interests of Participants. Furthermore, the problem remains of how the NDIS and Planners will deal with circumstances where disagreement between opinions of a person’s functionality exist between professionals and NDIS assessors. Given the unsupported statements by the Productivity Commission104 and the NDIS105,106 that professionals are at high risk of bias in their functional assessments, it would be surprising if the Independent Assessment regime would not be considered, in our view incorrectly, the more objective and reliable.

102 See section 2.6 following for an example. 103 E.g., One of the submission authors (LC) recommended an evidence-based therapy for a child with Autism Spectrum Disorder. The Planner made several suggestions about other therapies and requested further justification as to why one of the Planner’s alternatives was not recommended. This resulted in a cost to the NDIS for the clinician to prepare a response, even though the Planner’s other suggestions for intervention would have attracted similar costs for the NDIS to the originally recommended evidence-based therapy. 104 Above n 28. 105 Above n 8. 106 Above n 9.

P a g e | 23

2.7 h. the circumstances in which a person may not be required to complete an

independent assessment

It is our submission that the only time a person would require an Independent Assessment would be if he or she has not been provided with one prior to the application to join the NDIS.

2.8 i. opportunities to review or challenge the outcomes of independent assessments

We submit that a truly independent body would need to be established to review challenges to Independent Assessment outcomes. Given the proposed approach to these assessments from the NDIS, it is our submission that there is a high risk that any review body would be overwhelmed by applications, especially where already-existing funding were cut for current Participants. The costs of reviews and potential court actions has not been taken into genuine consideration under the economic models proposed by the NDIS.

To this end, the Independent Assessment proposal may require a change to the NDIS legislation to remove Participant ‘choice and control’. Such action would contradict the principles of the NDIS articulated by the Australian Parliament and current legislation. An opinion on whether the principles articulated in the NDIS legislation might give rise to offending amendments being struck out by a court is beyond our expertise. However, appeals to superior courts are not only expensive for plaintiffs, but also for governments, especially if NDIS Participants succeed in any such action.

2.9 j. the appropriateness of independent assessments for particular cohorts of

people with disability, including Aboriginal and Torres Strait Islander peoples,
people from regional, rural and remote areas, and people from culturally and
linguistically diverse backgrounds

We expect that advocates for Aboriginal and Torres Strait Islander Australians and for people from culturally and linguistically diverse (CALD) backgrounds will provide compelling reasons why the Independent Assessment proposal will further disadvantage these vulnerable Australians, whose disabilities are often poorly recognised. However, we submit that one example will demonstrate this point better than multiple academic references.107

In 2017, 62-year-old Bhutanese refugee Gauri Adhikari took her own life in Cairns, Far North Queensland. Ms Adhikari had spent 22 years in a refugee camp in Nepal, where she experienced and witnessed horrendous physical and sexual violence, as well as having to survive in squalor for over two decades. Ms Adhikari’s experience as a refugee left her with a ‘disabled’ right arm, Posttraumatic Stress Disorder, and Depression. She did not read, write, or speak English, having only arrived in Australia in 2013.

Despite her circumstances, in 2016 Ms Adhikari was sent notices from an employment agency stating that she had breached her Centrelink job seeker obligations and her social security payments would be stopped unless she took part in a ‘job plan’ and

107 Gregory, K. (2017, January 18). Family of refugee who took her own life calls for improved access to disability pension (summary of story reported on ABC Radio program PM).

P a g e | 24

attended appointments with the agency. Ms Adhikari’s family advocated on her behalf with the agency, which was allegedly ‘independent’ of government, and Centrelink. In part due to their challenges with English, Ms Adhikari and her family were unable to report on sufficient symptoms to be classified as 20-points on an impairment scale for her to be granted the Disability Support Pension (DSP). As the notices and telephone calls became more frequent and the tone more intimidating, Ms Adhikari’s mental health deteriorated markedly, to the extent she was unable to leave her home, stopped eating, and communicated minimally. She was able to obtain a medical certificate that provided her an exemption from job seeking for three months. As soon as that period ended the job agency started sending intimidating letters again.

One of the authors of this submission (PH) was consulted in her role as a psychologist. She was so concerned about Ms Adhikari’s mental health and physical presentation that she organised urgent assessment and inpatient admission to the Cairns Hospital Mental Health Unit. As with many of her other interactions in Australia, Ms Adhikari was unable to communicate at an adequate level with hospital staff. It was determined that Ms Adhikari was not being assisted by being in the unit and was discharged a week later. Ms Adhikari was provided with a discharge summary from the hospital, which the job agency rejected, allegedly stating, “it doesn’t work like that”. Her psychologist requested a formal report on Ms Adhikari to assist in making an application to Centrelink for the DSP, but was advised that public health psychiatrists were not permitted to do that.

Although she was uncommunicative and in a near-catatonic state, 10 days after her hospital discharge she waited until her family members had left the house and took her own life. Evidence recovered confirmed that Ms Adhikari’s actions were the result of her sense of being intimidated, harassed, and unsafe as a result of the job provider’s behaviour.

It is our submission that Ms Adhikari was disempowered by a system she did not understand, which treated her without any understanding of her individual presentation and circumstances. Participants from Indigenous Australian and CALD backgrounds whose language and cultural norms are very different from mainstream Australian society are likely to be the most affected by the Independent Assessment proposal. Ms Adhikari was labelled as recalcitrant, uncooperative, and someone who would not ‘help herself’, because of a combination of mental illness and language issues that were beyond her control. Any claim that people from Indigenous Australian and CALD backgrounds with a disability will have an equal opportunity to participate in the NDIS with urban Participants under the proposed Independent Assessments is, in our opinion, unfounded on the basis of well-recognised and documented disadvantage.

2.10 k. the appropriateness of independent assessments for people with particular

  disability types, including psychosocial disability

A stated principle of the NDIS is that Participants will be treated with dignity, respect, and with reference to individual needs. There is no evidence of how that will be supported through the current Independent Assessment proposal other than aspirational statements.

P a g e | 25

Our submission on the issue of appropriateness is outlined to a significant degree in sections 2.1, 2.3, 2.4, 2.5, and 2.9 above. However, it is important to reiterate that different disabilities will have variable impact on participation, responding, and emotional reaction to proposed Independent Assessment. Some of the issues likely to affect specific groups include:

  • Individuals on the autism spectrum often take a literal approach to language. Any abstract language in task instructions or scale items can easily be misinterpreted.
  • Many people with a psychosocial disability have difficulty interpreting the intentions and emotions of others. This will affect responding, possibly leading to defensiveness that has the potential to be inaccurately labelled as an ‘uncooperative attitude’.
  • People with Autism Spectrum Disorder and psychosocial disability often downplay or have limited insight into their level of impairment.108 It requires a skilled clinician with much experience not to simply accept a superficially positive presentation.
  • As noted above, some of the scales chosen by the NDIS as part of the Independent Assessment proposal do not have well-established reliability and validity for specific and general disability presentations (e.g., relying on a sample only 39 individuals in a reliability study for the PEDI-CAT [ASD]).

We would like the opportunity to thank the panel for considering not only our submission, but in reviewing the Independent Assessment proposal in such detail. People with disabilities often experience limited opportunities to participate in society on anywhere near equal footing to mainstream Australians. The removal of any level of choice and control from these people erodes our credibility as a society, having already taken the step to set the choice and control principle as a foundation of the NDIS.

3. Summary and conclusions

The main conclusions in our submission are:

  • There is insufficient evidence to support the need for the current Independent Assessment proposal for NDIS applicants and Participants.
  • There is a sufficient number of incorrect statements and unsupported interpretations in the NDIS reports to raise questions over the validity of their conclusions.
  • In a sufficient number of cases, what has been reported as evidence in the NDIS documents is not consistent with good practice in reviewing literature or conducting studies. This is especially relevant when it has the potential to negatively affect hundreds of thousands of Australians with disabilities, their families, and the community.

108 Writing this with great respect to our clients, the authors of this submission have met several young men with Autism Spectrum Disorder who report their serious intention of being famous rock guitarists, all of whom either could not play guitar or did not own one at that time.

P a g e | 26

  • There are multiple instances of NDIS decisions in the reports that are contrary to the criteria set out in the framework. Indeed, many of the issues of concern reported by the NDIS in the current system of functional assessment either remain present in the Independent Assessment proposal or are likely to be exacerbated by its introduction.
  • The Independent Assessment proposal is contrary to the stated intention of the Australian Parliament, NDIS legislation, and the rights of people with a disability.
  • There remains too much uncertainty in how the implementation of Independent Assessments in accessing supports through the NDIS will be balanced, transparent, and open to challenge in the event that the process is not just.

Dr Larry Cashion BPsych(Hons), PhD, GradCertEd, GradCertForBehSci Consultant Psychologist AHPRA Registration No. PSY0001485896 NDIS Behaviour Support Practitioner ID No. A3759 APS Endorsed Autism Spectrum Disorder Practitioner Member International Neuropsychological Society Member Australian Society for the Study of Brain Impairment Member Australian Association of Psychologists

Philomena Holdforth BAgSci, DipEd, GradDipPsych, PostgradDipPsych Consultant Psychologist AHPRA Registration No. PSY0001485876 NDIS Behaviour Support Practitioner ID A2381 Member Australian Society for the Study of Brain Impairment Member Australian Association of Psychologists

P a g e | 27