Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry
Dec 2022 NDIS misconceptions of functional capacity assessment and the fallacy of functional decline of NDS participants
Allied is a group of allied health professionals committed to working towards more equitable health and disability services. We welcome the opportunity to contribute to the Joint Standing Committee (JSC) on the NDIS – NDIA Capability and Culture. This Submission follows an earlier Submission to this inquiry dated October 2022. This submission was prompted by the publication of the AFSR in November 2022 [1].
The assessment of functional capacity to identify participant support needs, and subsequent allocation of support funding, is a critical element of the NDIS. This paper calls for a testing of the underlying assumptions that underpin the allocation of support and funding, in the context of the legislation and the UNCRPD. In addition, we call for a re-test of the validity of the use of impairment-based functional ‘levels’ to inform actuarial modelling informing and predicting financial sustainability of the NDIS. We also recommend that in the context of the current Royal Commission into the Robodebt Scheme, The NDIA releases information to reassure participants and families that all automated elements of the fund allocation process are consistent with relevant legislation.
Currently, an impairment-based functional assessment generates a score and associated functional ‘level’; the score and associated level are used as an input to generate a ‘Typical Support Package’ (TSP), alongside additional inputs ( disability type, age band, gender, Supported Independent Living status) [ASFR 2021-2022 p.5] [1]. The process of assigning a ’ functional level’ through the NDIS planning process, as disclosed in the NDIA’s response to a question from Senator Jordon Steele-John, taken on notice at a meeting of the Joint Standing Committee on 18 May 2021, requires scrutiny from the Joint Standing Committee (JSC) on the NDIS – NDIA Capability and Culture. Please refer to Appendix 1 which described the NDIS process for determining impairment- based functional ‘level’ based on functional assessment scores, which was current at May 2021 [2].
Specifically, we call for a re-test of the following assumptions:
- That the NDIS has a robust and consistent process to assess functional capacity
- The functional assessment scores can reliably inform funding decisions in an NDIS plan
in a manner consistent with the legislation, in the absence of a process for support needs identification
- The assessment tools informing functional ‘levels’ have capacity to assessment functional capacity.
- There is an evidence base for impairment-based functional levels; particularly, equity and parity between impairment-based cohorts of people with disability
- That functional ‘levels’ based on inconsistent functional assessment methods, should be used as a core feature of actuarial modelling informing and forecasting financial sustainability of the NDS
- That data trends based on functional ‘levels’ are indicative of at-scale functional decline of NDIS participants.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry Dec 2022
Table 1
Assumptions
Evidence challenging assumptions
Impact on NDIS participants
-
The NDIS has a robust and consistent process for assessment of functional capacity
-
Functional assessment scores can reliably inform funding decisions in a NDIS plan
The 2021-2022 AFSR highlights the inconsistent approach to functional assessment utilized by the NDIS [AFSR p. 39, 58, 152], and includes the recommendation:
Recommendation 6: The NDIA should, consistent with the NDIS Act, actively pursue via co-design an assessment approach which measures functional capacity in line with relevant chapters of the International Classification of Functioning, Disability and Health, and which incorporates environmental and personal factors that are salient to the determination of reasonable and necessary supports.
The JSC Inquiry to Independent Assessment final report (2021) have identified the need to develop a robust and consistent functional assessment [3}
A fundamental principle of the NDIS Act (2013) is the identification of reasonable and necessary supports, that are individualised and goal-oriented. Assessment processes that focus on average or ‘typical’ support funding, is contrary to individualised, tailored, goal-oriented support identification.
The current impairment-based functional assessment process generating a score to determine a functional ‘level’ leading to a TSP, does not include needs assessment, or a process to identify, reasonable and necessary supports.
The current assessment process impacts the rights of Australians with a disability to an evidence-based, robust and safe process for assessment of functional capacity to inform reasonable and necessary support, and funding, from the NDIS.
For example, for psychosocial disability, funding and support packages that do not address the, often invisible, individual reasonable and necessary support needs, may lead to negative outcomes and harm. These include, but are not limited to, increased reliance on the acute mental health system through mental health decline; unnecessary, costly and distressing hospitalisations or crisis service engagement; further reduced capacity for social and economic participation; social isolation; difficulty completing personal care and daily living tasks.
The impact of a functional assessment that is not fit for purpose is insufficient assessment of substantially reduced functional capacity, which cannot reliably inform funded tailored to support needs.
The NDIS process for support and funding allocation is built on unstable foundations — resulting in under/over funded plans and plans that are not tailored to actual disability support needs.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry
Dec 2022
3. The assessment tools
informing functional ‘levels’ have capacity to assess functional capacity as it is defined under the NDIS Act (2013)
The impairment-based functional assessment tools completed by NDIS planners do not have validity, reliability or sensitivity to accurately measure functional capacity as it is defined by the NDIS Act (2013)
The NDIA has provided a table of the numerous assessment tools completed by Planners for each disability type, and the score ranges (cut-off scores) determining high, medium and low levels of function that are used as an input to determine funding level, or ‘Typical Support Package’ (TSP) [2].
Construct validity is the ability of the tool to actually measure what it is intended to measure. There are issues with construct validity in the NDIS functional assessment tools listed in Appendix 1, Table 1. For example, the construct being measured with LSP- 16 is life skills, not functional capacity.
In Australia and internationally, research highlights the global absence of a single assessment tool or suite of tools, proven to have the ability to assess functional capacity [4].
For example, let’s consider psychosocial disability. Under the impairment based functional assessment list, Both the LSP-16 and WHODAS 2 can generate a score to feed the NDIS functional level system, which then informs TSP allocation [2]. However, the LSP does not assess functional capacity, it measures life skills. Two of the four subscales of the LSP-16 assess irrelevant and outdated concepts that have no place in a contemporary social model of disability, these are ‘compliance’ and ‘antisocial behavior’. Scores generated from LSP-16 have negligible correlation to functional capacity. The second option for people with psychosocial disability, is the WHODAS 2.
The WHODAS-2 has limitations e.g. does not comprehensively assess functional capacity for self-care because it does not assess the capacity or barriers to consistently wash; get dressed; plan, organise and prepare a meal; manage medication; implement daily routine. The WHODAS-2 only considers the past 30 days, not sufficient to capture the fluctuating capacity experienced by many people with psychosocial disability. The WHODAS-2 does not provide insight into the person with psychosocial disability’s capacity for economic participation.
Neither tool offer an accurate assessment of functional capacity for psychosocial disability.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry
Dec 2022
4. The evidence base for impairment-based functional levels; particularly, equity and parity within and between impairment-based cohorts of people with disability
There is no published evidence supporting the premise that a set of impairment-based functional assessment scores corresponding to a functional ‘level’ can reliably inform NDIS funding decisions, in the absence of an individualised support needs assessment.
Furtherly, there is no published evidence for the validity of functional ‘levels’ — these levels were constructed by the NDIA, and were not researched as part of the development process of individual assessment tools.
There is an assumption within the current NDIS process [2] that scores from individual assessment tools can be correlated and compared e.g. Appendix 1, Table 1 indicates that a score of 23 or higher on the LSP-16 equates to a score of 25–67 on the WHODAS, to indicate a medium ‘level’ of function for a person with psychosocial disability. This is equated to a score of 41–55 on the Vineland 2 for a person living with intellectual disability or autism. In reality, these scores on each respective tool provide vastly different information on vastly different constructs. There is no published evidence to indicate that respective scores on different assessment tools effectively mean the same thing, in terms of informing plan funding decisions — or within the NDIA constructed functional ‘level’ table.
The July 2021 Quarterly Report Addendum document titled ‘Analysis of reported level of function trend’ details the ‘level of function trends’, and indicates that factors such as who completed the assessment, and how they completed the assessment, are key to the determination of ‘functional level’(2021, p. 9) [5] — highlighting a further source of inconsistency in determining functional levels.
The impact on participant support and funding allocation due to the assumptions underpinning the functional levels, the score range cut-offs for high, medium and low function, and subsequent TSP allocation, are unknown. Using impairment-based functional assessment scores corresponding to a functional ‘level’ to generate a TSP is an example of a short-cut on process and a reliance on automated decision making. This is potentially a significant source of funding inequity based on impairment/disability type and requires transparency.
The July 2021 Quarterly Report Addendum [5] highlights a greater apparent trend towards functional decline over 5 years, when the functional level is determined by the internal, generic assessment tools (the WHODAS-2 and the PEDI-CAT). The proportion of participants assessed as ‘low’ functioning grew by 15% using the internal assessment tool administered by NDIS Planners/delegates. There was a significantly lesser trend towards functional decline over 5 years when the functional level was determined external by allied health professionals using consistent, tailored assessment strategies (presumably, participant provided documentation). The proportion of participants assessed as ‘low’ functioning grew by 4% using external (presumably, participant provided documentation) assessment reports (2021, p. 9).
There is a considerable difference between a 15% increase in participants showing functional decline, and a 4% increase in participants showing functional decline. This discrepancy casts significant doubt on the quality and accuracy of the agency’s internal assessment process, given that the external assessments are delivered by allied health professionals with particular expertise in functional assessment
Submission to Joint Standing Committee on the NDI
NDIA Capability & Culture Inquiry Dec 2022
It is clear from the 2021-2022 AFSR [1] that functional levels are a core feature of actuarial modelling informing and forecasting financial sustainability of the NDIS. While this report highlights unreliability in functional assessments (p. 39, 58, 152) [1], it seems to miss the point that the much-referenced ‘functional levels’ that underpin TSP’s are based on these same unreliable functional assessments.
That functional ‘levels’
based on inconsistent functional assessment methods, should be used as a core feature of actuarial modelling, informing and forecasting financial sustainability of the NDIS
Inaccurate functional assessment scores cannot lead to accurate generation of functional levels, for individual participants or at- scale for the broader NDIS participant population. It is alarming to consider that inconsistent and unreliable functional assessment processes generate population-level data framed within NDIA-constructed ‘functional levels’ — and this data is heavily relied upon within actuarial forecasting and modelling the financial sustainability of the NDIS.
The AFSR report (p.33) [1] states that trends indicating participant functional decline at scale are likely due to inconsistent assessment measures, rather than a reflection of actual functional decline.
That data trends
based on functional ‘levels’ are indicative of
at-scale functional The June 2021 NDIA Supplementary Submission to the JSC stated
Unknown. Merits further investigation.
The now disproven narrative that NDIS participants experience functional decline at -scale has been damaging to the public view and trust of the NDIS as Australia’s disability safety net. In addition, The narrative has served to undermine the significant benefits of the NDIS
decline of NDI “the trend is not reflective of participant function decreasing, but participants. is rather indicative of the current inconsistent methodology and data collection on functionality”.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry
Dec 2022
CONCLUSION
The absence of transparency on the NDIA’s approach to functional capacity assessment and needs identification, and subsequent allocation of supports, provides an insight into the broader capability and culture of the NDIA. The current approach is indicative of a culture centred around the tension between cost-containment based on a NDIS financial unsustainability narrative, and authentic identification of and funding of reasonable and necessary supports for participants. The focus on cost-containment has enabled a culture where short-cuts are sought, including automated decision making and neglect of individualised and tailored identification of support needs.
Very significantly, the current, flawed process of functional capacity assessment scoring is utilised to auto-determine participant impairment-based functional ‘levels’, which then generate ‘typical’ support packages. These impairment-based functional levels, based on the same flawed functional assessment scoring, are used at scale within actuarial modelling to inform and predict financial sustainability of the NDIS. The recent AFSR report widely references the flawed dataset based impairment-based functional ‘levels’, and indicates they represent at-scale functional decline of NDIS participants. This paper provides evidence around why this is not an accurate conceptualization of functional trends and patterns amongst NDIS participants.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry Dec 2022
Public Hearing — 18 May 2021
ANSWER TO QUESTION ON NOTICE
National Disability Insurance Agency
Topic: High, Medium, Low Functionality Question reference number: [Q21-000015
Senator: Jordon Steele-John Type of Question: Spoken. Hansard Page: 29 Date set by the Committee for the return of answer: 01 June 2021
Question:
Senator STEELE-JOHN: I’m genuinely trying to understand this. The agency has got a tool—an IT tool, I imagine—that takes these different inputs. Do you weight them against each other? My problem, Mr McNaughton, is: there isn’t a set ‘high’, ‘medium’ or ‘low’ score that is universal across disability. Do you have a tool that’s averaging these out across these different inputs?
Mr McNaughton: All the information we have is the WHODAS. We just use that information to develop a high, medium or low score based on that as well as the person’s primary disability. If we’ve got other, more granular information from an assessment supplied by a medical professional, we’ll use that level of rating as well.
Senator STEELE-JOHN: But how would that be weighted against the WHODAS? If I’ve just got the WHODAS versus a bunch of other stuff, how are you combining those two things, or multiple data inputs, together?
Mr McNaughton: At an individual level, if the only assessment we have is a WHODAS we will just use that level of information to help develop that person’s plan and to help understand the person’s typical support package, plus the other inputs they might need. We won’t then overlay it with other things, but for reporting purposes we can see [inaudible] than a lower range. What we’re seeing is more people move to [inaudible] or WHODAS.
Mr Hoffman: I can see that you are trying to ask a very genuine question here about the conversion of multiple different sources of information into a high, medium and low reported score.
Senator STEELE-JOHN: Yes.
Mr Hoffman: Can we take on notice to come back to you on this and effectively convert the conversion tool from those multiple sources of input provided by the participant, their Pagéo#tof:$ their treating professionals or our planners into high, medium and low? We’re happy to provide that tool.
Senator STEELE-JOHN: That’s the key that I need to be able to understand this. Without that, we as a committee can’t really interpret this.
Submission to Joint Standing Committee on the NDIS: NDIA Capability & Culture Inquiry
Dec 2022
Page 8 of 8