National Disability Insurance Scheme Amendment (Securing the NDIS for Future Generations) Bill 2026

Submission 2333

This submission addresses section 59B Automation of administrative action. It recommends that use of computer-based tools include provisions for human override of any automated decisions, along with regular audits of computer-based automation performance, and extensive testing and end user training.

I am an IT professional with over 25 years of experience in the IT industry. I currently work as a Data Engineer as a consultant providing services in the AI and Data space. As such, I have a lot of experience with computer-based tools and their limitations.

I have seen reporting indicating that earlier designs of the NDIS changes included this kind of proposal:

   There will be no right of appeal if funding amounts calculated by these algorithms is wrong - only the input data can be challenged, not the black box calculation. Human planners overseeing this process will not be allowed to manually adjust algorithm output.

This is very concerning, as this setup will inevitably lead to very dangerous outcomes in some cases. It has the potential to be “Robodebt 2.0”, with an even more vulnerable population on the receiving end.

I have read through the NATIONAL DISABILITY INSURANCE SCHEME AMENDMENT (SECURING THE NDIS FOR FUTURE GENERATIONS) BILL 2026 EXPLANATORY MEMORANDUM https://parlinfo.aph.gov.au/parlInfo/download/legislation/ems/r7487_ems_35e6531f-c440-4faf-98d6- 7c7ddd8bd539/upload_pdf/JC018272.pdf;fileType=application%2Fpdf#search=%22legislati on/ems/r7487_ems_35e6531f-c440-4faf-98d6-7c7ddd8bd539%22

It seems that this proposal has been rethought, and the intention is to allow human oversight, which is encouraging. However, I believe that the legislation should explicitly require the NDIA to include avenues for human intervention and appeal.

My experience is that there is no possible computer-based tool (including AI driven tools) that can reliably handle all complex scenarios, including unexpected edge cases, without sometimes making bad recommendations that would cause serious and damaging outcomes. This is true even for relatively simple businesses, with far less complexity than the kinds of complex medical issues that the NDIA deal with.

Override capability

It is always necessary for a human with knowledge of the situation to be able to override a computer-based tool recommendation. It is reasonable to include protections around this, such as requiring the person overriding the decision to document their reasons, and (for large adjustments) to get sign off from a supervisor or senior person in the area. But the capability to adjust and override computer-based software driven decisions MUST exist.

National Disability Insurance Scheme Amendment (Securing the NDIS for Future Generations) Bill 2026

Submission 2333

Testing and auditing

It is essential that any computer-based tool used for assessments be rigorously tested prior to being used and regularly reviewed against real world outcomes while being used. I suggest that a requirement to regularly (at least annually) audit computer-based tool performance and update it to address issues, be a requirement of the use of computer-based tools by the NDIA.

End user training

I am concerned that there is a tendency for many people who do are not familiar with how computer-based tools are designed and developed, and their inevitable limitations, to rely too heavily on computer-based outputs and judgements.

I recommend that end user training cover the limitations of computer-based tools for assessment and ensure that the users understand clearly the pathways to override the recommendations when it is appropriate. Training should be repeated at regular intervals, and cover issues that have been found in any annual audit of the computer-based tool.