NDIA Accountability Project

Scrutiny of Algorithms and Governance in NDIS Operations

‹ PrevPage 1 of 38 · Source p. 1Next ›

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

JOINT STANDING COMMITTEE ON THE NATIONAL DISABILITY INSURANCE SCHEME

Senate Inquiry - NDIS Implementation and Forecasting Inquiry

Submission By:

Marie Johnson CEO Centre for Digital Business Pty Limited ABN: 16 162 122 072

29 October 2021

1

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

To the members of the Joint Standing Committee on the National Disability Insurance Scheme Thank you for the opportunity to make a submission to the Inquiry into the current scheme implementation and forecasting of the NDIS. I would also like to thank the Committee for your efforts regarding the Inquiry into Independent Assessments. In making this submission, I draw upon my previous detailed submissions to this Committee and specifically my submissions to the Inquiry on Independent Assessments (Submission number 33, and Supplementary Submission). These are provided as an attachment to this submission. This submission draws attention to key issues and questions from my previous submissions, relevant to this Inquiry. Specifically, the role of algorithms in all areas of NDIS operations need to be scrutinised. Algorithms not only impact human rights and with risk to safety, but in the case of the NDIS could also affect the governance role and budget impact of State and Territory governments. Such algorithms affect people’s plans and budgets – and whether or not people are accepted into the NDIS. This means that State and Territory governments have a vested interest in the co-design and governance of the algorithms. Algorithms and the assumptions underpinning automated decision making and the automated generation of plans – ie “personalised plans” – need to be independently governed. The co-design of algorithms is essential so as to avoid bias and mitigate risk. Because of the extreme risk of bias and impact on human rights, the control framework of the NDIS algorithms – to the extent that one exists - should not rest with the NDIS Actuary. Similarly, I have highlighted other questions on co-design to be further examined by the Committee. The recommendations of the Report of the JSCNDIS Inquiry into Independent Assessments that co-design be legislated is significant. Co-design affects all areas of policy and operations, and raises questions about the role of State and Territory governments in the formulation of the legislated co-design framework. The example of Independent Assessments is a case in point. But there will be many other policy and process interventions that should be also subject to co-design and that will have impact on State and Territory governments – and across the sector. The NDIA should not be the arbiter of what is and is not subject to co-design. The Committee would have a critical role in the determination of the governance and oversight of the legislated co-design function Of note, the Australian Human Rights Commission Report on “Technology and Human Rights” made a number of significant recommendations with implications for the NDIS. This is quite a remarkable report. The AHRC recommended that there be a moratorium on the use of decision-making technology which significantly affects individuals. Similarly, I believe that the Committee has a critical role in the determination of governance regarding algorithms, given the impact of algorithms on human rights and on scheme sustainability. The AHRC also called for the NDIS to improve funding for assistive technology as this is now considered an enabling human right. Together with the AAT decision of 28 October regarding support for funding for Internet access to ensure the functioning of assistive technology, there needs to be a considerable economic analysis of the beneficial role of assistive technology on scheme sustainability. This is a long-standing deficiency in the analysis of scheme sustainability. In summary, the following areas should be examined in this Inquiry given the implications for scheme sustainability:

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

  • How has the NDIA taken into account the recommendations of the Australian Human Rights Commission Report into Human Rights and Technology, that there should be a moratorium on the use of decision-making technology which significantly affects individuals?

Co-Design

  • What approach will the NDIA take in the development of the co-design framework? How will the community be involved? How will the States and Territories be involved in the development of the co-design framework? Will the co-design framework be published?

  • To support the JSCNDIS recommendation regarding co-design to be legislated, what actions are being undertaken by the agency to develop an in-house co-design capability?

  • What is the governance of the co-design framework? Will there be rules?

Financial Sustainability and Economic modelling

  • Economic modelling needs to examine the red tape impact of NDIS systems and processes. In particular, the catalogue (price guide and support catalogue) as one of the most critical elements of the NDIS services infrastructure lacks critical design and function. Service providers have to manually interpret the data and manually manipulate the data into their systems and is incomprehensible for participants. The catalogue needs to be re-engineered and co-designed with providers and participants.

  • There is an urgent need for independent economic modelling on the beneficial social and scheme sustainability impact of assistive technology.

Your sincerely

Marie Johnson CEO Centre for Digital Business Pty Limited

Attachment: Submission number 33 and Supplementary Submission to JSCNDIS Inquiry on Independent Assessments https://www.aph.gov.au/DocumentStore.ashx?id=92a45ded-c00c-4946-9776-812353f486df&subId=703536

                                                   3

Joint Standing Committee On The National Disability Insurance Scheme

Senate Inquiry - Independent Assessments

Submission By:

Marie Johnson CEO Centre For Digital Business Pty Limited ABN: 16 162 122 072

3 March 2021

1

SubmissionSubmission 3336

EXECUTIVE SUMMARY

I am the CEO of the Centre for Digital Business, a digital services and artificial intelligence company. I am an inclusion and accessibility advocate. I advise organisations globally, and speak and commentate on issues such as innovation, technology, digital identity, biometrics, co-design and artificial intelligence. My background includes extensive public and private sector experience in Australia and internationally. This experience covers global technology strategy; policy development; major programme delivery; digital transformation; operational service delivery of call centres; web and digital services; face to face client services; large scale technology services; and global innovation. In writing this submission, I am drawing on my somewhat unique experience: of lived experience in addition to my deep internal knowledge and experience as former Head of the NDIS Technology Authority. I wrote the business case for the NDIS ICT systems, and for this to be based on co-design and the principles of the UN Convention on the Rights of Persons with Disabilities. I have an exceptionally deep knowledge of the NDIS processes, and the NDIS ICT system which were delivered by DHS. I have a deep understanding of NDIS capability, operating model and culture. I also have considerable operational knowledge of the cross government capability, systems and architecture on which the NDIS is dependent. Perhaps somewhat unique, I believe there would be very few people with this deep level of direct internal knowledge of and experience in the NDIS operating environment, in addition to the lived experience of interacting with the NDIS. My husband suffers a chronic genetic heart condition, with multiple heart surgeries. He has significant neurological and movement disorders and is losing the use of his hands. My beautiful daughter has a complex and very significant combination of psychosocial disability and physical disability, and has suffered some horrific experiences. And two of my grandsons have cognitive and communication disability. I detailed my continuing and very significant concerns with the NDIS ICT systems in a Submission to the Senate Inquiry into the NDIS ICT Systems (August 2018) outlining the issues, as I believed that no other independent commentator and person with both lived experience and internal experience would be able to. In that Senate Committee submission, I indicated further detailed information and references are contained in the internal report: “Technology Authority Handover Report 9 June 2017”. Additionally, I also provided a submission to the Joint Standing Committee Inquiry into the General Issues Around the Implementation and Performance of the NDIS (Committee Report December 2020). This submission was made together with my adult daughter, who is an NDIS participant. The purpose of providing that joint submission was to illustrate the far reaching and systemic deficiencies of the NDIS for people with psychosocial disability. That submission described my daughter’s catastrophic experience in dealing with the NDIS. Her two young sons, my grandsons, also have disability. My daughter’s interaction with the NDIS has not only been tortuous for herself but has been exacerbated by the extreme anguish in dealing with the NDIS for her sons. My daughter made a detailed and harrowing personal statement in that submission, and I respectfully refer members of this Committee to her statement. To reiterate from my previous submissions, I anticipated the trauma that my daughter would face and yet even with all my detailed internal knowledge and ability to engage solicitors, I could not prevent the damaging impact of my daughter’s interaction with the NDIS. There would be perhaps few other NDIS applicants or families who would have the insight at the beginning of their journey to make such detailed documentary recordings from the outset. And yet, the systemic issues that I predicted and described in my previous submissions remain unresolved. In the application of the proposed Independent Assessments, these systemic defects present an imminent threat to people with disability, especially people with psychosocial disability. My daughter now feels sheer terror at the prospect of being forced to endure an Independent Assessment. I am grateful for the opportunity to provide this submission to the Joint Standing Committee Inquiry into the Independent Assessments under the NDIS.

  • Key Issues

The Independent Assessment process is an utterly flawed, unethical and dangerous concept on every level. It is fundamentally different to the Independent Assessment concepts envisaged by the Productivity Commission and Tune Review.

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Submission 3336

Effectively, the NDIA is proposing to undertake human research driven by an actuarial doctrine, without ethics oversight. This is verging on human experimentation. The view that this can happen and be justified on flimsy “actuarial” grounds has to be exposed and quashed.

ACTUARIAL OVER-REACH AND LACK OF CO-DESIGN

Whilst the NDIS is an insurance scheme that funds services, the NDIA as an organisation is also in the business of servicing. However, the essential capabilities of service design and co-design do not exist in the NDIA.

Together with the lack of an ethics framework, the lack of co-design and service design capabilities are the root cause of most of the NDIS operational and servicing problems.

The Committee must rectify this. The original business case funded an ongoing co-design capability to be established and built up within the agency, staffed by NDIA staff (not consultants) including staff with disability. Core to the purpose of the NDIS, co-design is necessary as an ongoing capability – not an activity to be undertaken by consultants or via periodic reference groups. As stated above, co- design is not mentioned in any of the current corporate documents. In the absence of a co-design capability and ethics framework, there is an over-reach of the actuarial function influencing NDIS systems, services and processes. “User testing” is not the same as co-design.

        The human is out of the loop: the single most significant cause of system defects and failures.

The actuary function does not provide expertise in service design. These are fundamentally different and specialised areas of expertise but equally necessary for ethical governance, and safe and ethical operations. In addition to the extraordinary and known risk inherent in commencing the intervention of Independent Assessments in the absence of an ethics framework, the statistical methodology discussed at Senate Estimates is flawed. There is no way that an unknown person – in a 20 minute interview - could comprehend the magnitude, severity and fragility of my daughter’s psychosocial and physical disability. Or any persons. Her condition so significantly fluctuates. And for that to somehow provide “consistency” is ludicrous. Further commentary on the notion of “consistency” and psychosocial factors is provided below in the section “Psychosocial”. Furthermore, the statements of the NDIA at Senate Estimates regarding the satisfaction survey following the Independent Assessment pilot program, demonstrate an appalling lack of survey design and its application in a servicing setting involving vulnerable people. Compliance, or acquiescence, is a well-known psychological response for people (and especially vulnerable people) dealing with the power of bureaucratic institutions and this compliance factor skews survey responses. People just say yes. Inclusion Australia has noted the acquiescence factor is a critical issue for people with intellectual disability, and the assessment tools do not take this into account. There is decades of international peer reviewed research on this. People feel enormous pressure; confusion; and intimidation. They do not understand the complex concepts. People are anxious and many cannot understand the nature of the questions or the significance of their responses. Not only were the number of responses not statistically significant, but the responses from participants and their families would likely be invalid due to the compliance factor. This defective survey design and misrepresentation of “findings” from pilot activities occurs because there is a lack of an ethics framework. This would have to call into question any actuarial analysis or forecasting based on such questionable “findings”.

PSYCHOSOCIAL

The NDIA has repeatedly stated that it has a “psychosocial pathway”. This is a specious statement to be vigorously challenged. The NDIA simply does not have the culture nor the capability to manage the psychosocial caseload. It beggars belief, that a government agency can baldly claim it has a “psychosocial pathway” and at the same time proceed with an intervention such as Independent Assessments – which has been shown to damage people, including suicide – in the absence of any apparent concern regarding the necessity of an ethics framework.

SubmissionSubmission 3336

This is either organisational ignorance and incompetence or wilful action that chooses to ignore the known risk of harm to people with disability. The lack of an ethics framework is evidence of the mendacious claims to a “psychosocial pathway”. I believe this is a very significant safety issue for people with psychosocial disability. It is our experience that the NDIA processes are not safe. The lack of an ethics framework and organisational disinterest in co-design is continuing evidence of this. On my daughter’s initial application, the NDIA lost her entire NDIS application - including the 400+ pages lever arch folder of medical evidence. It would be more than two years before she would be accepted into the NDIS and have a plan. As we have documented in extensive details, my daughter’s condition worsened very significantly during the whole NDIS application and review process. Not only was my daughter initially refused the supports desperately needed (due to the NDIA administrative stuff-ups) and suffered and struggled for almost two years in the cruellest way – but my daughter and her psychiatrist both questioned whether it was worth damaging her mental health even further. As her mother, and with the inside knowledge of the NDIA as to what was causing these issues for my daughter (and others), this situation was incredibly traumatic for me. Persevere and have the system damage my daughter’s mental health, or give up and have my daughter and her family denied justice. This was a sickening Faustian bargain. I anticipated the trauma that she would face and that’s why I took photos at the beginning of this nightmare journey - and yet with all my detailed internal knowledge and ability to engage solicitors - I could not prevent the damaging impact of my daughter’s interaction with the NDIS. As stated previously, there would be perhaps no other NDIS applicant or family who would have the insight at the beginning of their journey to make such documentary recordings from the outset.

SubmissionSubmission 3336

Our family experience and evidence is that over a protracted period of time, the NDIS psychosocial support team and NDIS psychosocial pathway processes - simply do not exist. Various statements in public documents on the NDIS website, in the media and in public forums - regarding the improved psychosocial “pathway” - is utterly false. If there was any truth to the statement that there was an improved psychosocial pathway, then my daughter would not have suffered this damaging and traumatic experience over a protracted period of time. “,

To reiterate our statements in previous submissions, which describe in detail my daughter’s catastrophic experiences in applying for and dealing with the NDIS, all this underscores the sheer terror that she feels at the prospect of being forced to endure an Independent Assessment.

And for what purpose? And what “safeguards” are there in place that would anticipate such adverse reactions. None. It would appear that the actuarial doctrine which has driven such systemic complexities and inconsistencies – and through which my daughter has horrifically suffered - will somehow be made “more consistent” through a 20 minute outsourced high-risk arrangement. My daughter is not an actuarial experiment.

And nor is any other participant or family. Following my daughter’s appeal and acceptance into the NDIS, her experience with the LAC for the planning process was equally traumatic and de-humanising.

This is an excerpt of our email correspondence with the LAC, detailing my daughter’s treatment at her planning meeting.

“>Today’s meeting was set up a month ago, as soon as [my daughter] received her NDIS Access Approval letter. As we mentioned in the meeting, the process of assembling the documentation, and the application and Internal Review process took more than three years and involved us engaging solicitors as the NDIS had lost [my daughter’s] application and documentation. The NDIS Access Approval Letter, stated that [my daughter] should commence preparing for the planning meeting - we have actually been planning for this meeting for a very long time, given [my daughter’s] very complex condition and range of disabilities. When [my daughter] booked today’s appointment a month ago, she asked that the meeting be with a planner with psychosocial disability experience. [My daughter] called a number of times to confirm this - most recently last Friday, when planners were changed and you were included into [my daughter’s] planning meeting. So approaching today’s meeting, [my daughter] was extremely agitated and stressed. From my perspective, given that the NDIS encourages participants to prepare for the planning meeting, our expectation is that you and [LAC organisation] should have been similarly prepared given the advanced notice of this planning meeting. You were clearly unprepared and had not read [my daughter’s] extensive file. A chat for a few hours does not cover the extent of her circumstances and needs - including safety considerations. Given the NDIS and government emphasis on better supporting people with psychosocial disability in navigating the NDIS, today’s meeting was grossly unacceptable on many levels. The whole meeting was a form filling box-ticking eercise to generate a plan at the end of this one meeting - we appreciate that you are operating under the directions of [LAC organisation], but we felt incredible pressure to push on to complete the planning meeting, even though we have been going for nearly two hours and [my daughter] was clearly unwell. Your comment that there is only one planning meeting, and we appreciate this is the [LAC organisation] approach, is inconsistent with statements made by the NDIS - that participants have the opportunity to have a number of planning meetings to ensure that their circumstances have been understood and an appropriate amount of time has been invested in developing a plan.”

SubmissionSubmission 3336

Our deep concern today with this rushed, apparently KPI driven approach, is that [my daughter’s] documentation had not been read beforehand and [my daughter] was unwell and not coping. [My daughter] was not even offered a glass of water or a break.

We appreciate that you have scheduled a second planning meeting for [my daughter] this Friday. As I mentioned, there is still a lot to go through and I would like to flag the possibility of the need for a third planning meeting after Friday. The reasons for this potential third meeting are outlined below - and I reference the attached documents which are the documents in the red folder we gave you today. A number of these documents should be on the NDIS system, but we assembled these in the red folder as a focus of the planning discussion today.

[My daughter] has requested that you read all the documents attached in this email (documents from the red folder today), before we meet again on Friday. [My daughter] also requests that you confirm if you have access to the 30 documents and medical reports referred to in the NDIS letter granting [my daughter] NDS access and read these before Friday’s meeting.”

My Daughter’s Personal Statements. From Application, Review and Planning

“This document combines three separate personal statements that [my daughter] has made over the course of the past several years, through the NDIS application and review process. The three statements have been scanned into the one document. These statements provide significant detail of [my daughter’s] life and her day to day - and [my daughter] has described in significant detail the types of supports required, against the various sections of the NDIS legislation. What [my daughter] discussed today in terms of goals and statement of supports, was not a wish list but developed using the various guides provided by the NDIS. [My daughter] requests that you read her personal statements before Friday - if you are unable to do so, [my daughter] would prefer that the meeting is rescheduled to another time to make sure that you have read the documents given this is essential to the planning process.”

And all this interaction was with a planner who the LAC organisation stated had experience in psychosocial disability as a mental health nurse. [My daughter] was forced to state over and over again the extraordinarily intimate details of her complex disability – a humiliating, dehumanising and traumatising experience - to a stranger - who had not even read her file and in a process absolutely compromised by time pressure. This is the traumatic experience which I believe will be the experience with the Independent Assessment process. The traumatic process is then re-experienced when the LAC discusses the results of the Independent Assessment at the planning meeting, but the person is not able to challenge the results of the report. Given the systemic process defects that my daughter suffered over a protracted period of time, this is a terrifying prospect for her as a person with complex psychosocial and physical disability.

The person is subjected to the trauma of the Independent Assessment with a stranger and then is forced to endure the trauma of an LAC planning exercise also with a stranger to again go over the intimate details of their life.

And with the absence of an ethics framework, what happens when things go wrong in the Independent Assessment process? This will happen. This happened to us in the LAC planning exercise as I have described above. There was no-one to help us as we faced a very grave situation caused by the process. Quite the contrary, there was pressure for the process to continue at all costs. Just “stopping” is not the answer and certainly not enough. What care and supports are provided to the person and their family? The whole person must be considered and this takes time – time clearly that the outsourced LAC planning arrangements do not provide for. And time that a contracted “Independent Assessment” also does not provide for. And for people with complex disabilities and co-morbidities, the theoretical notion of a “disability agnostic” process runs counter to the multi- faceted determinants of the “whole person”.

This “disability agnostic” concept appears to be a continuation of the actuarial fiction of “primary disability” that has created so many problems for people with disability in accessing the NDIS and being provided with the necessary supports.

                                                   8

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

SubmissionSubmission 3336

There appears to be no effective control framework to safeguard data integrity nor of the many processes - such as opt-in processes, document management, and personal communication with participants – that are dependent on it. Together, the lack of an ethics framework and lack of co-design create the conditions for this to happen.

  • NDIS Outward Bound Calls

The outward bound call regime further demonstrates how detached the NDIA communication ethos is from the needs and circumstances of people with disability they are serving. Calls that have no identification number. Scripts that immediately ask about a participant. As with the scam-like “invitation” text messages, these are the type of phone scams that lure people into divulging personal banking and financial information. Furthermore, the following examples show how the instructions given by participants about how they need to be communicated with are consistently ignored or mismanaged. In the first example below, the commentary describes the experience of people with brain injury and stroke survivors being subjected to complex scripted outward bound recruitment calls about the Independent Assessments. The experience was described as “overwhelming”. Extensive commentary in online forums and in other submissions, indicates that this is a common experience for people living with psychosocial disability, cognitive impairment and intellectual disability. Of great concern, not only does this person’s commentary point to fundamental issues of lack of accessibility – overwhelming content, long and complex web addresses – but the overwhelming experience that potentially deprives people of the ability to advocate for themselves. People’s human agency is compromised. In the second example, a hearing impaired person describes the continuing communication problems of receiving phone calls and voice mail messages from the agency – that they cannot hear – even though this person has requested communication via email. In the third example, the discussion continues about a blind person receiving a written letter about the Independent Assessment. People are concerned that they will be exited from the scheme because they miss responding to the Independent Assessment invitation. These are not isolated instances. To the contrary, these are widespread.

This is systemic discrimination that undermines the rights of people with disability by the very agency that should be an exemplar.

     Two years of outsourcing has systematically entrenched discrimination on the basis of disability.

I pointed this exact problem out more than two years ago in my submission to the JSCNDIS Inquiry into the NDIS Systems (August 2018. I stated:

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

Current Scheme Implementation and Forecasting for the NDIS

Submission 36

SubmissionSubmission 3336

BIOGRAPHY: MARIE JOHNSON

Marie Johnson is the CEO of the Centre for Digital Business. She is an accessibility advocate, and a recognized global speaker and commentator on artificial intelligence; technology; digital services; ehealth; cyber; identity; biometrics; and innovation.

An internationally experienced entrepreneur, Marie has an unparalleled track record across the public and private sector delivering significant technology, innovation and digital services transformation programs encompassing revenue, business, social services, payments, identity, immigration visa operations and disability services. The diversity of roles covers service delivery operations, global technology industry strategy, Chief Information Officer, Chief Technology Architect, board director and advisor. Marie was the Head of the Technology Authority of the National Disability Insurance Agency.

Marie conceived and led the global co-design effort with people with disability to deliver “Nadia” the first AI powered digital human for service delivery. Grounded in human rights, this work sparked a global industry and appetite for AI powered digital humans.

Following Nadia, Marie designed and created the artificial intelligent “Digital Human Cardiac Coach”, introduced at the global Singularity University Exponential Medicine Conference in San Diego November 2019.

The strategic framework “Co-Design for AI ©” which she developed, embeds Human Rights in service design.

In addition to large scale service delivery operations, Marie has led the strategy and implementation of significant reform programs across the digital machinery of government:

  • Australian Business Number (ABN) registration in joint task force with the ATO.
  • The Business Entry Point, initiative of the three levels of government.
  • Chief Technology Architect for the $1Billion Health and Human Services Access Card programme.
  • Collaboration with the Reserve Bank of Australia on innovation in payments and information services initiating Payment Delivery Reform.
  • Service Delivery Reform technology business cases bringing together Centrelink, Medicare Australia and Child Support.
  • Developed and delivered the $700 million Visa Pricing Transformation (VPT) programme; and delivery of the Global eMedical system to 100 countries in partnership with Citizenship and Immigration Canada at Department of Immigration and Citizenship (DIAC).

Marie was influential in the Australian Financial Systems Inquiry, which recommended the need for a federated digital identity framework as a critical element for the future robustness of Australia’s financial systems.

The egovernment and digital initiatives Marie has led have been also been recognised globally.

These include the United Nations Public Service Award in the category “Application of ICT in government: egovernment” for the Business Entry Point (www.business.gov.au) which she led for 5 years.

In 2005, the US Government awarded Marie an O-1 Visa (Individuals with Extraordinary Ability or Achievement) to take up the role leading Microsoft’s Worldwide Public Services and eGovernment business, including Microsoft’s Identity Strategy in Government. Microsoft noted Marie’s egovernment knowledge “…is unique in the world and is of particular interest to Microsoft as we pursue our egovernment strategies”.

In 2006–2007, Marie was named “Innovative CIO of the Year – Australia”. In 2013, Marie was named one of Australia’s “100 Women of Influence”. In 2019, was awarded the “Exceptional Woman of Excellence” at the Women Economic Forum (WEF) in Perth. WEF is the largest gathering of women entrepreneurs & leaders worldwide.

For many years, Marie was an invited member of the Accenture Global CIO Advisory Council; an Independent Member of the Australian Federal Police Spectrum Programme Board; and an elected National Board Director of the Australian Information Industry Association. In 2019, Marie was faculty at Singularity University Exponential Medicine (San Diego).

Marie served as an Inaugural member of the Australian National University (ANU) Cyber Institute Advisory Board.

In 2020, Marie delivered the 2020 Kenneth Jenkins Oration to the National Disability Services National Conference.

In 2021, Marie is the Patron of the Tech Girls Movement Foundation. www.techgirlsmovement.org

Qualifications

MBA (Melbourne Business School); Bachelor of Arts; Harvard University Kennedy School of Government Senior Executive Fellows Program; and Graduate of Australian Institute of Company Directors

More Information: www.marie-johnson.com

Submission Submission 3336

REFERENCES

21

JOINT STANDING COMMITTEE ON THE NATIONAL DISABILITY INSURANCE SCHEME

SENATE INQUIRY – INDEPENDENT ASSESSMENTS

ADDITIONAL STATEMENT:

Marie Johnson CEO Centre for Digital Business Pty Limited ABN: 16 162 122 072

30 April 2021

1

Submission 33 - Supplementary

INTRODUCTION

My name is Marie Johnson and I am the CEO of the Centre for Digital Business, a digital services and artificial intelligence company. I would like to sincerely thank the Committee for the opportunity to provide further evidence building on my submission (submission number 33) into the Independent Assessments and previous submissions. I am drawing on my somewhat unique experience: of lived experience in addition to my deep internal knowledge and experience as former Head of the NDIS Technology Authority. I wrote the business case for the NDIS ICT systems, and for this to be based on co-design and the principles of the UN Convention on the Rights of Persons with Disabilities. I have an exceptionally deep knowledge of the NDIS processes, and the NDIS ICT system which were delivered by DHS. I have a deep understanding of NDIS capability, operating model and culture. I also have considerable operational knowledge of the cross government capability, systems and architecture on which the NDIS is dependent. Perhaps somewhat unique, I believe there would be very few people with this deep level of direct internal knowledge of and experience in the NDIS operating environment, in addition to the lived experience of interacting with the NDIS. I have disability in my family: my husband, daughter and grandsons. My beautiful daughter has a complex and very significant combination of psychosocial disability and physical disability, and has suffered some horrific experiences.

I also provided a submission to the Joint Standing Committee Inquiry into the General Issues Around the Implementation and Performance of the NDIS (Committee Report December 2020). This submission was made together with my adult daughter, who is an NDIS participant. The purpose of providing that joint submission was to illustrate the far reaching and systemic deficiencies of the NDIS for people with psychosocial disability. The submission described my daughter’s catastrophic experience in dealing with the NDIS. Her two young sons, my grandsons, also have disability. My daughter’s interaction with the NDIS has not only been tortuous for herself but has been exacerbated by the extreme anguish in dealing with the NDIS for her sons. My daughter made a detailed and harrowing personal statement in that submission, and I respectfully refer members of this Committee to her statement. To reiterate from my previous submissions, I anticipated the trauma that my daughter would face and yet even with all my detailed internal knowledge and ability to engage solicitors, I could not prevent the damaging impact of my daughter’s interaction with the NDIS. There would be perhaps few other NDIS applicants or families who would have the insight at the beginning of their journey to make such detailed documentary recordings from the outset. And yet, the systemic issues that I predicted and described in my previous submissions remain unresolved. In the application of the proposed Independent Assessments, these systemic defects present an imminent threat to people with disability, especially people with psychosocial disability. My daughter now feels sheer terror at the prospect of being forced to endure an Independent Assessment. Other participants also now express the feeling of terror at what is before them.

KEY ISSUES

Building on the issues detailed in my submission (Submission number 33) to this inquiry I would like to cover the following issues which fundamentally affect the operation of the NDIS and the emergent changes which on my experience, present an unacceptable risk to participants and will cause the NDIS to implode. The issues I will cover are:

  • NDIA Defective Systems and Processes: Additional Evidence
  • The Bias of the Doctrine of “Fairness”
  • The Risk of Algorithms in Combination with the Bias of “Fairness”
  • Assistive Technology: Paucity of Actuarial Analysis and Agency Capability

DEFECTIVE SYSTEMS AND PROCESSES: ADDITIONAL EVIDENCE

Lost documents – 80% In my submission to this Inquiry and in previous submissions, I have provided detailed evidence describing the defective NDIS systems and processes. The widespread systemic and worsening issue of lost documents is a catastrophic symptom of the defective systems and processes. On my daughter’s initial application, the NDIA lost her entire NDIS application - including the 400+ pages lever arch folder of medical evidence. In addition to the photos we took at lodgement, we maintained a detailed chronology and analysis of the various interactions with the NDIS. This chronology shows that on TWELVE times, information was provided/re-sent/re-requested. The NDIS admits to not having all the documents, and inconsistencies on the part of the NDIS as to what they had and when they received it. In total, more than 30 medical reports and assessments have been provided to the NDIA over a period of 22 months as part of a seemingly never-ending process of application and review, describing in extensive detail, the diagnoses and impacts of my daughter’s long-standing, complex, significant and permanent psychosocial disability. It would be more than two years before she would be accepted into the NDIS and have a plan. As we have documented in extensive details, my daughter’s condition worsened very significantly during the whole NDIS application and review process. Not only was my daughter initially refused the supports desperately needed (due to the NDIA administrative stuff-ups) and suffered and struggled for almost two years in the cruellest way – but my daughter and her psychiatrist both questioned whether it was worth damaging her mental health even further. As her mother, and with the inside knowledge of the NDIA as to what was causing these issues for my daughter (and others), this situation was incredibly traumatic for me. Persevere and have the system damage my daughter’s mental health, or give up and have my daughter and her family denied justice. This was a sickening Faustian bargain. I anticipated the trauma that she would face and that’s why I took photos at the beginning of this nightmare journey - and yet with all my detailed internal knowledge and ability to engage solicitors - I could not prevent the damaging impact of my daughter’s interaction with the NDIS.

Submission 33Submission- Supplementary36 Submission

Submission 33Submission- Supplementary36 Submission

Submission 33Submission- Supplementary36 Submission

Note: it was because of my detailed knowledge of the “participant journey”, that I had the misery of knowing exactly what my daughter was in for.

THE BIAS OF THE DOCTRINE OF “FAIRNESS”

The concept of “fairness”, which has made its way into NDIA internal communication and communication with participants needs to be challenged as a biased and discriminatory concept. As elaborated in detail in my submission, the lack of an ethics framework creates the very conditions for such problematic concepts to become normalised. It is not fair that 80% of people’s documents get lost: documents that are of the most personal nature, revealing peoples most private and sensitive thoughts. It is not fair that people get denied access because their documents are lost: as happened with my daughter and many other people. It is not fair that people wait two years for a wheelchair and when it arrives it is no longer suitable. It is not fair that the NDIA is not a Model Litigant And it’s not fair that people live in fear and terror of dealing with the NDIA and losing their supports.

The Doctrine of “Fairness” and Indigenous Communities

In June 2018, a report was prepared for the Department of the Prime Minister and Cabinet, on the “NDIS East Arnhem Co-Design Project: Evaluation Report.” It is a disgraceful read of the failure of the implementation of the NDIS in remote Indigenous Communities. This Committee has heard first hand evidence of that. The NDIS East Arnhem Co-Design Project: Evaluation Report found that: “Stakeholders agree that the Co-Design project is incorrectly named as it was designed in the National Disability Insurance Agency (NDIA) head office without their input.”[Emphasis added]

   "The lack of resources about the NDIA in Language was reported by 12 of fourteen service
    providers to have significantly hindered the ability of East Arnhem communities to understand
  and engage with the NDIS."

      "It is unrealistic for individuals to work out how to access service providers, how to get on the
    portal etc. and draw down on funds, and navigate all the logistics of the scheme to manage
     their plans. Even businesses can’t navigate it yet."

And so it is not fair that co-design has not occurred at all and not with Indigenous peoples. This is active discrimination. It means that the language barrier is a major determinant / obstacle to Indigenous people’s understanding in context and access to services.

Plan Value to Reduce Over Time

It is not fair that the NDIA tells participants that they should expect their overall plan value to reduce over time. This statement is utterly absurd. And mendacious. I for one would like to see the analysis this statement is based on – and any assumptions – open for public review and debate. The impact of people’s disability changes and deteriorates over time. People’s parents die. Children grow. People don’t grow new legs. And the condition of people with psychosocial disability fluctuates. This is akin to saying that people’s use of and dependence on Medicare reduces over time.

Submission 33Submission- Supplementary36 Submission

Submission 33Submission- Supplementary36 Submission

In the UK Post Office case, it has been recently reported that after fighting for decades, 39 people are having their convictions over-turned in what is being reported as the largest miscarriage of justice the UK has ever seen. Society is only at the beginning of the era of algorithms, and yet we see the horrific impact of the triangulation of these three factors. In the case of the Independent Assessments and broader changes being prosecuted by the NDIA, these three factors – algorithms, the doctrine of “fairness” and the reverse onus of proof – are being applied intentionally as part of a strategy targeting a group within the community: people with disability.

   This is not inadvertence. This is systemic discrimination by design and it will harm people.

I would like to re-emphasise here my commentary in media reporting through InnovationAus: “What they’re saying is ‘we have to make it fair’, but one of the problems with algorithms is the bias that’s built into algorithms. What we’re going to be seeing here is a bureaucratic notion of fairness constructing the algorithm without any visibility or transparency, or co-design about what the algorithm actually says.” “Whose notion of fairness are they actually talking about? This is going to create enormous problems.” The widespread application of algorithms changes the relationship between the citizen and the state, creating an enormously disproportionate power imbalance. The doctrine of “fairness” shapes opaque algorithms that enable policies of the reverse onus-of-proof and non-appealable processes targeting people with disability.

ASSISTIVE TECHNOLOGY: PAUCITY OF ACTUARIAL ANALYSIS & AGENCY CAPABILITY

I would like to address a number of serious factors around Assistive Technology which directly impact Scheme sustainability. These factors appear to not have been taken into account in the narrative around sustainability and this would appear to be a significant omission that would distort the forward analysis of Scheme sustainability. The first factor is looking at the size of the AT market, the transformative impact of AT on jobs and the NDIS, and the astonishing lack of analysis regarding AT: an absence of analysis that must call into question the agency’s prosecution of the narrative around sustainability. The second factor as part of reform, is the need to create a culture of ROI on the beneficial adoption of AT, and for this to be a factor in the analysis of Scheme sustainability.

The AT market

In November 2017, in my then capacity as a National Board Director of the Australian Information Industry Association (AIIA), I appeared before the Senate Committee on the Delivery of Outcomes Under the National Disability Strategy 2010-2020. The estimated $40 billion assistive technology market over the next 10 years is of enormous interest and significance to the Australian technology and innovation sectors, and the Australian economy more broadly. If, as the Productivity Commission has estimated, the spending on human services is expected to reach $300 billion per annum, the Assistive Technology (AT) market could potentially be around $4 billion per annum.

Submission 33

Supplementary

A much larger market than the $1.06 billion per annum estimated in the 2015 NDIS assistive technology strategy. And that $40 billion over 10 years is just the Australian market. Emerging from global R&D efforts underway, breakthroughs in computing power and design are driving the parallel convergence of technology solutions for disabled and ageing populations, with considerable mutual benefit. And universally, innovations emerging from and for these populations, will benefit all people who in everyday life experience functional or situational disability. Commercially, this means that improved features, functionalities and user design are beneficial to all consumers universally. Accessibility is THE most significant global commercial opportunity. And this is why I had proposed the establishment of a “Council of Advanced Innovation” for extreme accessibility in the business case, through which Australia could shape, influence and engage local and global research and development, in standards and technology innovations.

Paucity of Analysis

So given the massive AT market there is a paucity of analysis by the NDIA. Deep analysis and future casting is urgently needed, to provide essential insight for the sector, on the shifts and possible futures of the AT and innovation industry. Without this, of what use is financial forecasting on Scheme sustainability? In a number of Submissions, the Australian Rehabilitation and Assistive Technology Association (ARATA) highlighted two critical points. Firstly, there is insufficient rigorous research evidence regarding assistive technology. And secondly, there are currently systemic deficiencies and a culture against presenting an ROI of NDIS funded AT over time. Specifically ARATA emphasised the need for…

“> …methods to create a culture of selection of AT based on ROI.”

And this AT market research analysis needs to be linked to not only a future view of the AT market for products and services – but what the future jobs market looks like. Exponential technologies are re-shaping what we understand to be “assistive technology” and the inter-relationship with and the nature of jobs. What does the rapid adoption of exponential technologies mean for the massive Australian AT market and the disability and aged care workforces? The dilemma for the fragile aged care system and disability services, is that whilst these are the epicentre of the “jobs growth”, more of the same will not fix the problem. For years, government, industry and research bodies have produced reports looking at “future jobs” and areas of growth and gaps. Consistently, the two areas with the biggest projected employment growth are health care and social assistance - 38 per cent employment growth in these two categories alone. Various reports estimate that the health and disability care sectors are driving a voracious demand for 70,000 additional jobs in the next few years. But 70,000 of the same jobs of today – for the future – is a meaningless statement.

Submission 33Submission- Supplementary36 Submission

So the question is, how is Scheme sustainability a function of this figure in combination with assistive technology?

          We don’t know, because the analysis has not been done.

What we do know is that there are 3 billion people globally engaging with the massive online gaming platforms, and these environments, platforms and technologies are rapidly becoming mainstream support and therapy services. That is, these technologies augment jobs, therapies and services. Are the 70,000 forecasted Australian jobs and people ready for that world? The 2016 Senate Inquiry into The Future of Australia’s Video Game Development Industry acknowledged the growing demand for gaming innovations in health, education and training. But even with the Government’s National Innovation and Science Agenda, the Senate Inquiry lamented:

   “…it is not apparent that the Government has given explicit attention to interactive game development and the potential for Australia’s future that this industry could provide.”

Assistive communication tools and immersive technologies based on gaming platforms are in clinical and consumer use for behaviour adaptation and sensory therapies. Easy-to-use at home devices already exist, Internet of Things, sensors and companies like Apple continue to add health and safety monitoring capabilities to their smartwatches. In my evidence to the Senate Committee on the Delivery of Outcomes Under the National Disability Strategy 2010-2020, I spoke about an actual example of a physiotherapist consulting with patients on the other side of the country by using a digitally connected exoskeleton. This would be a life changing servicing innovation for many people and create new domestic and export therapeutic services. ROI in this case is not a like-for-like comparison between a wheelchair and exoskeleton. An exoskeleton does not replace a wheelchair: the combined augmented life-long benefits were documented across all dimensions of life. This actual example from an exoskeleton NDIS provider demonstrated the human impact, time and cost of proving ROI involved in introducing new servicing innovations for NDIS participants. But ROI is not accepted which must undermine the concept of value for money. And because of the paucity of analysis of AT, the impact of such innovations on Scheme sustainability and economic benefits is not known. The article “Second Best” by a former Senior Local Area Coordinator (LAC) also spoke about the NDIA’s resistant culture regarding the investment benefit of AT innovations. [REFERENCE: https://therebuttal2.com/2021/04/26/second-best/] I would add to questions around culture, there is a significantly capability deficit within the NDIA regarding knowledge about AT innovations. As I mentioned previously, the reason why this is important is that advances in AT innovations not only support independence and are an investment over time, but these shape the nature of jobs. All these factors are determinants of Scheme sustainability. In the “Second Best” article, the former LAC described the situation of a woman who had a lower limb amputation. This woman needed to upgrade her prosthetic as it was old and often fell off. The woman had numerous reports from various OTs regarding a prosthetic that had a microprocessor as this was more appropriate.

Submission

Supplementary36 Submission

The NDIS wouldn’t approve it. They wanted this woman to have a cheaper fixed type of prosthetic which was not suitable for her circumstances. The article stated that whoever the NDIA expert was didn’t feel the benefits of the microprocessor were enough to justify the cost. The participant was virtually housebound for two years because the prosthetic that she had was unsafe. In another example, the “Second Best” article talks about the situation of hearing impaired people who have been fighting with the NDIA over visual alert systems such as “Visualert” and haptic alert systems. In Australia smoke alarm legislation is very strict. For hearing people the alarm must be heard from wherever they are in the home. The “Second Best” article offered that hearing impaired people would need a similar system. OTs and audiologists around Australia have being recommending visual alert systems because it keeps hearing impaired people safe. According to the “Second Best” article, the NDIS is refusing these. Instead funding only cheaper systems that rely on batteries, pagers and WiFi. These systems do not meet the same strict safety requirements that hearing systems must meet. The “Second Best” article goes on to observe:

“Governments are always happy to spend billions of dollars on roads because it creates jobs and stimulates the economy. For some odd reason when it comes to investing in a similar way in people with a disability it’s no longer an investment.” As ARATA stated, creating a culture of ROI – not just “reasonable and necessary” - is absolutely necessary to fully leverage the $40 billion AT market to transform the jobs and skills market. Can you imagine robots as support workers, monitors and companions in people’s homes and in supported accommodation? The robotics industry is undergoing what has been described as a “…Cambrian explosion of growth…”. Can you imagine a robot training a person with disability, to perform work remotely. And for this to be funded as capacity building as part of a person’s NDIS package. The result is that a host of jobs that seemed out of reach for remote work are likely to be firmly in the remote-work orbit within the next 10 years. And people with disability, as avid gamers and experienced users and developers of these technologies, could translate these skills into remote work opportunities. And even the most fundamental of human rights and basic human care - for an incontinent person to be kept clean – is a domain of radical innovation. Innovation almost impossible to imagine, given the reports from the Royal Commissions of the appalling rationing of incontinence garments. The 2020 Report of the “Global Incontinence Products Industry” projects that the global market for incontinence care products will reach US$17 billion by 2025, with product innovation driven by RFID and sensor wafer chips. Sensors will detect when the diaper has been soiled, as well as monitor body temperature, detect abnormalities in urine composition, and even help prevent bed sores by tracking how long it’s been since a person has moved. Could smart diaper data, sensors and data analytics become part of a quality and safeguard framework. Think about what this would mean for the skills and management of attendant care staff.

Submission 36 - Supplementary

And while there is an urgent need for discussion about jobs as part of Scheme sustainability, this discussion is incomplete without a discussion on the rapidly changing nature of work, an augmented services and care ecosystem of AI, robotics, immersive technologies, sensing and remote servicing. Given the pace of change, these exponential technologies provide a new paradigm for rethinking what we mean by unmeetable demand and sustainability. So given the changing mix of presence, scale and exponential reach in terms of time and location, I wonder how the definitions of regional, remote and very remote might need to adapt so as not to unintentionally limit innovation. Or unnecessarily exacerbate or cause “thin markets”. So once again, how can an actuarial perspective on Scheme sustainability not take into account the advances in and participant investment funding for AT innovations? The original PC Report recognised that innovation was absolutely necessary for Scheme sustainability.

It is unfathomable that the agency and the government is driving a narrative on Scheme sustainability, when the actuarial forecasting itself lacks any analysis on the adoption of AT and exponential technologies.

SUMMARY

We all accept that the NDIS needs reform and in my Submissions I have detailed specific areas where reform is urgently needed so that people do not continue to suffer harm and disadvantage. Like Medicare, NDIS covers all Australians. I believe that all Australians should be gravely concerned that the flawed Independent Assessments and related legislative changes have progressed so far, without oversight - measures that have been shown to damage people. Reform starts with the internal operations of the NDIA – including the actuarial function – and legislative amendments to include a legislated co-design framework and a legislated ethics framework. Transparency is essential. A prominent legal advocate in administrative law, Darren O’Donovan recently offered a model upon which reform might be based:

“We seriously need the domestic version of a Peace Summit for the NDIS. Public submissions, actuarial modelling public, software for building plans public. Appoint a trained arbitrator and publish all minutes.”

The disability community has long fought to end the institutionalisation of disabled people: this is the promise of the NDIS. The changes that are being advanced by the Government and NDIA will have the effect of imposing a digitised automated form of institutionalisation and discrimination.

       The first principle in any government servicing must be: "first, do no harm".

With this first principle of an ethics framework, beneficial co-designed reform can take place.

Marie Johnson CEO Centre for Digital Business Pty Limited

30 April 2021

                                       12

Submission 36

BIOGRAPHY: MARIE JOHNSON

Marie Johnson is the CEO of the Centre for Digital Business. She is an accessibility advocate, and a recognized global speaker and commentator on artificial intelligence; technology; digital services; ehealth; cyber; identity; biometrics; and innovation.

An internationally experienced entrepreneur, Marie has an unparalleled track record across the public and private sector delivering significant technology, innovation and digital services transformation programs encompassing revenue, business, social services, payments, identity, immigration visa operations and disability services. The diversity of roles covers service delivery operations, global technology industry strategy, Chief Information Officer, Chief Technology Architect, board director and advisor. Marie was the Head of the Technology Authority of the National Disability Insurance Agency.

Marie conceived and led the global co-design effort with people with disability to deliver “Nadia” the first AI powered digital human for service delivery. Grounded in human rights, this work sparked a global industry and appetite for AI powered digital humans.

Following Nadia, Marie designed and created the artificial intelligent “Digital Human Cardiac Coach”, introduced at the global Singularity University Exponential Medicine Conference in San Diego November 2019.

The strategic framework “Co-Design for AI ©” which she developed, embeds Human Rights in service design.

In addition to large scale service delivery operations, Marie has led the strategy and implementation of significant reform programs across the digital machinery of government:

  • Australian Business Number (ABN) registration in joint task force with the ATO.
  • The Business Entry Point, initiative of the three levels of government.
  • Chief Technology Architect for the $1Billion Health and Human Services Access Card programme.
  • Collaboration with the Reserve Bank of Australia on innovation in payments and information services initiating Payment Delivery Reform.
  • Service Delivery Reform technology business cases bringing together Centrelink, Medicare Australia and Child Support.
  • Developed and delivered the $700 million Visa Pricing Transformation (VPT) programme; and delivery of the Global eMedical system to 100 countries in partnership with Citizenship and Immigration Canada at Department of Immigration and Citizenship (DIAC).

Marie was influential in the Australian Financial Systems Inquiry, which recommended the need for a federated digital identity framework as a critical element for the future robustness of Australia’s financial systems.

The egovernment and digital initiatives Marie has led have been also been recognised globally.

These include the United Nations Public Service Award in the category “Application of ICT in government: egovernment” for the Business Entry Point (www.business.gov.au) which she led for 5 years.

In 2005, the US Government awarded Marie an O-1 Visa (Individuals with Extraordinary Ability or Achievement) to take up the role leading Microsoft’s Worldwide Public Services and eGovernment business, including Microsoft’s Identity Strategy in Government. Microsoft noted Marie’s egovernment knowledge “…is unique in the world and is of particular interest to Microsoft as we pursue our egovernment strategies”.

In 2006–2007, Marie was named “Innovative CIO of the Year – Australia”. In 2013, Marie was named one of Australia’s “100 Women of Influence”. In 2019, was awarded the “Exceptional Woman of Excellence” at the Women Economic Forum (WEF) in Perth. WEF is the largest gathering of women entrepreneurs & leaders worldwide.

For many years, Marie was an invited member of the Accenture Global CIO Advisory Council; an Independent Member of the Australian Federal Police Spectrum Programme Board; and an elected National Board Director of the Australian Information Industry Association. In 2019, Marie was faculty at Singularity University Exponential Medicine (San Diego).

Marie served as an Inaugural member of the Australian National University (ANU) Cyber Institute Advisory Board.

In 2020, Marie delivered the 2020 Kenneth Jenkins Oration to the National Disability Services National Conference.

In 2021, Marie is the Patron of the Tech Girls Movement Foundation. www.techgirlsmovement.org

Qualifications

MBA (Melbourne Business School); Bachelor of Arts; Harvard University Kennedy School of Government Senior Executive Fellows Program; and Graduate of Australian Institute of Company Directors

More Information: www.marie-johnson.com

Submission 33

Submission- Supplementary36 Submission

REFERENCES

Article: “No Fairness in the Harm Caused by Algorithms” https://www.innovationaus.com/no-fairness-in-the-harm-caused-by-algorithms/

Article: “Second Best” https://therebuttal2.com/2021/04/26/second-best/

Article: “Not just A-levels: unfair algorithms are being used to make all sorts of government decisions” https://theconversation.com/not-just-a-levels-unfair-algorithms-are-being-used-to-make-all-sorts-of- government-decisions-145138

Article: “Bad software sent postal workers to jail, because no one wanted to admit it could be wrong” https://www.theverge.com/2021/4/23/22399721/uk-post-office-software-bug-criminal-convictions- overturned

‘Robo-planning’ will ‘blow-up’ NDIS: key architect https://www.innovationaus.com/robo-planning-will-blow-up-ndis-key-architect/


                                           14

<!-- source-release-page: 39 -->

<a id="source-page-39"></a>

## Source release page 39

Submission 33Submission- Supplementary36   Submission