DOCUMENT 2

OFFICIAL: SENSITIVE

Strategic Leadership Team Meeting of 23 April 2024

The contents of this document are OFFICIAL: SENSITIVE

Agenda Item: #

Agenda Item:

Paper Type: For Noting

SLT Sponsor: Sam Porter, Chief Operating Officer

1. Purpose

1.1. That the Strategic Leadership Team note the National Disability Insurance Agency’s (NDIA) Copilot for Microsoft 365 trial.

2. Key Insights

2.1. The NDIA is participating in a Digital Transformation Agency (DTA) led pilot of Copilot for Microsoft 365 (a generative Artificial Intelligence (AI) product), with up to 300 licences available for Agency employees.

2.2. Copilot for Microsoft 365 (Copilot) is an AI/virtual assistant able to automate tasks, drafts content, take minutes of meetings and summarise documents. It uses a staff member’s access to existing Microsoft products as its source data. It does not access PACE or other non-Microsoft products.

2.3. As part of this trial, Chief Information Officer (CIO) staff are currently testing prompts (requests) and use cases for Copilot to determine the best value and application of this technology for the Agency. This will be progressed further with the establishment of an Agency working group.

2.4. All users are informed that they are responsible for reviewing any output from CoPilot, noting that this software is not a decision maker.

3. Generative AI

3.1. Generative AI a sub-field of AI capable of generating text, images or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics.

3.2. By automating repetitive tasks, generative AI can free up time for more strategic and creative work, helping organisations to ramp up their productivity and do more with less.

3.3. Trust and transparency are crucial for generative AI to work effectively. As an inevitable change approaches, it is important that the Agency takes the initiative to lead by adopting a strategic approach to AI implementation.

3.4. The NDIA does not currently have any formal policies regarding the use of AI, but has adopted the interim guidance on government use of public generative AI tools issued by the DTA in July 2023 .

3.5. The OCIO is establishing an internal working group to consider the implications of AI in the Agency. This group will have representatives from across the Agency and will work to ensure that our approach to AI is well-informed, strategic, and aligns with whole-of-

OFFICIAL: SENSITIVE

X.X.1 Page 19 of 189

OFFICIAL: SENSITIVE

Strategic Leadership Team Meeting of 23 April 2024

government policy (including the work of the AI in Government Taskforce, currently being led by the DTA).

3.6. The working group will bring a range of expertise, perspectives and will play a crucial role in shaping the future of AI within the Agency and ensuring that we are well-positioned to take advantage of the opportunities it presents.

4. Copilot for Microsoft 365

4.1. The NDIA is trialling Copilot for Microsoft 365 from January to June 2024 to safely leverage AI to streamline workflows and drive efficiency.

4.2. The Prime Minister announced the trial on 16 November 2023, and other federal agencies, including the Australian Taxation Office (ATO), Department of Foreign Affairs and Trade (DFAT), and Australian Bureau of Statistics (ABS), are also participating.

4.3. The use of Copilot varies across the APS, reflecting the different operating environments, data and risks.

4.4. The trial will consist of 300 NDIA staff, including SES and their support staff, members of the Employee Disability Network (EDN), and OCIO technical support teams.

4.5. The trial is scheduled to end in June 2024 and feedback will be used to shape the future of the Agency’s work environment.

4.6. Copilot is a transformative tool designed to enhance productivity and collaboration, but it is not a mechanism for making Agency decisions. Copilot can assist with a variety of tasks, including:

• summarising important documents and catching up on communications
• generating drafts of emails and documents
• searching documents for answers to key questions, and
• creating or writing different varieties of content.

4.7. Copilot is an AI assistant designed to be reactive to prompts, boost productivity by drafting content and help find answers. It is not intended to replace human intelligence or work independently, and cannot produce content or make decisions independently of users.

4.8. Personal privacy and ethical use of NDIA data is at the core of the trial, and all trial members are asked not to share personal details or use Copilot to analyse participant plans or any NDIA information that could be in breach of the Privacy Act.

4.9. Feedback from the NDIA will be supplied to the DTA.

4.10. A factsheet and training resources are available to staff to support the trial including:

• CoPilot for Microsoft 365 trial NDIA Factsheet (Attachment A),
• CoPilot for M365 Trial Training Modul (Attachment B),
• CoPilot Communications and Engagement Plan (Attachment D).

4.11. The Agency is currently drafting a Use of AI policy for SLT approval, leveraging the resources and approach published by the DTA. The policy will establish a framework for

OFFICIAL: SENSITIVE

X.X.2 Page 20 of 189

OFFICIAL: SENSITIVE

Strategic Leadership Team Meeting of 23 April 2024

ethical, legal, and effective use of AI, Generative AI, and Machine Learning (ML) within the Agency, focusing on applications within the NDIA. It aims to harness AI’s potential in enhancing services while ensuring responsible and ethical usage.

4.12. In developing the Use of AI policy, the Agency will consult with industry experts on best practices and emerging trends in AI and ML. The Agency has received advice from Gartner, a leading research and advisory company, on how to align the policy with the principles of trustworthy AI and ensure compliance with relevant standards and regulations (see Attachment C).

5. Risk and Sensitivities

5.1. The Agency has previously banned the use of ChatGPT and other public generative AI applications due to concerns about the privacy, security and integrity implications associated with their use. These risks either do not arise, or are mitigated, through the Copilot trial, because:

• All CoPilot data is stored in Australia, in an appropriately secured data centre.
• All prompts stay within the NDIA’s CoPilot environment and are not accessed or used by Microsoft.
• CoPilot’s access to information is based on the user’s preexisting permissions in Microsoft products.
• NDIA staff are granted access to SharePoint and other products based on their role’s requirements (job type, Branch etc). These permissions restrict CoPilot to only what the user is already authorised to access.
• The CoPilot project team is working with Legal to develop a CoPilot Privacy Impact Assessment to further assess any unknown privacy impacts.

5.2. The Agency will also put the following guardrails in place to ensure that trial participants effectively and responsibly use Copilot. These will include:

• The NDIA AI Ethical Guidance and Interim Policy are clear about the responsibilities of users when using these tools. This includes clear advice about individual responsibility on sending/using all AI generated content. This information is provided in different formats including LEAP training, which is mandatory for CoPilot users.
• The CIO is responsible for authorising all CoPilot user licences. The project team (initially 25 people) were authorised on the basis that they will provide scoping, identification of capabilities and assessment of CoPilot.
• The full use of licences (300) will include all SES and their support staff. The new users will be given the existing training and support from the Project Team.

6. Consultation

6.1. The Office of the Chief Information Officer has engaged with the Agency’s Legal Services Division, Cyber Security and broader ICT advice on the Copilot trial.

6.2. The Agency will continue to engage with the DTA and broader APS as the trial progresses.

OFFICIAL: SENSITIVE

X.X.3 Page 21 of 189

OFFICIAL: SENSITIVE

Strategic Leadership Team Meeting of 23 April 2024

7. Financial implications

7.1. The trial of CoPilot incurs minimal cost for the Agency, with the project and ICT costs being absorbed by OCIO.

7.2. A decision to permanently use Copilot is likely to lead to increased Agency costs in; licences, user support, integration into our environment.

8. Attachments Attachment A: Copilot for Microsoft 365 trial NDIA Factsheet Attachment B: Copilot for Microsoft 365 trial - Mandatory Training Module Attachment C: Generative AI: 4 Decisions to Make When Creating a Policy (Gartner)

Attachment D: CoPilot Communications and Engagement Plan

OFFICIAL: SENSITIVE

X.X.4 Page 22 of 189

OFFICIAL FOI 24/25-1356 - DISCLOSURE LOG DOCUMENT 2.1

Delivered by the National Disability Insurance Agency

Copilot for Microsoft 365 trial Factsheet

The Australian Government has announced a 6-month trial of Copilot for Microsoft 365 across the Australian Public Service (APS) commencing early 2024 to explore the safe, responsible and innovative use of generative Artificial Intelligence (AI).

About the trial

The trial will be administered by the Digital Transformation Agency (DTA) and will run from January to June 2024. The DTA is working with Microsoft to arrange training, onboarding, and implementation assistance with participating agencies. The DTA, in partnership with the AI in Government Taskforce, will establish whole- of government guardrails and security requirements, oversee the trial, and coordinate evaluation. This includes assisting agencies to work through the various considerations of trialling Copilot for Microsoft 365 for their circumstances, risk tolerances and potential use cases.

Participating agencies will be required to partake in a program of activities and contribute to trial deliverables including induction and training, reporting against evaluation measures and development of case studies.

The DTA in partnership with the AI in Government Taskforce will evaluate the experience of users to assess opportunities, feedback, and report to government.

Security of the trial

Important security considerations

The DTA is consulting with the Australian Cyber Security Centre to understand the secruity requirements. An Infosec Registered Assessors Program (IRAP) assessment of Copilot for Microsoft 365 is currently underway and is planned for completion prior to the commencement of the trial.

The NDIA is currently undertaking our own security risk assessment in line with our own tolerance and circumstances.

Copilot for Microsoft 365 is a part of the Microsoft 365 family of products, which means it’s built into Microsoft 365 and inherits all the same security, compliance, and privacy controls that underpin Microsoft 365.

For more information, see Data, Privacy, and Security for Copilot for Microsoft 365.

The trial was open to all agencies licensed for Microsoft 365 E3 or E5 on the Whole of Government Microsoft Volume Sourcing Arrangement (VSA).

The DTA has placed a centralised order on behalf of the participating agencies for a six month period. Following the placement of the order, licences were transferred to

Page 23 of 189

OFFICIAL

the primary production tenancy of participating agencies which has been available for deployment since January 2024.

What were the Minimum requirements for trial participation?

The terms and minimum commitment for trial participation are an Agency commitment of:

• 300 Copilot for Microsoft 365 licences from agencies with 2,000 Microsoft 365 Qualified Users or more; or
• The greater of 15% of users or 10 Copilot for Microsoft 365 licences for agencies with fewer than 2,000 Microsoft 365 Qualified Users
• Commitment for the term of trial.

The overall benefit from Copilot for Microsoft 365 will be impacted by how heavily invested agencies are in their usage of Microsoft 365.

Agencies may opt to extend the term of their Copilot for Microsoft 365 licences after the trial for a further 12 months. The DTA will be contacting agencies with more information about extending their commitments as part of the annual licence reconciliation process.

Key steps and timeline

Page 24 of 189

OFFICIAL

Training and Support

• The DTA and Microsoft will be working with participating agencies to support them, through onboarding and readiness workshops, and additional training.
• Readiness workshops and training will be at no extra cost to agencies.
• An information pack containing FAQs addressing whole-of-government guardrails, possible use cases, evaluation approach and other information about the trial will be provided to trial participants early in the trial process.

Useful information

The following resources have been developed to assist implementing AI in the APS

Interim guidance on government use of public generative AI tools | aga (digital.gov.au)

Australia’s AI Ethics Principles | Australia’s Artificial Intelligence Ethics Framework - Department of Industry, Science and Resources

How might artificial intelligence affect the trustworthiness of public service delivery - PM&C (pmc.gov.au)

Product information

Copilot for Microsoft 365

Microsoft Copilot Copyright Commitment Security Information

Data, Privacy, and Security for Copilot for Microsoft 365

Microsoft Purview data security and compliance protections for Microsoft Copilot

Support and Feedback on the trial

The NDIA project team responsible for conducting the trial can be contacted via: redacted: s47E(d) - certain operations of agencies

Page 25 of 189

ndis

AI in Action:

Microsoft Copilot enhancing Productivity and Collaboration

Ajay Satyan Chief Information Officer

ndis.gov.au

What is Generative AI? ndis

• Generative AI uses algorithms to identify the underlying patterns and relationships in the data and can generate new, synthetic data that is statistically like the original data.
• Generative AI can create new content such as text, images, and music.
• It can be used to automate repetitive tasks and free up time for more strategic, creative work. redacted
• Generative AI can help organisations ramp up their productivity and do more with less.
• It can help unlock the full potential of data across businesses and empower users to gain distilled meaningful insights to guide every redacted: s47F - personal privacy.
• Trust and transparency are important for generative AI to truly work for businesses. redacted

Page 27 of 189

Generative AI at the NDIA ndis

• 2024 is the year where AI becomes “consumer-grade” and with this inevitable change it is critical that NDIA engages across Whole of Government forums on the safe and responsibile applications of AI across the APS and in the NDIA.
• Adopting a strategic approach to AI implementation can enhance our decision-making capabilities and drive innovation within the agency.
• A well-planned AI strategy can help us stay ahead redacted of the curve.
• AI is already pervasive in our personal lives, for example AI is built into Google search or in music suggestions from Spotify.
• AI can help to level the playing field for people with cognitive disabilities or people who are neurodivergent by interpreting text and summarising in a manner that is more easily understood. redacted

Page 28 of 189

AI and Microsoft Copilot Proof of

Concept Working Group ndis

• As part of the WoG Microsoft CoPilot trial, an AI Microsoft Copilot working group comprising of team members across the agency ahs been setup.

• A working group will be established in March 2024.

• The working group will consider the WOG and DTA policies related to AI and its applicability in the agency.

• The following table identifies members of the working group.

Page 29 of 189

FOI 24/25-1356 - DISCLOSURE LOG ndis

Introduction to the Copilot Trial

• The Australian Government announces 6-month trial of Copilot for Microsoft 365 which is led by the DTA.

• Trial to commence early 2024 across the Australian Public Service (APS) redacted

• Explore safe, responsible and innovative use of generative AI redacted

redacted

Image Caption: A pen drawing a graph

Page 30 of 189

About the trial ndis

• DTA administers trial from January to June 2024
• Collaborates with Microsoft for training and implementation
• DTA and AI in Government Taskforce establish guardrails including advice on interim policy and security requirements
• Oversees trial and coordinates evaluation redacted
• Participating agencies partake in program of activities
• Contribute to trial deliverables including training and reporting
• DTA and AI in Government Taskforce evaluate user experience redacted
• Assess opportunities, feedback, and report to government redacted

Page 31 of 189

What is Microsoft Copilot? ndis

• Copilot is built on advanced AI technology, such as Generative AI.
• Copilot is designed to enhance productivity and collaboration by enabling administrative efficiencies. redacted
• Copilot can assist with a variety of tasks, including summarising important documents, generating drafts of emails and documents, searching documents for answers, and creating/writing different varieties of content.
• Copilot has limitations and is not a mechanism for making agency decisions.
• Copilot’s features will be added progressively to NDIA supplied devices on the trial.

Image Caption: A screenshot of Microsoft Copilot in Word. The prompt is to draft a proposal based on meeting notes from a redacted OneNote file

Page 32 of 189

What is Copilot? (cont.) ndis

• Copilot has native integration with:
  • Word,
  • Excel,
  • PowerPoint,
  • Outlook,
  • Teams,
  • Notepad,
  • Whiteboard, and
  • Loop
• Copilot does not have integration with other agency applications such as PACE or ESSentials.

Image Caption: The words “Copilot for Microsoft 365” and “Microsoft 365 Apps” along with the Copilot logo and logos for the Microsoft Office applications.

Page 33 of 189

What is Copilot? (cont.) ndis

Page 34 of 189

Copilot Live Demo

DTA Key steps and timeline ndis

• 2023:
  • November 24th: Agencies receive Copilot for M365 trial license agency commitment and order form.
  • December 8th: Agency commitments received & collated
  • December 15th: DTA Finalise and Sign Contract
  • December 18th - 21st: Receive agency invoices from SDO & send to agencies
• 2024:
  • January 1st: Copilot trial commences redacted
  • February/March: Agencies required to notify Microsoft if seeking to extend Copilot. This will be in line with the ASC confirmation process for CCC and ECC enrollments
  • April (Early): Expected trial evaluation by DTA
  • June 27th: Copilot trial end

Page 36 of 189

Where are we now? ndis

• The agency has an initial commitment for trial participation with 300 licences
• An initial group of 50 people have been granted access as part of a Proof of Concept to conduct initial usability testing.
• A wider group of 300 people will be provided access at the end of February 2024. The wider group includes the following groups:
  • All SES and support staff,
  • Members of the Employee Disability Network (EDN), redacted
  • Office of the CIO technical support teams.
• A policy covering the responsible use of AI has been drafted and a working group has been established to work through the usage of AI in the agency.
  • The policy take guidance from government organisations redacted such as DTA and APSC. It also takes into consideration guidance from authorities such as Gartner.
• Agency communications are in development with redacted Communications.

Page 37 of 189

Important security considerations ndis

• DTA is consulting with Australian Cyber Security Centre
  • Understanding security requirements
  • IRAP assessment of Copilot for Microsoft 365 has been completed
• NDIA is undertaking our own security risk assessment
  • In line with own tolerance and circumstances redacted
• Copilot for Microsoft 365 part of Microsoft 365 family
  • Built into Microsoft 365
  • Inherits same security, compliance, and privacy controls redacted
• More information available
  • See Data, Privacy, and Security for Copilot for Microsoft 365 redacted

Image Caption: Close-up of a computer server

Page 38 of 189

Training and Support ndis

• DTA and Microsoft will support participating agencies through onboarding and readiness workshops, and additional training.
  • Readiness workshops and training will be at no extra cost to agencies. redacted
• An information pack will be provided to trial participants early in the trial process.
  • The information pack contains FAQs addressing whole-of-government guardrails, possible use cases, evaluation approach and other information about the trial. redacted
• The OCIO have developed their own change management material and worked with internal learning teams to develop a LEAP module. redacted

Image Caption: Yellow paper ship leading among white ships

Page 39 of 189