FOI 24/25-1356 - DISCLOSURE LOG DOCUMENT 3
OFFICIAL: SENSITIVE
The contents of this document are OFFICIAL: SENSITIVE
Board meeting: [Board sec to insert date]
Agenda Item: #
Agenda Item: Extension of the Microsoft Copilot Trial
Paper Type: For Decision
SLT Sponsor: Samuel Porter, Chief Operating Officer
Link to Corporate Plan: Program 1.2 — Agency costs, Key activity 2 — Develop a high performing NDIA for participants
Draft resolution: The Board noted the NDIA’s use of Microsoft Copilot and endorsed a continuation of the trial.
1. Purpose
1.1. To advise the Board on the Digital Transformation Agency (DTA) Microsoft Copilot (Copilot) trial, including learnings, use case and risks.
1.2. To seek Board approval to extend the Copilot trial to better understand the value of Artificial Intelligence (AI) technology to the NDIA.
2. Recommendation
2.1. The Board notes the history and preliminary findings of the Copilot Trial.
2.2. The Board endorses NDIA to continue with the Copilot Trial, engage on the ethical and responsible use of Copilot, and inform our evolving governance and policy position on AI.
3. Key Insights
3.1. The NDIA continues to explore how AI can aid administrative tasks for NDIA staff and increase efficiencies within standard operations.
3.2. The Copilot for Microsoft 365 (M365) trial has demonstrated a significant positive impact on accessibility for employees with disabilities. Copilot has demonstrated value in aiding an inclusive work environment.
3.3. The working group found broad value in streamlining routine administrative tasks, particularly minute taking and summarising email chains.
4. Background
4.1. In November 2023, the NDIA was invited to be one of 30 Agencies to take part in a managed trial of Copilot for M365 coordinated by the Digital Transformation Agency (DTA).
4.2. The trial of Copilot for M365 provided a safe and integrated AI environment that is controlled by the Agency. The trial began in February 2024 with 300 initial licences that included members of the NDIA Employee Disability Network.
4.3. Copilot is integrated within M365 and is designed to operate as an AI administrative support system. It only operates within the M365 suite, with programs and files that the user has authorisation to access. Copilot does not have access to Agency or Participant data outside of the Agency’s standard suite of Microsoft products.
OFFICIAL: SENSITIVE
x.x.1
Page 64 of 189
FOI 24/25-1356 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
5. Copilot Trial
5.1. The Copilot Trial was conducted from February to June 2024, with access to 300 licences
(at a cost of redacted: s47G - business inform), 194 licences were used in the trial. The trial set out to better
understand the functionality and applicability of Copilot as an AI tool, its compliance with
cyber security requirements, and overall usability.
5.2. The trial initially used feedback from experts in the ICT, Cyber Security, Legal, and Accessibility Technology teams to identify risks and benefits.
5.3. On completion of the trial the Copilot Project team will submit a report outlining the outcomes of the trial, key insights, and recommendations.
5.4. Preliminary findings of the Copilot trial indicate:
- 5.4.1. Support for SES in Microsoft applications: Users have seen significant timesaving benefits using Copilot to summarise information from the Microsoft suite of applications. This means users can search for key information and Copilot can provide a summary combining information from all applications.
- 5.4.2. Extensive Use in Microsoft Teams: Copilot is being utilised for minute taking, recapping meetings, and providing actions from meetings in MS Teams.
- 5.4.3. Enhanced Accessibility across M365 Applications: Copilot has significantly improved accessibility within M365 applications for employees with disabilities, enabling them to interact with technology more efficiently.
5.5. A Survey of trial participants conducted in May 2024, found that Copilot exceeded expectations in Word, Outlook, and PowerPoint. Copilot exceeded user expectations for 67.4% of trial members and 84% of users were now highly confident with the use of AI due to their participation in the trial.
6. Extension of the Copilot Trial
6.1. The DTA Copilot Trial found value in the AI tool, noting there continues to be broad discussion regarding aspects such as value for money, integration into operating environments and risk management.
6.2. While the Agency seeks approval to continue the Copilot trial, further work will be undertaken to understand the risk exposure that AI poses to the Agency and will evaluate key mitigation strategies that are in place on completion of the Copilot trial.
6.3. Continuation of the trial will allow further opportunities for the Employee Disability Network (EDN) to explore the accessibility benefits of using Copilot in their business-as- usual activities. This will provide further data on the use of Copilot as an accessibility assistant.
6.4. An extension of the trial will be separate from the current DTA trial, which finishes in June
2024. Microsoft has quoted redacted: s47G - busine per Copilot licence to allow a continuation of the trial,
with our current pool of 300 licences, equating to a cost of redacted: s47G - business inform (GST Inc) for
2024/25.
6.5. The NDIA Trial will utilise all 300 licences by including most NDIA SES, and their Executive Assistants. This will expand the trial participants into more business areas and work types, further testing the capability, value, and risks of Copilot.
OFFICIAL: SENSITIVE
x.x.2
Page 65 of 189
FOI 24/25-1356 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
7. Risks
7.1. An interim policy was implemented to address the risk of using AI and machine learning. The policy is supported with mandatory learning modules for users that address the ethical, legal and security risks.
7.2. The AI Working Group is coordinating with the Agency Risk, Audit and Resilience Division to undertake a documented risk assessment, consolidating existing risk mitigation strategies and to identify any emerging risks.
7.3. The Agency’s Strategic Leadership Team has decided that any uses of AI and machine learning must be considered by the Agency’s Ethics Committee.
7.4. The NDIA conducted a Cyber Security risk assessment on the Microsoft suite of applications that incorporate Copilot. The DTA has also undertaken an assessment of Copilot, which the NDIA has reviewed for risks that may impact the Agency. The Cyber Security risks of using Copilot are considered LOW due to:
- 7.4.1. Copilot operates as a private tool, which is separated from other Microsoft customers and the public. Copilot only accesses information and data in the Microsoft suite of applications which the user has authorisation to access.
- 7.4.2. Sensitive NDIA systems and data (such as the CRM and Protected Enclave) cannot be accessed by Copilot.
- 7.4.3. The NDIA’s use of Copilot is designed to work within the boundaries of the NDIA ICT environment and does not access the wider internet. Microsoft has made a commitment to the Australian Government that the use of Copilot will never train its large language model.
8. Participant Impact
8.1. Stable, accessible, secure, and participant-orientated technology is a crucial enabler for the delivery of the NDIA.
9. Sustainability Impacts
9.1. Increased administrative productivity will lead to time savings and quicker responses improving participant and stakeholder satisfaction, increasing Agency reputation, and supporting the long-term sustainability of the Scheme.
10. Other Impacts
10.1. The extended trial will be overseen by the current AI Working Group and will provide further understanding of the impact, risks, benefits, and value of Copilot as a productivity support for NDIA staff.
11. Responsibility and next steps
11.1. The Office of the Chief Information Officer will administer the extended trial with continued oversight provided by the Agency AI Working Group.
11.2. The Branch Manager of Enterprise Architecture and Governance will allocate Copilot licences across the Senior Leadership Group.
Internal use only –removed for meeting
OFFICIAL: SENSITIVE
x.x.3
Page 66 of 189
FOI 24/25-1356 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
| Prepared by: | [Action Officer] | Approved by: | [GM Name] |
|---|---|---|---|
| Division: | [Action officer’s division] | [GM title] | |
| Phone: | [Action officer’s phone number] | Phone: | [GM phone number] |
| Cleared by ELT | [ELT date] |
OFFICIAL: SENSITIVE
x.x.4
Page 67 of 189