NDIA Accountability Project

This project is in DRAFT

NDIA | 3P Tasmania Test Evaluation

‹ PrevPage 1 of 74 · Source p. 1Next ›

NDIA | 3P Tasmania Test Evaluation

BM Interviews

February 2023

Acknowledgement of Country

Before we begin, I would like to acknowledge the Traditional Owners and Custodians of the Country on which we meet today, and their continuing connection to land, sea, and community. I pay my respects to their Elders, past present and emerging.

I would like to extend that acknowledgement and respect to any Aboriginal and Torres Strait Islander peoples here today.

3P Tasmania Test Evaluation – BM Interviews

Key Themes

The session discussion will be focused on the following key themes:

  • Communication & Engagement
  • Training
  • Cut-over approach and Transition Management
  • Hypercare Support and Management
  • Experience and Effectiveness (People)
  • Productivity and Quality (Process)

Additional Feedback

Additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Key Theme 1

Effectiveness of the communication and engagement approach in informing staff of the new service model

What worked well? What could we do better? What have we learnt?

Key Theme 2

Effectiveness of the training in preparing staff for the change in the new service model

What worked well? What could we do better? What have we learnt?

Page 5 of 74

Key Theme 3

Impact of the cut-over and transition of the new service model on staff

What worked well? What could we do better? What have we learnt?

Page 6 of 74

Key Theme 4

Effectiveness of the go-live (Hypercare, and feedback channels) support and management for staff

What worked well? What could we do better? What have we learnt?

Page 7 of 74

Key Theme 5

Overall Experience and Effectiveness for staff

What worked well? What could we do better? What have we learnt?

Page 8 of 74 8

Key Theme 6

Impacts on Productivity and Quality (Process)

What worked well? What could we do better? What have we learnt?

Page 9 of 74

Additional Feedback

Additional Feedback: If you have any additional feedback, please

pop it into the session for review.

What worked well? What could we do better? What have we learnt?

Page 10 of 74 10

Thank you for your feedback

NDIA | 3P Tasmania Test Evaluation

March 2023

Acknowledgement of Country

Before we begin, I would like to acknowledge the Traditional Owners and Custodians of the Country on which we meet today, and their continuing connection to land, sea, and community. I pay my respects to their Elders, past present and emerging.

I would like to extend that acknowledgement and respect to any Aboriginal and Torres Strait Islander peoples here today.

Introduction – Tasmania Test Evaluation Focus Groups

FOI 23/24-0145

Purpose

  • make sure our systems and processes work as designed and proposed outcomes are on track to be achieved
  • hear from stakeholders who were part of the test and identify areas for improvement
  • inform how we roll out the system nationally using our test experience

Scope

  • Key themes have been identified via data collected from January’s satisfaction survey and feedback gathered from evaluation focus groups held in February, to guide the session.
  • There are three questions that we are looking to collect feedback on:
    • What worked well?
    • What we could do better?
    • What have we learnt?
  • The capturing of these insights will feed directly into the Evaluation Framework

Session Guidelines

  • The time set aside for each section will be enforced to ensure all sections are covered.
  • No cutting off others while they are speaking (be respectful).
  • Feedback can be added to the chat function and will be captured.
  • This session will be recorded.
  • Please respect the guidance from the facilitator (to stay on time and purpose)

— Page 14 of 74

Agenda

Introduction – 5 Minutes

  • Purpose
  • Scope
  • Session Rules

Key Themes - 40 Minutes (7 minutes each)

The focus group session discussions will be focused on the following key themes:

  1. Staff System and Process Confidence: What factors are currently influencing staff confidence using PACE?
  2. Staff Support Requirements: What support requirements do you currently have?
  3. Increasing Usability and Efficiency for PACE Users: What needs improving to increase usability and efficiency?
  4. Supporting Participants Adjust: What additional resources do you feel participants require to adjust to 3P related changes?
  5. Preparing Staff for PACE: What resources would best prepare staff to use the new PACE system and processes?
  6. Staff PACE Experience: What is your current experience of the new PACE system?
  7. Communicating PACE updates to Staff: What would be the most effective way to communicate PACE updates?

Additional Feedback

Additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Key Theme 1 - Staff System and Process Confidence

What factors are currently influencing staff confidence using PACE?

What worked well? What could we do better? What have we learnt?

Key Theme 2 - Staff Support Requirements

What support requirements do you currently have?

What worked well? What could we do better? What have we learnt?

Page 17 of 74 6

Key Theme 3 - Increasing Usability and Efficiency for PACE Users

What needs improving to increase usability and efficiency?

What worked well? What could we do better? What have we learnt?

Key Theme 4 - Supporting Participants Adjust

What additional resources do you feel participants require to adjust to 3P related changes?

What worked well? What could we do better? What have we learnt?

Page 19 of 74 8

Key Theme 5 - Preparing Staff for PACE

What resources would best prepare staff to use the new PACE system and processes?

What worked well? What could we do better? What have we learnt?

Page 20 of 74 9

Key Theme 6 - Staff PACE Experience

What is your current experience of the new PACE system?

What worked well? What could we do better? What have we learnt?

Page 21 of 74 10

Key Theme 7 - Communicating PACE updates to Staff

What would be the most effective way to communicate PACE updates?

What worked well? What could we do better? What have we learnt?

Page 22 of 74 11

Additional Feedback

Additional Feedback: If you have any additional feedback, please pop it into the session chat for review.

What worked well? What could we do better? What have we learnt?

Page 23 of 74 12

Thank you for your feedback

NDIA | 3P Tasmania Test Evaluation

Agency PACE Users - Focus Group

February 2023

Acknowledgement of Country

FOI 23/24-0145

Before we begin, I would like to acknowledge the Traditional Owners and Custodians of the Country on which we meet today, and their continuing connection to land, sea, and community. I pay my respects to their Elders, past present and emerging.

I would like to extend that acknowledgement and respect to any Aboriginal and Torres Strait Islander peoples here today.

Agenda

FOI 23/24-0145

Introduction – 5 Minutes

  • Purpose
  • Scope
  • Session Guidelines

Key Themes - 40 Minutes (7 minutes each)

The focus group session discussions will be focused on the following key themes:

  1. Effectiveness of the communication approach in informing Agency PACE Users of the new service model
  2. Effectiveness of the change management approach in facilitating Agency PACE Users adoption of the new service model
  3. Effectiveness of the training in preparing Agency PACE Users for the change in the new service model
  4. Impact of the cut-over and transition of the new service model on Agency PACE Users
  5. Effectiveness of the go-live support (Hypercare, and feedback channels) and resource support for Agency PACE Users
  6. Extent to which the 3P program impacted the Agency PACE Users experience
  7. Impact of the new service model on the Agency PACE Users efficiency

Additional Feedback

Additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Introduction – Tasmania Test Evaluation Focus Groups

FOI 23/24-0145

Purpose

  • make sure our systems and processes work as designed and proposed outcomes are on track to be achieved
  • hear from stakeholders who were part of the test and identify areas for improvement
  • inform how we roll out the system nationally using our test experience

Scope

  • Key themes have been identified through observations collected in Hypercare & OILs feedback and are being used to guide the session
  • There are three questions that we are looking to collect feedback on:
    • What worked well?
    • What we could do better?
    • What have we learnt?
  • The capturing of these insights will feed directly into the Evaluation Framework

Session Guidelines

  • The time set aside for each section will be enforced to ensure all sections are covered
  • No cutting off others while they are speaking (be respectful)
  • Feedback can be added to the chat function and will be captured
  • This session will be recorded
  • Please respect the guidance from the facilitator (to stay on time and purpose)

— Page 28 of 74

Key Theme 1

Effectiveness of the communication approach in informing Agency PACE Users of the new service model

  • Internal engagement forums

  • Leader lead cascade approach

  • Formal consultation

  • Hub

  • Horizon

  • Huddle

  • Leadership updates

  • Emails

  • Vods

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Key Theme 2

Effectiveness of the change management approach in facilitating Agency PACE Users adoption of the new service model

  • Change leadership engagement forums

  • ADKAR

  • Superusers

  • Intranet

  • Moments that matter

  • Change champions

  • Persona (role specific) change journeys

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Key Theme 3

Effectiveness of the training in preparing Agency PACE Users for the change in the new service model

  • UAT

  • Interpersonal skills — role changes

  • QRGs

  • eLearning

  • Just In Time Resources

  • Videos on demand

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

— Page 31 of 74

Key Theme 4

Impact of the cut-over and transition of the new service model on Agency PACE Users

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 32 of 74 8

Key Theme 5

Effectiveness of the go-live support (Hypercare, and feedback channels) and resource support for Agency PACE Users

  • Daily 3P notice

  • Superusers

  • 3P Hub

  • Escalation pathway

  • Floor walkers

  • Jira tickets

  • 3P Ongoing questions, ideas and feedback form

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Key Theme 6

Extent to which the 3P program impacted the Agency PACE Users experience

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 34 of 74 10

Key Theme 7

Impact of the new service model on the Agency PACE Users efficiency

What worked well? | What could we do better? | What have we learnt?

Please indicate brief context/situation related feedback comment

Page 35 of 74 11

Additional Feedback

Additional Feedback: If you have any additional feedback, please pop it into the session chat for review.

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 36 of 74 12

Thank you for your feedback

NDIA | 3P Tasmania Test

Evaluation

February 2023

Acknowledgement of Country

Before we begin, I would like to acknowledge the Traditional Owners and Custodians of the Country on which we meet today, and their continuing connection to land, sea, and community. I pay my respects to their Elders, past present and emerging.

I would like to extend that acknowledgement and respect to any Aboriginal and Torres Strait Islander peoples here today.

Agenda

Introduction – 5 Minutes

  • Purpose
  • Scope
  • Session Rules

Key Themes - 40 Minutes (7 minutes each)

The focus group session discussions will be focused on the following key themes:

  1. Effectiveness of the communication approach in informing partners of the new service model
  2. Effectiveness of the change management approach in facilitating partner adoption of the new service model
  3. Effectiveness of the training in preparing partners for the change in the new service model
  4. Impact of the cut-over and transition of the new service model on partners
  5. Effectiveness of the go-live support (Hypercare, and feedback channels) and resource support for partners
  6. Extent to which the 3P program impacted the partner experience
  7. Impact of the new service model on the partners efficiency

Additional Feedback

Additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Introduction – Tasmania Test Evaluation Focus Groups

FOI 23/24-0145

Purpose

  • make sure our systems and processes work as designed and proposed outcomes are on track to be achieved
  • hear from stakeholders who were part of the test and identify areas for improvement
  • inform how we roll out the system nationally using our test experience

Scope

  • Key themes have been identified through observations collected in Hypercare & OILs feedback and are being used to guide the session
  • There are three questions that we are looking to collect feedback on:
    • What worked well?
    • What we could do better?
    • What have we learnt?
  • The capturing of these insights will feed directly into the Evaluation Framework

Session Guidelines

  • The time set aside for each section will be enforced to ensure all sections are covered
  • No cutting off others while they are speaking (be respectful)
  • Feedback can be added to the chat function and will be captured
  • This session will be recorded
  • Please respect the guidance from the facilitator (to stay on time and purpose)

Page 41 of 74

FOI 23/24-0145

dis’

Key Theme 1

Effectiveness of the communication approach in informing partners of the new service model

  • 3P Overview Briefings

  • 3P Information Sessions

  • Tasmania Working Group

  • 3P Tas Test Partner Change Engagement Meeting (Friday meeting)

  • Email updates/3P Tas Test Emails

  • 3P Hub

  • Horizon

  • ADKAR briefing packs

  • Leader lead cascade approach

  • Huddle

  • Tas test Operational Comms

What worked well? What could we do better? What have we learnt?

FOI 23/24-0145 ndis

Key Theme 2

Effectiveness of the change management approach in facilitating partner adoption of the new service model

  • Facilitated Working Group Sessions
  • Change Impact Assessments
  • Deed of variation
  • End to end journey maps
  • Change engagement forums
  • ADKAR approach/briefing packs
  • Superusers/Super user forum
  • Intranet/3P Hub
  • Moments that matter/LEAP
  • Persona (role specific) change journeys
  • Hypercare

What worked well? What could we do better? What have we learnt?

Key Theme 3

Effectiveness of the training in preparing partners for the change in the new service model

  • PACE Demonstrations

  • FAQ

  • UAT

  • 3P Hub

  • Superusers

  • Interpersonal skills – role changes

  • QRGs

  • Moments that matter

  • Work Instructions

  • eLearning

  • Just in Time Resources

  • Videos on demand

  • Facilitated Training

What worked well? What could we do better? What have we learnt?

Key Theme 4

Impact of the cut-over and transition of the new service model on partners

  • Cutover briefing sessions

  • Cutover packs and resources

  • Cutover timing/changes

  • Cutover approach effectiveness

  • Data migration approach

  • Cutover training/comms

What worked well? What could we do better? What have we learnt?

— Page 45 of 74

Key Theme 5

Effectiveness of the go-live support (Hypercare, and feedback channels) and resource support for partners

  • Daily 3P notice/ Ops Comms

  • Superusers

  • 3P Hub

  • Escalation pathway

  • Floor walkers

  • Jira tickets

  • 3P Ongoing questions, ideas and feedback form on Hub

  • P&CC/Partner Performance lead support

  • Hypercare process

What worked well? What could we do better? What have we learnt?

FOI 23/24-0145

Key Theme 6

Extent to which the 3P program impacted the partner experience

  • BAU work
  • Participant/person experience
  • Confidence and skills
  • Ability to adapt
  • Organisational readiness for large scale change
  • Roles & responsibilities
  • Leadership support & confidence
  • Trainer support & confidence
  • Using/learning a new system/process
  • Understanding and explaining the change

What worked well? What could we do better? What have we learnt?

FOI 23/24-0145

Key Theme 7

Impact of the new service model on the partners efficiency

  • KPI’s

  • BAU work

  • Confidence in providing a service

  • Backlogs

  • Time off tools

  • Working in the new system

  • Dual system

  • Work instructions/work arounds

  • Hypercare process / Logging tickets

What worked well? What could we do better? What have we learnt?

Additional Feedback

Additional Feedback: If you have any additional feedback, please

pop it into the session for review.

What worked well? What could we do better? What have we learnt?

Page 49 of 74 12

Thank you for your feedback

NDIA | 3P Tasmania Test Evaluation

Tasmania Providers - Focus Group

February 2023

Agenda

FOI 23/24-0145

Introduction – 5 Minutes

  • Purpose
  • Scope
  • Session Guidelines

Key Themes - 40 Minutes (7 minutes each)

The focus group session discussions will be focused on the following key themes:

  1. Effectiveness of the communication approach in informing providers of the new service model
  2. Effectiveness of the change management approach in facilitating providers adoption of the new service model
  3. Effectiveness of the training in preparing providers for the change in the new service model
  4. Impact of the cut-over and transition of the new service model on providers
  5. Effectiveness of the go-live (Hypercare, and feedback channels) for providers
  6. Extent to which the 3P program impacted the provider experience
  7. Impact of the new service model on the provider efficiency

Additional Feedback

additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Introduction – Tasmania Test Evaluation Focus Groups

FOI 23/24-0145

Purpose

  • make sure our systems and processes work as designed and proposed outcomes are on track to be achieved
  • hear from stakeholders who were part of the test and identify areas for improvement
  • inform how we roll out the system nationally using our test experience

Scope

  • Key themes have been identified through observations collected in Hypercare & OILs feedback and are being used to guide the session
  • There are three questions that we are looking to collect feedback on:
    • What worked well?
    • What we could do better?
    • What have we learnt?
  • The capturing of these insights will feed directly into the Evaluation Framework

Session Guidelines

  • The time set aside for each section will be enforced to ensure all sections are covered
  • No cutting off others while they are speaking (be respectful)
  • Feedback can be added to the chat function and will be captured
  • This session will be recorded
  • Please respect the guidance from the facilitator (to stay on time and purpose)

Page 53 of 74

Key Theme 1

Effectiveness of the communication approach in informing providers of the new service model

  • 3P Overview Briefing

  • 3P Information Sessions

  • Tasmania Provider Working Groups

  • Session Presentations

  • Email updates

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Key Theme 2

Effectiveness of the change management approach in facilitating providers adoption of the new service model

  • Facilitated Working Group Sessions

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Key Theme 3

Effectiveness of the training in preparing providers for the change in the new service model

  • PACE Provider Portal Demonstrations
  • FAQ/QRG’s
  • Provider Information Pack

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

— Page 56 of 74

Key Theme 4

Impact of the cut-over and transition of the new service model on providers

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 57 of 74

Key Theme 5

Effectiveness of the go-live (Hypercare, and feedback channels) for providers

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 58 of 74 8

Key Theme 6

Extent to which the 3P program impacted the provider experience

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 59 of 74 9

Key Theme 7

Impact of the new service model on the provider efficiency

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 60 of 74 10

Additional Feedback

Additional Feedback: If you have any additional feedback, please pop it into the session chat for review.

What worked well? What could we do better? What have we learnt?

Please indicate brief context/situation related feedback comment

Page 61 of 74 11

Thank you for your feedback

NDIA | 3P Tasmania Test Evaluation

March 2023

Acknowledgement of Country

Before we begin, I would like to acknowledge the Traditional Owners and Custodians of the Country on which we meet today, and their continuing connection to land, sea, and community. I pay my respects to their Elders, past present and emerging.

I would like to extend that acknowledgement and respect to any Aboriginal and Torres Strait Islander peoples here today.

Agenda

Introduction – 5 Minutes

  • Purpose
  • Scope
  • Session Rules

Key Themes - 40 Minutes (7 minutes each)

The focus group session discussions will be focused on the following key themes:

  1. Provider PACE Experience: What is your current experience of the new PACE system?
  2. Provider PACE Support Requirements: What support requirements do you currently have?
  3. Preparing Providers for PACE: What resources would best prepare providers to move to the new PACE system and processes?
  4. Communicating PACE updates to Providers: What would be the most effective way to communicate PACE updates?
  5. Increasing Usability and Efficiency for Providers: What needs improving to increase usability and efficiency?
  6. Supporting Participants Adjust: What additional resources do you feel participants require to adjust to 3P related changes?

Additional Feedback

Additional feedback we didn’t get to cover today, please pop into the session chat and we will review this post session. Thank you.

Page 65 of 74

FOI 23/24-0145

Introduction – Tasmania Test Evaluation Focus Groups

Purpose

  • make sure our systems and processes work as designed and proposed outcomes are on track to be achieved
  • hear from stakeholders who were part of the test and identify areas for improvement
  • inform how we roll out the system nationally using our test experience

Scope

  • Key themes have been identified via data collected from Januarys satisfaction survey and feedback gathered from evaluation focus groups held in February, to guide the session.
  • There are three questions that we are looking to collect feedback on:
    • What worked well?
    • What we could do better?
    • What have we learnt?
  • The capturing of these insights will feed directly into the Evaluation Framework

Session Guidelines

  • The time set aside for each section will be enforced to ensure all sections are covered.
  • No cutting off others while they are speaking (be respectful).
  • Feedback can be added to the chat function and will be captured.
  • This session will be recorded.
  • Please respect the guidance from the facilitator (to stay on time and purpose)

— Page 66 of 74

Key Theme 1 - Provider PACE Experience

What is your current experience of the new PACE system?

What worked well? What could we do better? What have we learnt?

Key Theme 2 - Provider PACE Support Requirements

What support requirements do you currently have?

What worked well? What could we do better? What have we learnt?

Page 68 of 74

Key Theme 3 - Preparing Providers for PACE

What resources would best prepare providers to move to the new PACE system and processes?

What worked well? What could we do better? What have we learnt?

Page 69 of 74

Key Theme 4 - Communicating PACE updates to Providers

What would be the most effective way to communicate PACE updates?

What worked well? What could we do better? What have we learnt?

Page 70 of 74 8

Key Theme 5 - Increasing Usability and Efficiency for Providers

What needs improving to increase usability and efficiency?

What worked well? What could we do better? What have we learnt?

Page 71 of 74 9

Key Theme 6 - Supporting Participants Adjust

What additional resources do you feel participants require to adjust to 3P related changes?

What worked well? What could we do better? What have we learnt?

Page 72 of 74 10

Additional Feedback

Additional Feedback: If you have any additional feedback, please

pop it into the session chat for review.

What worked well? What could we do better? What have we learnt?

Page 73 of 74 11

Thank you for your feedback