Summary of Research Provided to the Joint Standing Committee by Martin Hoffman (Question reference number: NDIA IQ21-000014)
Sympathy bias
None of the articles provided to the Committee by Mr Hoffman (referred throughout below as #1- #10) document actual examples of ‘sympathy bias’ by any kind of professional.
Nor do they provide any assessment of how common the alleged sympathy bias might be in any scheme comparable to the NDIS. There is also no quantification of the risk of such bias occurring in the NDIS process, and therefore any associated potential contribution to inappropriate budget allocation is also unknown.
#6, the Productivity Commission’s Disability Care and Support Report, is presumably the origin of the use of the phrase ‘sympathy bias’ by Mr Hoffman, but contains only one reference to it:
‘Assessors should also be independent of the person being assessed to reduce the potential for ‘sympathy’ bias. This means that health professionals — GPs and others — with past treatment and support responsibilities for the person, would not undertake assessments. It is clear from the experiences of VCAT appeals on TAC benefit decisions that treating professionals are often placed in an invidious position when asked by their patients to make an assessment that determines the person’s eligibility for benefits’ (327).
However, other than the broad reference to TAC decisions, #6 provides no documentation of any studies or other evidence of sympathy bias. There is a similar lack of elaboration regarding the ‘sympathetic bracket creep’ of which independent assessors are said to be at risk (327; see also #7).
The two articles highlighted in Mr Hoffman’s Answer to Question on Notice, #1 and #2, make no reference at all to sympathy bias. The only actual mention of ‘bias’ is in relation to the issue of possible bias in the research studies reviewed. #1 does make one potentially relevant reference:
‘Further tensions may also exist for frontline staff between their ethical obligations to promote empowerment and self‐determination whilst honouring their legal obligations to limit access to individualized funding (Ellis, 2007)’ (7).
However, ‘tensions’ are not bias, and given the wording of the NDIS legislation, the above quote is just as true for the scheme as a whole. No further discussion is provided.1
1 Footnoted is Ellis, K. (2007), ‘Direct payments and social work practice: The significance of “Street‐Level Bureaucracy” in determining eligibility,’ British Journal of Social Work 37(3) 405–422. This article is not freely available, but see Ellis, K. (2009), Revisiting ‘StreetLevel Bureaucracy’ in Post-Managerialist Welfare States, PhD Thesis which demonstrates how insufficient the term ‘bias’ is to describe the processes in which social workers are enmeshed, regardless of whether or not they are providing direct services to participants.
Three articles (#3, #4 and #5) discuss what might at first glance be interpreted as bias but which is actually conflict of interest, with the latter potentially leading to the former. The professions discussed here are doctors/surgeons/psychiatrists in medicolegal processes.2
Considering the three articles in more detail, #3 is a literature review and personal opinion, written by a doctor to clarify the guidelines and responsibilities of expert witnesses and independent medical evaluators in legal processes. Conflicts of interest are defined as:
‘when secondary interest(s) have the potential to influence a physician’s judgment, actions, or opinions regarding a person who is the subject of litigation’.3
‘Conflicts of interest [COIs] are inevitable, occur throughout medicine, and cannot be completely avoided. COIs are not inherently negative. The problem is that COIs have the potential to lead to unconscious bias, which might influence opinions, decisions, or treatment.’
The writer of #3 is mainly concerned with financial conflicts of interest in the process of giving evidence in legal proceedings. However, even here, the author concludes that this need not mean a doctor cannot be the medical expert for their patient. In #3’s view, all expert witnesses and independent medical experts have a potential conflict of interest and the potential for bias. The author recommends instead that treating medical experts follow the same guidelines expected of an independent medical expert.
The full article was not freely available for #4, but the abstract states:
‘Recent decisions in Canada have allowed for opinion evidence by treatment providers (Westerhoff v. Gee Estate), which seem to ignore the potential bias of the treatment provider.’
The article appears to aim to clarify the distinction between treating expert witness physicians and independent consultants, in legal processes. Using Mr Hoffman’s own summary in Attachment A, #4, like #3, seems more concerned with managing potential bias rather than eliminating the role of the treating witness.
#5 is the American Academy of Psychiatry and the Law’s ‘Ethics guidelines for forensic psychiatrists in legal processes’. It is the only one of the three articles that recommends against combining the roles of treater and expert witness / independent evaluator. However, the Academy also suggests that this dual role might be unavoidable and therefore should entail a balancing exercise.
#8, #9 and #10 are all based on the same study of long-term home care of the elderly and people with disabilities in the Netherlands, which was undertaken before 2015 when independent assessments were abolished. In terms of bias, #9 claims that ‘assessors could make decisions more favorable to some categories of the population’, and adds in a footnote:
‘In sociology, empirical studies of decision-making by street-level bureaucrats have documented that background circumstances of applicants can considerably influence entitlements for social benefits (see, e.g., Scott, 1997). Furthermore, the “representative theory agency” posits that case workers will advocate the case of culturally similar patients more strongly (see, e.g., Meier and Bohte (2001)).’4
2 I am not sure how commonly these professions are used currently to assess disability for the purposes of NDIS funding, particularly if they are used as the sole source of expertise, and therefore how relevant the three articles are to the NDIS proposals. 3 Where page number references are not provided above, publications were read in web-based format. 4 For a more complex analysis, see the references in footnote 1.
However, neither of the quoted examples are analogous to the position of treater assessors. Both examples could also be more justifiably applied to NDIA personnel making decisions about plans and budgets (see ‘The rationale for independent assessments’ below).
#8 observes:
‘Traditionally, the needs of patients are assessed by their health care providers. But these providers may be self-interested and have superior information about patients’ needs, resulting in principal-agent problems for the patient and the third-party payer’ (41).
Evidence for the existence of this bias is not provided, despite reference in #8 to a 1963 article in American Economic Review, and it is unclear what ‘superior information’ means. However, the notion of ‘self-interest’ is elaborated upon as one aspect of what the researchers term ‘moral hazard and supplier-induced demand’.5 They suggest that moral hazard and supplier-induced demand are particularly likely in the context of public long-term care (LTC) financing because:
‘Receiving more care and support than strictly needed is likely to generate positive marginal benefits for patients because it is offering additional comfort. Moreover, if LTC providers are paid fee-for-service and are allowed to perform needs assessment themselves, they may be inclined to induce more demand for their services than strictly necessary. Hence, if due to the presence of comprehensive LTC insurance the marginal costs for patients are low, the risk of moral hazard and supplier-induced demand may be particularly high for LTC services’ (41).
The notion that provider-assessors have a financial interest, and hence risk exhibiting bias (a key part of the moral hazard and supplier-induced demand), is a similar concept to that in #3. The premise of the Netherlands study is that this assumption of (particularly financial) bias is the reason that a number of health/care systems have delegated the assessment of patients’ needs to an independent assessor. The central focus of #8-#10 is therefore an examination of whether independent needs assessment is actually likely to constrain LTC demand (see ‘The rationale for independent assessments’ below).
#7, a submission to the Joint Standing Committee from an inaugural board member of the NDIA, also mentions bias in passing. It recommends that assessments should be undertaken by trained assessors engaged by the NDIA, and that:
‘To promote independent outcomes, assessors should not have a longstanding connection to the person’ (2).
While the relevance and nature of the ‘longstanding connection’ are not elaborated upon, #7 refers by analogy to the insurance industry:
‘. . . it is well-established within Australian injury insurance scheme design, on which the NDIS was modelled, that “reasonable and necessary” entitlement, without a way to objectively and consistently define that entitlement on an individual basis, leads to inequity of resource allocation and threatens the sustainability of schemes. Typically, this leads to extensive and damaging disputes, and ultimately to scheme redesign, usually restricting benefits either to eligibility or entitlement or both. . .’ (2)
#7 refers to ‘many examples across the 600 accumulated years of Australia and New Zealand’s 20 modern injury schemes’ (2) but does not elaborate. But is insurance – despite the NDIS
5 The 1963 American Economic Review article discusses the interplay of various factors in the ‘medical-care industry’, including a brief description of ‘moral hazard’ similar to the concept used in #8-#10.
nomenclature – really the appropriate comparator? Most insurance aims to make a profit,6 and is fundamentally shaped by litigation. If independent assessments are introduced, this aspect of the NDIS process is not proposed to be reviewable, let alone appellable.
And if it is the interpretation of ‘reasonable and necessary’ that lies at the root of the ‘bias’ problem, as discussed below, this is a test applied by public servants, not assessors of function.
The rationale for independent assessments
It is important to examine articles #1-#10 to understand how they link any assumption of treater/provider -assessor bias to a claimed need for independent assessments.
#1 is a review of 73 studies of individualised funding support for people with disabilities. It concludes that the personalised budget approach is largely positive. However, importantly, this review does not consider assessments of function at all, and appears more akin to a review of the planning stage once the functional assessment has been undertaken (eg 44-45). The review seems to assume that the relevant people with disabilities were already ‘in the system’ and were only changing how their funding was used (see 12, 16).
#2 is described by the authors in #1 as ‘a rapid evidence assessment rather than a rigorous systematic review’, and similarly does not focus on assessments, but instead on personal budgets. The dot point Mr Hoffman provides in relation to #2 (Attachment A) is the only reference in #2 to assessments. (Note too that it is ‘Carter Anand’, not ‘Carter and Anand’.)
In contrast, the Productivity Commission’s proposal for independent assessments in #6 applies to the whole process of assessment – not only of a potential participant’s function, but also the determination of the financial amount assigned to them in a budget (eg 313). The current NDIS proposal splits this process in two and requires the second (and the only reviewable) aspect – the actual funding amount and creation of the plan – to be determined not by independent personnel but by NDIA staff.
The introduction of independent assessments as a solution to the claimed sympathy bias of existing assessors then seems at odds with Minister Reynolds’ recent comments that the NDIS is too reliant on individual public servants’ judgment and their natural empathy.7 For this part of the process, the NDIA’s proffered solution is to reframe the definition and process of determining ‘reasonable and necessary’, rather than removing public servants from making assessments.
This proposed outcome is therefore not dissimilar to the conclusions of the discussions of medical conflicts of interest above. And if that can be done for public servants, why not for existing assessors of functioning who may already be treating the person with disability? That is surely the purpose of designing a rigorous assessment tool or adhering to a universal set of assessment competencies.
Only the cluster of three articles from the Netherlands study (see ‘Sympathy bias’ above) actually examine the impact of independent assessments on the functioning of a system that is at least partly comparable to the NDIS. It should be borne in mind that independent assessment in the Dutch system also determined initial eligibility as well as personal budget.
6 See eg Microsoft Word - P Nov09 SI in CTP FINAL.doc (actuaries.asn.au) which is referred to by both #7 and Mr Hoffman. 7 https://www.theguardian.com/australia-news/2021/may/18/linda-reynolds-says-ndis-is-too-reliant-on-natural-empathy-of-public-servants .
According to #8, the Dutch researchers set out to examine the hypothesis that:
‘When providers and recipients have an interest in providing or obtaining more or more expensive care than strictly needed, independent needs assessment may reduce overprovision and inefficiently high expenditures’ (42).
The study examined whether the maximum amount of care set by the (independent) assessor limited the amount of care actually used by people.8 They found that it did not, because people underuse what is available to them anyway, regardless of whether the assessment is independent or not. On this basis, the researchers deduced that independent assessment does not save the system money.9
#8 therefore concludes:
‘The limited impact of independent needs assessment on LTC [long term care] use raises the question about its effectiveness, at least within the Dutch context. . .[and] the limited effect of needs assessment on the intensive margin of home care use in the Netherlands also raises questions about the effectiveness of the independent assessment in other countries, where the demand and supply of LTC are often more restricted through other measures’ (45, emphasis added).
This conclusion is not reflected in Mr Hoffman’s summary of #8 in Attachment A.
The Dutch study assumes that provider-assessor bias underpins financial allocation, but the researchers were not centrally concerned with clarifying the specific relationship between any assessor bias and the setting of constraints on funding, or the extent of moral hazard and supplier- induced demand.
Rather, via investigating whether independent needs assessment effectively restricts LTC use, the researchers deduce that because the finding was that independent needs assessment does not impose a binding constraint on LTC use:
‘it is highly unlikely that [independent needs assessment] reduces moral hazard and supplier-induced demand among those who are eligible for care. Hence, this would remove an important reason for organizing the independent needs assessment’ (45).
Consequently, it is not possible from #8-#10 to conclude whether the difference in funding allocations between treater/provider -assessors and independent assessors was due to what might be regarded as bias – whether solely or in part. We also do not know whether both groups of assessors could be viewed as biased, but in different ways, and the extent to which any risk of bias is inherent in the relevant assessment tools and/or training rather than the assessor themselves.
Further speculation is encouraged by the study’s findings on whether independent assessments affect equity. #9 notes variation in home care entitlements among some (elderly) population groups with similar needs, including:
‘quite large differences across [regional offices] in the entitlements for given needs and in the conversion of these entitlements into use . . . despite the Dutch need assessment agency being theoretically centralized. We are unable to unravel whether this reflects
8 Study references to the ‘intensive’ margin concern the needs assessment (amount of care) stage, whereas the ‘extensive’ margin is related to the initial assessment of eligibility. 9 The Dutch findings were more equivocal about the impacts on people’s access to the scheme itself: more people were turned away, but more data was needed to understand which applicants and why (45).
local differences in preferences or in the provision of informal care, systematic variation in the supply of home care across municipalities, or differences in practices across [offices]’ (emphasis added).
#8 also notes:
‘. . . the independent assessor may also be biased due to pressure by stakeholders, financial restrictions or regulations, and may be less able to make an appropriate needs assessment because the assessor may be less informed about the specific needs of the patient than a provider due to a less personal and frequent contact with the patient’ (42, emphasis added).
It is difficult to see how this form of independent assessor bias could be adequately addressed by any strategies claimed to be capable of countering ‘sympathetic bracket creep’.
#9 did find that independent assessment ‘dampens’ inequity in access by socioeconomic status, but the NDIS reforms do not contemplate using independent assessment at that stage. Mr Hoffman’s use of #9 in Attachment A to support the proposed reforms is therefore disingenuous.
Conclusion
The articles referred to the Committee by Mr Hoffman do not amount to evidence for the existence of ‘sympathy bias’ among existing assessors of function for would-be NDIS participants.
Nor do they unpack any possible distinction between any found bias of the assessors and any found bias in the existing tools of assessment.
The list of references – and, at least to date, rationale from the NDIA and the Government – also fail to provide any evidence-based linkage between existing assessments and financial ‘blowout’ of the NDIS.
Further, the articles do not support the argument that replacing existing assessors by independent assessments would address the claimed problem. Indeed, the one country where such an approach has been examined has now abolished independent assessments, at least partly on the grounds that they do not reduce any potential bias.
Postscript
There are other findings and commentary in the provided articles that support the various concerns about the NDIS reforms raised by disability organisations and health professionals.
#1 notes the ‘importance of strong, trusting and collaborative relationships’ (4) and suggests, if only by implication, some parallels with Australian participant concerns about independent assessments:
‘A number of barriers, whilst viewed as generally manageable in the short term, were considered potentially problematic in the longer term. These include: inaccurate or inaccessible information sometimes due to an unclear understanding of individualized funding . . . a lack of resources/available support, exacerbated by an inaccurate estimation of need and subsequent delay in reviewing /adjusting budgets. This, amongst other things, can lead to conflict and tensions in working relationships, which are also hampered by disabling practices (e.g., exclusion from decision‐making)’ (5, emphasis added).
Further to disability activists’ and advocates’ essential requirement for participant participation in all decision-making, #7 argues:
‘The challenge for the NDIA is to walk with participants and their representatives along this journey of discussion in a transparent and honest fashion, for the mutual benefit. This engagement must include a more empowered Independent Advisory Council, with appropriately supported Subcommittees and Reference Groups, which must also include the more forceful advocates critical of the proposals. A way forward will only be found with active communication between these representatives and the operational and risk management arms of the NDIA.’ (6)
#7 also agrees with criticism that the assessment tools inappropriately cherry-pick, and that assessors need to be properly invested in rather than fast-tracked, and the pilot approaches critically analysed.
Dr Chris Atmore Manager, Policy and Advocacy
25 May 2021