There can come a point in growing older when staying in the home you love depends on someone helping you out of bed, making a meal, or making sure you’ve taken your medication. That care can mean holding on to your independence, to familiar surroundings, and to a life that still feels like your own. But now, the government is using an algorithm to determine home care funding - replacing assessors’ judgement with a formula that can overlook the complexities of someone’s life. And assessors can't override its decisions - even when they believe they're wrong. Today, aged care expert Professor Kathy Eagar, on the human consequences of handling those decisions to an algorithm, and the people whose needs are not being met as a result. If you enjoy 7am, the best way you can support us is by making a contribution at 7ampodcast.com.au/support. Socials: Stay in touch with us on Instagram Guest: Aged care researcher at UNSW, Professor Kathy Eagar Photo: Michaela Rehle/Reuters
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Speaker 1I'm Daniel James, and you're listening to 7am. There can come a point in growing older when staying in the home you love depends on someone helping you out of bed, making a meal, or making sure you're taking your medication. That care can mean holding on to your independence, to familiar surroundings, and to a life that still feels like your own. But now, the government is using an algorithm to determine home care funding, replacing assessor's judgement with a formula They can overlook the complexities of someone's life, but
Speaker 1with no ability for humans to override the decision, even when they know it's wrong. Today, aged care expert Professor Cathy Eger on the human consequences of handing those decisions to an algorithm and the people whose needs are not being met as a result. It's Friday, October 30th. Cathy, thanks for joining us. The government is using an algorithm
Speaker 1to determine how much funding older Australians receive for care at home. You've managed to reverse engineer that algorithm to understand how it makes those decisions. What did you discover?
Speaker 2What I discovered is two things, really. One is something I already knew before I did the engineering bit, which is we should be using these sort of tools to as decision support tools, not as decision-making tools. And the second is that the algorithm itself is fundamentally flawed from a technical perspective because it was designed by people who don't understand algorithms.
Speaker 1Right.
Speaker 3And what kind of decisions is the algorithm making? So a clinical assessor assesses somebody. And in the past, the clinical assessor said, well, I've assessed Kathy Eager.
Speaker 2This is what she needs. And these are the services she's going to need. And the cost of those services is X. The process now reverses that logic and it assesses, this is what the assessor says you need. This is how much the algorithm says that translate to in dollars. And then the assessor has to work out what services you can afford to from the dollars you've been given. Things like whether the person has dementia, whether they live alone,
Speaker 2whether they're homeless and live in a car. None of those things are taken into account. The other thing that's missing is that it doesn't actually take into account whether they have a carer, and if so, what their carer needs, what support they need to continue being carers. There's thousands of factors involved that an experienced clinical assessor will take into account about a person that no algorithm is ever
Speaker 2going to be able to take into account.
Speaker 1So why did the government introduce this algorithm or these algorithms?
Speaker 2It's a very good question and it doesn't make any sense to use an algorithm in aged care because aged care is a care and support scheme. It's not a financial entitlement scheme. In my view, the government's never been transparent about why it introduced the algorithm and it said things like, oh, assessors gave people too much before, which is not true.
Speaker 4Under the previous system, we had $ 4 billion of value allocated to people incorrectly and therefore unable to be deployed to look after older people.
Speaker 2There's no evidence for the claims that they've given. I think they've introduced the algorithm in support at home because this is the prototype that they're intending for the National Disability Insurance Scheme, which has been full of cost blowouts and scams and all those sorts of things that we've seen in the media. Aged care has not experienced those problems, but the government has put the two together. The NDIS is an insurance scheme. But the aged care system is
Speaker 2a care and support scheme, and I don't think the government understands the difference between those two schemes.
Speaker 1An algorithm, of course, can't take into a factor the important nuanced health information of individuals and support needs of their family. How badly can that distort the funding someone receives?
Speaker 2It can distort the funding in two ways. Obviously... the cases that are speaking out are the people who don't get the funding that they need. But it's also the case that some people are getting too much funding because the algorithm doesn't take into account so many other factors. So two people might look exactly the same in the algorithm, but one person lives in a home they own that's all on one level, that's got ramps and rails, that
Speaker 2has safety devices, et cetera. The other person lives on the second floor of a block of flats that they rent and they're not allowed to put a nail in a wall. Even though they look the same on paper or in the algorithm, their need for care is really different and an assessor can take that into account and an algorithm can never take that into account. We will now.
Speaker 5Recommence the discussion inquiry into support at home. And I now welcome Professor Cathy Eager. Professor Eager, thank you so much for joining us.
Speaker 1And Cathy, you presented to a Senate inquiry and demonstrated that a single point on the algorithm scoring system could mean the difference between someone getting $ 40, 000 or $ 80, 000 in annual care funding.
Speaker 2And on the basis of a one-point difference in a 48-point scale is a $ 40, 000 difference for the person.
Speaker 1How is that even possible?
Speaker 2Yes, it's crazy. This is due to the fundamental errors in the design of the algorithm. If somebody gets a score of 24 or less, they can actually be allocated up to 80,000. But if they get a score of 25, a one-point difference. Then they might actually get 41,000. That's crazy. And it's because of technical design flaws in the algorithm.
Speaker 1We'll get to those design flaws in a moment. But our understanding is that the algorithm itself can't be overruled by the person doing the assessment. Is that correct?
Speaker 2Yeah, it's true. So what happened was the people who designed the system and the assessors were trained to believe that that in the event of an error in the algorithm, that the assessor can override the algorithm. And all the manuals were said that, the training was done on that basis, and the week before the system went live, everybody was told, well, actually, no, assessors are not able to override the algorithm. So even
Speaker 2though the IT system allows for override, They are not allowed to override it, even when they know that it's resulting in huge fundamental errors.
Speaker 1Now, Cathy, you've designed healthcare funding algorithms yourself. How did the government end up with a system that, as you say, has such fundamental problems? Was it even tested before it was rolled out?
Speaker 2The process was long and drawn out. And the first part involved engaging consultants, a consultancy firm, not a research group, but a consultancy firm, to design the assessment tool and the algorithm. So there were two parts. One is design what's called the integrated assessment tool that assessors use, and then that's fed into the computer algorithm. It appears that the public servants managing the project didn't like the algorithm
Speaker 2and that the consulting firm developed. So they then put that aside and a group of them formed a little working party and they designed the algorithm in-house.
Speaker 1Do we know how much expertise these public servants had when they were redesigning this algorithm?
Speaker 2Yeah, well, the public servants are very good expert public servants, but they're not clinicians and none of them have got a background in algorithms or or in funding system design, they are career public servants. That's what a public servant does. So they were doing something well beyond the expertise of a public servant. And it's not the fault of the individuals. The public service should never have taken on that job.
Speaker 2It did not have the skills for it. And even now, A year later, it still can't admit, the public service can't admit, that they got it wrong.
Speaker 1Still to come, the damning new report taking the aged care system to task.
Speaker 5So critically, we think that the legislation needs to be amended to ensure that there can be a change to an outcome if it's clinically not appropriate.
Speaker 1Cathy, this week the Commonwealth Ombudsman released a damning report into the assessment system. He found a reasonable person would lack confidence that the system was fair or consistent. Having investigated this yourself, what do you make of those findings?
Speaker 2Oh, look, I'm delighted by the Ombudsman and what he's come out and said in that review. The ombudsman didn't look at the algorithm per se. He rightly said that the ombudsman's office does not have the clinical expertise. So the ombudsman recognised that you need clinical expertise even if the department didn't. The ombudsman assessed how the system operated, and the most important recommendation from my perspective is the
Speaker 2first one, which is that assessors should be allowed to override the algorithm if the algorithm gets it wrong. The ombudsman also made a really interesting recommendation. which is that the department should undertake a root cause analysis to identify how it is that it made so many mistakes and where we've got such a flawed system. And that would be extraordinarily valuable if the department does it, which it
Speaker 2has actually agreed to do. The last one is a recommendation that the ombudsman made, which is that the Australian National Audit Office should now do a performance audit of the whole system and assess whether the department's use of public resources was efficient, economical and ethical. And I'm taking that to mean how is it that you could bring in a consulting group, pay them millions, and then on
Speaker 2the most important bit, which is the algorithm, a group of public servants completely ignored it and made up their own.
Speaker 1Yeah. The government has defended the algorithm, arguing that it's delivering faster and more consistent assessments and that relatively few people have challenged their results. What do you make of that defence?
Speaker 2There are two parts to that. One is that it's more consistent results. And what I said at the Senate about this is very true. If you want to consistently get things wrong, use an algorithm that's the wrong algorithm for the job. And I think the consistency argument is being used... in an incorrect way to imply that somehow that's equitable. But the algorithm does work for some people and it doesn't work for others. It's not equitable. It's just the same. And the second issue is that the
Speaker 2system is so complex. We are talking about frail, older people, a sizeable percentage of whom have dementia and Most people will be in this program for three years before they either die or go to residential aged care. Many people are just giving up because the only way you can appeal is to write a really complicated, fill in a really complicated form. When they went live, you had to
Speaker 2write a letter, put a stamp on it, take it and post it to a PO box in Adelaide. It was designed to make it almost impossible for older people to appeal, and so hardly anybody did. But that's not a sign that the algorithm's working. It's a sign that the system is designed to alienate a very frail population.
Speaker 1Cathy, what does it say about the way we're approaching aged care, that we're replacing the judgment of experienced assessors with an algorithm that rather than using technology to help them do their jobs better.
Speaker 2Well, what we're happening at the moment too, and I should just add this, is that a lot of very experienced assessors are leaving. They feel like data entry clerks and they hate the idea that they assess somebody and they know that they should be getting level six and they're getting level two and they're the ones who are getting the flack. We are losing experienced assessors in favour of an algorithm that is not fit for purpose. And
Speaker 2what government is doing is using an algorithm to ration support at home. And they think they're making a cost saving to government. But a person sitting at home not getting the support they need is not just quietly sitting there saving the Commonwealth money. They're ending up in emergency departments and in hospitals. They're going to residential aged care because they've had a fall. They've forgotten to take their
Speaker 2medication when they need to. They've got malnutrition. All these other things that go wrong when you leave an old, frail person at home without the support they need. So it's penny wise, pound stupid. And I think there are two issues. One is Government has adopted this new technology without actually understanding how to use it. And the second is that they're using this new technology well beyond what
Speaker 2the technology is capable of doing.
Speaker 1Well, Cathy, it sounds like a fundamentally flawed system. Thank you for bringing it to our attention.
Speaker 2Thank you.
Speaker 1Also in the news, a Darwin teenager will face court after being charged with terrorism-related offences related to possessing violent extremist material and instructions for making 3D-printed guns. The charges follow a joint investigation by Federal and Northern Territory Police and ASIO after a tip-off about extremist material being shared online. The Australian Federal Police says it's increasingly concerned about the speed at which young Australians are becoming radicalised online. And the Albanese government's free trade agreement with the European Union
Speaker 1is facing growing opposition, with farming groups urging Parliament to reject the deal. The National Farmers Federation says the agreement fails to deliver meaningful improvements for Australia's beef, dairy and sugar exporters, despite government claims it will boost the economy. The Coalition is also opposed to the deal in its current form, raising questions about whether it can secure the parliamentary support needed for the agreement to take effect. I'm Daniel James. You've been listening to 7am. We'll be back tomorrow.
At COP31 in Türkiye, Australia has been helping lead global negotiations over the future of fossil fuels and the fight against climate change. But while the Albanese government pushes for stronger international action,...
Anthony Albanese runs a notoriously tight ship. Across his two terms as prime minister, there’s been very little leaking, very little sniping, and very few of his Labor Party colleagues willing to speak out against him....
This week, Australia’s leading health research body released its advice on puberty blockers, saying they should be available for some young people under 18. The Queensland and the Northern Territory governments, which...
In 1945, in the Crimean city of Yalta, three global superpowers - America, Great Britain and the Soviet Union - met to carve up the world between them, each claiming their own sphere of influence in the final year of...
Last week it was Qantas, last month it was NAB: two giants of Australian business cutting jobs due to the growth of AI. The explanations were familiar. Qantas says it will allow the airline to unlock innovation faster...