Australia moves to curb social media algorithms and faces scrutiny over aged care AI tool

Australia moves to curb social media algorithms and faces scrutiny over aged care AI tool Getty Images

Australia is pushing further into tech regulation, with the government introducing new legislation that would force social media companies to allow users to opt out of algorithmic content feeds. The country already became the first in the world to ban social media for minors.

The proposed laws also aim to strengthen online child protection. Opposition lawmakers have argued the measures would amount to government censorship.

The move comes as Australia faces separate and growing scrutiny over another algorithm — one used within its aged care system to determine funding packages and priority of need for older Australians requiring home support.

Documents released under freedom of information laws reveal that people with dementia were warned not to apply for home support assessments, due to fears their care would be downgraded by the government's aged care algorithm, known as the Integrated Assessment Tool, or IAT.

The warnings emerged within weeks of the IAT's rollout in November. Days before the algorithm went live, the government barred aged care assessors from overriding its decisions, even where the level of funding or priority allocated was considered wrong or inadequate.

Dementia Australia wrote to senior health department staff on 8 December, stating that since the algorithm's introduction, "We remain very concerned about the feedback we are receiving."

The organisation cited the case of a man caring for his wife with dementia, who held a level three home care package entitling her to around eight hours of support per week. As her condition progressed and a reassessment was needed, the husband "was advised by his local assessment service not to pursue a reassessment as they have seen a number of people with dementia receiving less or no support following IAT assessment, even if their care needs are increasing."

The Older Persons Advocacy Network sent a paper to department officials in February warning of deaths occurring due to the failing assessment system, and of people with "extreme needs" being left without adequate support before their deaths.

"The inability of clinicians to intervene" and "over-reliance on automated decision-making is contributing to preventable deterioration and in some cases, death," the document stated.

In one example cited by the network, the algorithm refused a funding upgrade for a rapidly deteriorating person, forcing them to pay for private care despite the assessor's view that their existing package was grossly inadequate.

Internal documents also show a senior health department staff member shared concerns ahead of a meeting with advocacy groups in late December. "Assessment orgs are telling [people] not to get an assessment as there is no guarantee they will get more funding," the internal email read. It added that assessors were telling advocacy groups they had not been properly trained on the IAT.

Ageing Australia, the national peak body for aged care, also sent senior department officials a comprehensive discussion paper in February outlining issues with assessments, including examples of people being under-assessed.

Prof Tanya Buchanan, CEO of Dementia Australia, said: "We remain concerned about the IAT and the aged care assessment process being able to meet the needs of Australians living with dementia, their families and carers." She called for the return of clinical oversight within the system.

A health department spokesperson said the government would not return human override powers, but would instead legislate a new "assessment escalation pathway," giving assessors the option to raise cases to a system governor — the secretary of the Department of Health, Disability and Ageing.

"It will sit alongside the existing review provision where an older person can request a review by contacting the department," the spokesperson said. "Assessment organisations themselves will initiate a referral based on the clinical judgement of the assessment delegate."

Carers and advocates have repeatedly raised concerns that review processes for incorrect algorithm decisions are lengthy, and that older people risk deteriorating or dying while they wait.

Samantha Edmonds, director of policy at the Older Persons Advocacy Network, said that without further details of the escalation pathway, "it is difficult to determine the effectiveness of the process."

"The IAT algorithm has been an ongoing challenge and concern for older people," she said. "These assessments have a real impact on an older person's life, so it is crucial that human oversight remains part of the decision-making and outcomes."