What happened when a ‘wildly irrational’ algorithm made crucial healthcare decisions | US news

[ad_1]

Going up against an algorithm was a battle unlike any other Larkin Seiler had faced.

Because of his cerebral palsy, the 40-year-old, who works at an environmental engineering firm and loves attending sports games of nearly any type, depends on his home care support person for assistance with things most people take for granted, like meals and bathing.

Every morning, Seiler’s support worker lifts him out of bed, positions him in his wheelchair and helps him get dressed for the coming workday. The worker checks back in at lunch time to help with lunch and toileting, then returns again in the evening.

But when Seiler’s home state of Idaho created an automated system – an algorithm – to apportion home care assistance for people with disabilities in 2008, it cut his home care budget in half. He faced being unable to even use the bathroom at reasonable intervals.

“It was awful,” said Seiler, who feared he would be forced into an institution. “I can’t even get up in the morning if I don’t have help. This would take all my freedom and independence.”

Like Seiler, thousands of disabled and elderly people in more than a dozen states have had to fight against decisions made by an algorithm to get the support services they need to remain in their homes instead of being institutionalized.

The cuts have hit low income seniors and people with disabilities in Pennsylvania, Iowa, New York, Maryland, New Jersey, Arkansas and other states, after algorithms became the arbiters of how their home health care was allocated – replacing judgments that used to be primarily made by nurses and social workers.

Larkin Seiler, who has cerebral palsy, depends on his home care support person for assistance with things most people take for granted, like meals and bathing.
Larkin Seiler, who has cerebral palsy, depends on his home care support person for assistance with things most people take for granted, like meals and bathing. Photograph: The Guardian

In Washington DC, “on the worst end, we’ve had clients who actually died, because their services were cut and they were not receiving the care that they needed” said attorney Tina Smith Nelson about the effects of a new algorithmic system introduced in 2018. Over 300 seniors have had to file administrative appeals after their home care was cut by a new algorithmic system.

“I think as a society we move into unsettling territory when we rely solely upon algorithms and data to make determinations about health care needs,” Nelson said. “We reduce a person’s humanity to a number.”

Kevin De Liban, an attorney with Legal Aid of Arkansas, began fighting the cuts after severely disabled patients started calling “en masse” in 2016. “The human suffering was just immense,” he said. “You had people lying in their own waste. You had people getting bed sores because there’s nobody there to turn them. You had people being shut in, you had people skipping meals. It was just incalculable human suffering.”

For Arkansas resident Tammy Dobbs, life became nearly unbearable after her state brought in an algorithm which decimated the amount of care she received in 2016.

Dobbs, 61, needs assistance getting into her wheelchair and doesn’t have use of her hands due to cerebral palsy, but suddenly no one was there to even help her use the toilet.

“Things were hard because I had to program myself to go to the bathroom at certain times,” Dobbs said. “I had to put off taking a bath every day because I didn’t have time. It just was bad.”

The situation is reflective of a reality increasingly affecting all users of American healthcare: algorithms – ranging from crude if-then charts to sophisticated artificial intelligence systems – are being deployed to make all sorts of decisions about who gets care.

Government officials have touted algorithmic decision-making systems as a way to make sure that benefits are allocated even-handedly, eliminate human bias and root out fraud.

But advocates say having computer programs decide how much help vulnerable people can get is often arbitrary – and in some cases downright cruel.

The underlying problem, experts say, is that neither states nor the federal government provide enough funding to allow people needing health assistance to remain safely in their homes – even though these programs usually end up being much less costly than putting people in institutions. The algorithms resort to divvying up what crumbs are available.

Dobbs’s experience in Arkansas exposed the arbitrary decision-making that can affect the healthcare of millions of Americans when algorithms are employed without proper scrutiny.

For years, she had received eight hours of help a day from a home aide for everything from getting out of bed to eating.

Despite her cerebral palsy, Dobbs managed to live on her own and keep active through writing poetry, gardening and fishing.

But, in 2016, a health care needs assessor showed up with a laptop and typed in Dobbs’s answers to a long battery of questions. Then she consulted the computer and perfunctorily informed Dobbs she would receive only about four hours a day of help.

“I just started going berserk,” said Dobbs, whose story was previously reported by the Verge. “I said ‘No, no I can’t do that!’”

“But the interviewer said, ‘Sorry, that’s what the computer is showing me,’” she said.

Dobbs said she feared she would end up being institutionalized.

“I’ve known people who were put in a nursing homes and I’ve seen how they were treated, and I’m not going,” she said.

It wasn’t until De Liban began unravelling the new computer program behind the care cuts that it became clear that an algorithm was at play.

Every year a nurse would come to each patient’s home to administer a computerized assessment: 286 questions covering everything from mental health to how much help they need in daily activities like eating or doing their personal finances.

Then an algorithmic tool sorted patients into various levels of need. Each level was assigned a standard number of hours of care.

De Liban’s legal team revealed flaws with the algorithm in court. It turned out, De Liban said, that the calculations had failed to factor in things like whether a patient had cerebral palsy or diabetes.

A single point in the scoring system – for instance a point added because the patient had had a fever in the last three days or had open pressure sores – could make a huge difference in how many hours they received for the entire year.

Other problems came from mistakes by assessors. In one case, a person with double amputations was marked as not having a mobility problem, because he could get around in a wheelchair.

“As the algorithm worked, it was, to our eyes, pretty wildly irrational,” said De Liban.

Arkansas state officials did not respond to a request for comment.

The designer of the algorithm, University of Michigan Professor Emeritus Brant Fries, acknowledged that the system isn’t designed to calculate how many hours of care people actually need. Instead he said it has been scientifically calibrated to equitably allocate scarce resources.

“We’re saying we will take whatever the size of the pie is and we’ll divide that pie in a scientific way, the most equitable way we can, for the individuals involved,” he explained. “…We’re not saying that the size of the pie is correct.”

Fries, who began developing the algorithm more than 30 years ago, acknowledged that the programs don’t address what many see as chronic US underspending on nursing home and home care for low income, elderly and disabled populations.

“If you don’t have enough resources, then these people aren’t going to get enough money and maybe they’re going to be in soiled clothes – but so will everybody else,” he said. “A pox on your house if you’re not providing enough care. But whatever the money is there, I’m dividing it more equally.”

After years of court battles, Arkansas’ use of the algorithmic system was finally thrown out in 2018. One state supreme court ruling said it was causing participants “irreparable harm”, and that they “have gone without bathing, have missed treatments and turnings, faced increased risk of falling, have become more isolated, and have suffered worsened medical conditions as a result of their lack of care”.

So Dobbs has been able to get the care she needs to stay in the cheerful, white clapboard house she rents in a woodsy neighborhood of Cherokee Village, Arkansas.

“There are problems getting machines to make fair decisions about people’s lives,” she said. “It’s just a computer. It doesn’t see our circumstances. It doesn’t see the individual.”

But across the nation, the battle continues.

In Washington DC, Pennsylvania and Iowa, legal services attorneys are plagued with calls from seniors complaining they have lost their care because of the algorithms recently adopted in those states. In a few Pennsylvania cases, patients were left with so little help that protective services had to be called in to make sure they weren’t facing neglect, according to Laval Miller-Wilson, director of the Pennsylvania Health Law Project.

In Missouri, attempts to gather public input to develop a new system have stretched on for years, with disabled advocates fearing thousands of people will lose eligibility. After years of work on developing an algorithm to decide who is eligible for home care, the state has decided to essentially grandfather-in existing clients for the next two years, said Melanie Highland, director of senior and disability services for the state. But she acknowledged some may end up losing eligibility for the services after this period.

Advocates for people with disabilities say that deciding care algorithmically fails to consider the subtleties of individuals’ situations. They worry that decisions get made in a black box with patients having no way of knowing why, thus making rulings hard to challenge.

“The idea of a machine that is free of individual people’s personal compunctions might seem appealing,” said Lydia XZ Brown, an attorney and disability rights activist with the Center for Democracy and Technology, who is also autistic. “However, what people forget is that when you trust a machine, you are always trusting the people who design the machine and the people who are using the machine to be acting appropriately, ethically, or responsibly.”

The algorithm that cut Seiler’s care in 2008 was declared unconstitutional by the court in 2016, but Seiler and other disabilities activists are still engaged in a court-supervised process trying to replace it.

And since the court case started, Seiler’s home care budget has been returned to its original level and frozen there. For now, he is able to hire the assistance he needs. He worries his living situation may be threatened once again by the new algorithm Idaho is developing.

“The thought of having to go to a nursing home is the worst,” he said. “It’s a nightmare.”

  • The Guardian wishes to thank Elizabeth Edwards of the National Health Law Program, Richard Eppink of the Idaho ACLU, Michele Gilman of the University of Baltimore and other legal aid professionals who helped with the reporting of this story.

[ad_2]

Source link

Share

Leave a Reply

Your email address will not be published. Required fields are marked *

Share This