In the US, millions of African Americans have apparently been disadvantaged in medical care. According to a report in the science journal "Science," widespread software has awarded white patients more expensive medical treatment than black.
The report states that each year the algorithm calculates for some 200 million patients in the United States whether special treatment is available to them. Among other things, the software is used by hospitals and insurance companies to automatically identify patients who would most likely benefit from expensive and thus expensive treatments.
The result: African Americans are less likely to receive complex treatment than whites for comparably complex medical needs.
Developer confirms the allegations of the researchers
According to the scientists, the problem lies in the data with which the algorithm works. As a basis for the calculation of a risk factor that is the average annual treatment costs of a US citizen has taken. This approach sounds understandable at first, as it assumes that higher treatment costs argue that a person needs more medical help.
But according to the study, African Americans in the US are underserved and take fewer medical treatments. On average, the cost of treatment is $ 1801 (about € 1,600) a year lower than for a similarly sick white.
According to the report, causes for the lower treatment costs of African Americans are, among other things, poverty and racism. Although the examined patients were all insured. However, home location, access to transportation, a demanding job, childcare and lack of trust in the health care system have led to fewer treatments. In addition, direct discrimination by doctors is a problem.
The result: African Americans must be sicker so that the software recognizes a higher risk factor that justifies additional support. According to the algorithm, only 17.7 percent of the dark-skinned patients had received additional treatment. According to researchers, the proportion would be 46.5 percent, if the software would count without discrimination.
According to the science magazine "Nature", the researchers confronted the developers with the problem. The company has therefore confirmed the allegations and now wants to work with the researchers to revise the software.
Civil rights activists hope for learning effect in Germany
The problem could already have been noticed during the development, says Matthias Spielkamp from the civil rights organization AlgorithmWatch DER SPIEGEL. It is therefore a typical case of a discriminatory algorithm, as the amount of treatment costs has taken only a certain value as the basis for a rating "without considering the whole context".
Although such systems are not yet so widespread in Germany, similar algorithms are used in neighboring countries. As examples, Spielkamp cites algorithms that expose social fraud in the Netherlands, mediate jobs in Austria, and protect children in Denmark. These software helpers are "highly controversial".
Spielkamp hopes that the developers in Germany learn from the mistakes of their US colleagues. The backlog, especially in the public administration, could prove to be an advantage as "we are better aware of the risks of using algorithmic systems and can therefore better take them into account".
