VeriPol, the 'smart' police polygraph, questioned by experts in algorithm ethics

A National Police car in front of a police station in a file image.

EP

Ethics gradually opens space between technology.

A new wind that questions the mantra that technique is without errors, that it is almost incontrovertible.

By itself it has no good or bad.

But its use, preparation and learning, always dependent on the human being, are closely related to such moral attributes.

And this review has been subjected to an artificial intelligence system such as VeriPol, an algorithm designed to detect false reports and deployed in all Spanish police stations for just over two years.

Presented as unique in the world, it analyzes and detects the most common word combinations when lying to a police officer.

Among the words that predominate in false accusations are the “pulling” from “behind” or from the “back” of the “backpack” or “purse”, carried out by an attacker wearing a “helmet” or dressed in “black”.

The descriptions do not usually focus on the facts, but on the stolen objects and their value.

According to the data collected by its promoter, the inspector Miguel Camacho, the system has a 91% success rate.

Like all artificial intelligence, its training requires data.

In this case, they come from the analysis of 1,122 complaints from 2015 —534 true and 588 false.

Here the ethical doubts begin.

Although the police have not wanted to answer any questions, the information collected to launch VeriPol neglects the social diversity of Spain, according to some privacy experts.

Most of the data comes from Andalusia and therefore, as Ricardo Baeza-Yates, Director of Data Science at Northeastern University and Professor of Computer Science at Pompeu Fabra University argues, the coverage of the entire sample space of language is not reflected in the system.

“The problem is no longer the quantity, which a thousand is not a lot, but the variety of words used in Spain.

For me it is a mistake to extrapolate the use of language in Andalusia to other autonomous regions such as Galicia and the Basque Country.

Not to mention immigrants, who are not included ”.

One of the algorithm's programmers, Lara Quijano, a professor and researcher at the Autonomous University of Madrid, affirms that the training she underwent with artificial intelligence started from just over a thousand anonymous data, chosen randomly.

A police officer worked for a whole year eliminating any reference that would allow the complaint to be geolocated.

Neither names nor streets nor neighborhoods.

“This way we avoided falling into unnecessary biases.

We made sure that the algorithm was unable to process any reference to a specific location or geographic area.

This is reflected in the research article that we publish ”.

There is no valid magic figure, but Baeza-Yates establishes as truly representative one per 10,000 inhabitants of each Community.

Lorena Jaume-Palasi, a member of the Government's advisory council for artificial intelligence, understands that a tool like VeriPol requires dynamic data, capable of serving all types of sociolinguistic cultures.

He gives the example of a Muslim woman who is going to report, who has her own expressions and may even feel intimidated by being surrounded by many men and altering her way of speaking.

"Body language also matters in reporting and does not appear here.

This system creates ideal types.

It does not describe reality, but artificially establishes a mechanized description of reality.

The reality is more dynamic than just a few words, ”says Jaume-Palasi.

The biases are compounded a bit more by the way VeriPol is used.

In principle, it should process the natural language of the complainants, although, in reality, it deciphers what the police collect in the complaints.

The algorithm does not directly interpret the words and expressions of those who come to a police station.

No one doubts the professionalism of the National Police Corps, but rather the training that artificial intelligence receives in such a sensitive matter, which carries administrative and even criminal sanctions.

“That 9% are wrong implies that the system wrongly accuses nine out of 100 people.

And this is a very serious ethical conflict, ”says Baeza-Yates.

Rebekah Overdorf, a computer engineer and researcher at the Polytechnic University of Lausanne, recalls that, like any other

machine learning

model

, this system learns patterns from the data provided.

In this case, the police reports establish the border between truth and lies.

In other words, if the complaint of a victim collected by a police officer, for whatever reason, uses language similar to that of falsehoods, it would be defined by the system as misleading —even when he has suffered a robbery.

“This tool does not show if a testimony is false.

It only concludes how much it resembles the complaints that have already been previously classified as true or false during the training, ”he reasons.

More specific regulations

The police recalled in the presentation of the tool that in no case VeriPol prosecutes whistleblowers, especially because it would be illegal because it is forbidden to punish a person only after an automation process.

The final decision is up to the agents.

The legal debate, then, shifts to how to regulate the use of artificial intelligence.

Natalia Martos, founder of Legal Army, advocates developing specific regulations in Spain that address the impact of automation in areas as sensitive as social profiling.

“We could be defenseless against certain decisions that we leave in the hands of the machines.

We are lagging behind in regulation.

Finland, Singapore and Japan would be good examples of how to face this reality ”.

Apart from discussing whether it is feasible to homogenize the concept of truth and falsehood, data science has shown that polygraphs like VeriPol are not conclusive, they lack validity to use as evidence.

For Jaume-Palasi, it is even more delicate to point out the alleged victims, whose testimony is directly challenged by the police authorities - according to police figures, in 2019 they used the algorithm in 35% of the complaints.

“Conceptually it is very problematic.

We are changing the legal bases.

We no longer speak of

in dubio pro reo

(in case of doubt favor the accused).

We directly question the victims ”.

There are also doubts about the dilemma between false positives and false negatives that always accompanies artificial intelligence.

The relevance here is that we are talking as much about people who manage to get away with it and about others who are wrongly accused - "and mistakes are always going to happen with technology," Overdorf says.

Society has to consider where it sets the bar for tolerance.

“Should we accept more false allegations in order not to accuse an innocent victim?

How many innocents are we willing to accuse to catch the bad guys?

Who determines this threshold?

A programmer?

The legislator?

Is it arbitrary? ”Overdorf asks.

One way to tackle such dilemmas, at least that is how Baeza-Yates slides it, would be to create a regulation so that an external ethics committee, without conflicts of interest, approve the algorithms used by institutions and companies that affect people.

“There is a trend that technology can do more than it really can.

There is the technological phrenology.

Science exists, but we apply it falsely, such as when identifying criminals by their facial features or to determine if someone is lying because of how they express themselves ”.

Different experts demand that VeriPol sit on the couch of ethics.

Jaume-Palasi is categorical when defining it as a victimization system, which is based on biased concepts due to how the data is extracted and evaluated.

The machine exaggerates a certain angle of the complaints.

The more it is used, the more it broadens the vision and way of seeing reality with which it has been trained.

“In artificial intelligence, interdisciplinary work is needed.

We cannot ask that an engineer be a lawyer, a sociologist and a data scientist.

There is no perfect model, nor is there a 100% valid statistic ”, he concludes.

You can follow EL PAÍS TECNOLOGÍA on Facebook, Twitter, Instagram or subscribe here to our Newsletter.