Icon: enlarge
In the future, AI software should independently discover recordings that show cluster munitions
Photo: Adam Harvey / VFRAME.io
Scientists and designers use 3D printers to produce jewelry models, prostheses, parts of houses or organs, while US artist and tech expert Adam Harvey prints cluster munitions such as the “AO2.5RT” instead.
It was developed in the Soviet Union, is about the size of American football - and is usually highly explosive.
More than a hundred countries around the world have signed the Convention on Cluster Munitions and are committed not to manufacture, pass on or store any more cluster bombs.
Nevertheless, they are still being dropped in Syria and Yemen, for example.
The bombs release many small explosive devices, which are spread over large areas - and make no distinction between warring parties and civilians.
Duds that don't explode instantly are still sharp decades later and can maim or kill people.
Icon: enlarge
Ammunition from the 3D printer: Recordings of the processed model are later used to train an algorithm
Photo: Adam Harvey / VFRAME.io
Adam Harvey, who lives in Berlin, has never seen a real cluster bomb: the 39-year-old researched details on the Internet and calculated the dimensions of the ammunition by comparing it with other objects in photos and videos.
He paints his 3D models, photographs them from many perspectives, some of them damaged - in environments that correspond to the original locations of real ammunition, for example on sandy ground.
With the recordings, Harvey trains an algorithm that is intended to support civil society initiatives in the future to search through millions of photos and videos for illegal cluster munitions - in order to prove who dropped them and where.
"They know what they're looking for," says Harvey.
"The tool helps you find the needle in the haystack."
Icon: enlarge
Cluster munitions, which are used in the civil war in Syria, for example, often kill and injure civilians
Photo: Ibrahim Hatib / Anadolu / Getty Images
Digital evidence, such as publicly available footage from social networks, is playing an increasingly important role in solving war crimes - and automated image recognition processes and other AI software could speed up their analysis considerably.
"The use of digital open source information as evidence is still relatively new in international human rights and war crimes trials, but is quickly becoming the norm," says human rights attorney Lindsay Freeman from the Californian "UC Berkeley Human Rights Center".
At the end of last year, the center published guidelines together with the UN Commissioner for Human Rights in order to establish standards for digital investigations.
The expert regards the Syrian civil war, which has now been going on for ten years, as the "first social media war" - the government, soldiers, police, rebels, citizens and activists photograph and film en masse everyday life and attacks and distribute content via social networks , the terrorist militia IS developed a professional terror PR strategy.
Icon: enlarge
Eyewitness with a cell phone: A young Syrian documents the damage after an air raid
Photo: Bassam Khabieh / REUTERS
But Syria is only the beginning of a much larger trend: "From Myanmar to Yemen to Venezuela, we see how citizens and other actors use their smartphones to document what is happening," observes Freeman.
"It's now hard to imagine that there could be any international criminal process that isn't based on such information in one way or another." Tech experts and legal teams are increasingly working closely together to advance the investigation.
The non-governmental organization Mnemonic plays a key role in the documentation of war crimes - but the digital detectives are drowning in the flood of images.
"We are currently trying to understand exactly what data we have accumulated over the past ten years," says the 36-year-old mnemonic founder Hadi al-Khatib.
"We hope that artificial intelligence can help us to identify certain content in our database and to organize our material more meaningfully."
When the confusing Syrian civil war broke out in 2011, al-Khatib began saving recordings from his home country with his project “The Syrian Archive” so that they would not be lost - because the search for digital evidence is a race against time.
Icon: enlarge
Initially, Hadi al-Khatib archived alone - today Mnemonic has 20 employees
Photo: HC Plambeck / HCPlambeck
Since terrorist attacks have increased in the West and the perpetrators use social networks specifically for their propaganda, the pressure on tech companies like Facebook to delete depictions of violence has grown.
The community guidelines have been tightened, and social networks also rely on AI software: algorithms filter or delete recordings showing weapons, violence and terror from the platforms more frequently and more quickly - but important information about crimes and grievances are also lost in the process.
The 20 employees at Mnemonic have so far been working largely manually: They search for specific keywords, dates and events in social networks such as Facebook, YouTube, Instagram or Twitter, evaluate messenger groups on Telegram or WhatsApp, but also receive mobile phone videos that provide them, among other things pass on to local human rights activists.
The team then also verifies the content using satellite images.
The archives on Syria, Sudan and Yemen already contain several million photos and videos - the organization recently published a database showing targeted air strikes on Syrian hospitals.
Together with human rights lawyers from the European Center for Constitutional and Human Rights (ECCHR) or the Open Society Justice Initiative, the digital forensic scientists are also trying to bring cases to court - a criminal complaint against the Assad regime and evidence of poison gas attacks in Syria are currently with the German Attorney General.
In the coming months, Adam Harvey's "VFRAME" software will identify and categorize different types of cluster munitions in the material stored in the archives.
This shows the extent to which certain actors continue to use them for attacks despite the convention: “The Russian and Syrian governments have so far denied that they are using cluster munitions against civilians in Syria - but they are the only ones in the conflict who have this type of Own weapons and be able to use them there, ”says Hadi al-Khatib.
Icon: enlarge
In Myanmar, security forces brutally attack peaceful demonstrators - including with live ammunition
Photo: Myat Thu Kyaw / NurPhoto / Getty Images
In autumn, the team wants to launch a database on cluster munitions that journalists and lawyers can use to research.
Harvey is currently refining the algorithm's hit rate with additional training material: "The algorithm must be reliable enough that it has an 80 percent probability of correctly marking videos," he says - so that the tool does not waste the team's time.
The application of artificial intelligence is still a field of experimentation for non-governmental organizations.
In many places there is a lack of time and money to develop complex AI projects that make sense for the special purposes of human rights activists or environmentalists.
Still, there are some creative approaches that show the potential.
Sound monitoring systems filter out noises from the rainforest, for example, and algorithms evaluate them in real time - local aid troops are alerted when chainsaws or shots indicate illegal loggers.
Researchers are trying to automatically evaluate satellite images or to calculate future forest destruction.
In the “Decode Darfur” and “Decode the Difference” projects, “Amnesty International” combined crowdsourcing and artificial intelligence to map attacks in Sudan: Thousands of volunteers used satellite images to search online for clues such as destroyed villages.
Even less spectacular AI applications can provide advantages for digital forensic experts: the "Samdesk" software automatically plows through the Internet and alerts Amnesty International's "Evidence Lab" if something happens - like the violence in Ethiopia or Myanmar.
On a clearly arranged platform, Sam Dubberley and his team of five can view eyewitness reports, information from social networks such as Twitter or Snapchat and local media - and thus come across explosive material such as videos or live streams, which they then check in detail
"We have to react very quickly because our goal is to stop the abuse of power immediately," says Dubberley.
The team recently demonstrated, for example, with an evaluation of 55 videos that Myanmar's military is targeting demonstrators with live ammunition.
Icon: enlarge
In the work of Sam Dubberley, director of the Evidence Lab at Amnesty International, it is important to react quickly
Photo: Amnesty International
The analysis of the videos and their comparison with satellite data is not yet automated here: "The machine learning techniques are not yet developed enough to be used in the heat of the moment," believes Dubberley.
"I wish we had more time for experiments - then I would look at the extent to which machine learning can evaluate satellite images or algorithms in thousands of videos from Africa can recognize military vehicles and armored vehicles."
Harvey's software for cluster munitions will in future serve precisely such purposes - and at some point not only detect cluster munitions, but also other objects such as certain types of vehicles, other weapons and ammunition, aircraft or hospitals.
Icon: The mirror
This contribution is part of the Global Society project
What is the Global Society project? Up arrow Down arrow
Under the title Global Society, reporters from
Asia, Africa, Latin America and Europe
report on injustices in a globalized world, socio-political challenges and sustainable development.
The reports, analyzes, photo series, videos and podcasts appear in the international section of SPIEGEL.
The project is long-term and will be supported for three years by the Bill & Melinda Gates Foundation (BMGF).
A detailed FAQ with questions and answers about the project can be found here.
What does the funding look like in concrete terms? Up arrow Down arrow
The Bill & Melinda Gates Foundation (BMGF) is supporting the project for three years with a total of around 2.3 million euros.
Is the journalistic content independent of the foundation? Up arrow Down arrow
Yes.
The editorial content is created without the influence of the Gates Foundation.
Do other media have similar projects? Up arrow Down arrow
Yes.
Big European media like "The Guardian" and "El País" have set up similar sections on their news sites with "Global Development" and "Planeta Futuro" with the support of the Gates Foundation.
Have there already been similar projects at SPIEGEL? Up arrow Down arrow
In recent years, SPIEGEL has already implemented two projects with the European Journalism Center (EJC) and the support of the Bill & Melinda Gates Foundation: The "Expedition The Day After Tomorrow" on global sustainability goals and the journalistic refugee project "The New Arrivals", as part of this several award-winning multimedia reports on the topics of migration and flight have been produced.
Where can I find all publications on Global Society? Up arrow Down arrow
The pieces can be found at SPIEGEL on the topic Global Society.