Enlarge image
Comparison of graphics: Scientists make the game look more realistic with their AI filter (right).
Photo: Rockstar Games / Intel
Razor-sharp palm fronds, cracks in the asphalt, reflections from the sun on car roofs: the virtual Southern California in the video game "GTA V" still looks impressive years after the video game was released, especially in its PC version.
At the same time, the world simulated in the game can still be easily distinguished from the real one.
Scientists at the American chip manufacturer Intel want to change that with image enhancement software.
The researchers have designed a software model (PDF) that is based on artificial intelligence (AI) and that makes the streets of the city of Los Santos look much more realistic.
The team uploaded a video to YouTube showing the improved graphics.
With the AI filter, the asphalt looks more authentic, the cars reflect the light more beautifully and the trees also look like they were filmed with a camera from inside the vehicle.
Stephan Richter has been working on the project with two colleagues for two years.
"Our goal was to make the game look as real as possible," said the Intel researcher in an interview with SPIEGEL.
"Game simulations often don't look realistic enough: The graphics are not yet sufficient to depict the real world." A lot of work went into the project before the images looked like they do now.
Software branches off information for the graphics card
In order to generate clean images, the researchers put on a double check: On the one hand, the fully rendered game image is sent through the AI filter.
At the same time, however, the software also uses so-called G buffers.
This contains information for the graphics card to correctly calculate geometric details such as the distance between buildings and the camera and the position of the light source.
The researchers branch off the G-buffers to make it clear to the AI not to place trees in the sky.
Finally, the resulting image is subjected to a reality check.
The software compares the result with street snapshots and assigns authenticity points.
Enlarge image
Even if the color is adapted to the German street mood of the training images, the original images (left) appear more artificial than the researchers' filtered video.
Photo: Rockstar Games / Intel
In order to depict reality as precisely as possible, the software uses, among other things, training images from Cityscapes.
This photo database contains streets from 50 predominantly German cities such as Erfurt, Cologne and Stuttgart.
The pictures clearly indicate whether an object is a pedestrian, a traffic sign or a bus stop.
Not everything is right
While roads, trees and cars really look more realistic, the image enhancement reaches its limits with some objects - also because palm trees on the roadside are rather rare in this country.
The software sometimes interprets cranes as traffic lights and park benches are turned into rolled up awnings.
According to a report by the scientists, pedestrians are also "processed less convincingly".
Despite such problems, researchers believe the technology could bring more realistic graphics to games.
"I can well imagine engine developers using this software as a reality filter," says Richter.
“GTA is only a proxy here.
This can be applied to many games. "
In its current form, however, the software is still too slow.
Even with the top model GeForce RTX 3090 graphics card from Nvidia, the image improvement only spits out the result half a second later.
According to Richter, at the moment it is a prototype "that cannot be plugged directly into a game at the moment."