Icon: enlarge
Atari console: Jump 'n' run games for AI systems
Photo: Robee Shepherd / FlickrVision / Getty Images
When it comes to complex games, humans usually lose out against computers.
Artificial intelligence (AI) has already beaten top players in chess or the Asian board game Go.
Even in curling, where tactical skills and a sure instinct are required, experienced teams were already subject to the technology.
It is all the more astonishing that in some computer games from the prehistoric times of the industry, humans have been ahead of the game.
Until now.
Because a computer system called Go-Explore now performed better against human opponents in all games for the Atari 2600.
Mind you: The console came on the market in 1977.
But with games like »Pitfall«, the algorithms used so far did not lead to success.
The platform game from 1982, in which the character Pitfall Harry is controlled by joystick through a jungle landscape and has to avoid obstacles, was too complex for the technology.
Researchers have now developed a kind of ambient memory for such tasks.
The computer system builds up an archive as it explores the game environment and always reverts to it when the character has to overcome new obstacles.
US researchers headed by Adrien Ecoffet from Uber AI Labs in San Francisco report in the journal Nature.
Basically, the researchers approached the problem through so-called reinforcement learning, learning through reinforcement.
A system learns through trial and error which actions are rewarded and which are punished, for example, by deducting points.
But: "Existing algorithms for learning through reinforcement seem to have difficulties when complex environments offer little feedback," write the scientists, referring to the structure of the game.
The system gets a memory
So they added a kind of digital memory to the system.
Individual game states are saved.
After each round of exploration, the AI selects the state or path in the archive that is most likely to be successful.
In this way, the AI uses the empirical knowledge, even if it has not yet been strengthened by the desired success, for example by creating a level.
With this addition to the reinforcement learning algorithms, Go-Explore was able to collect points at »Pitfall« - something that most AI systems have not yet succeeded in doing.
Go-Explore was slightly better than the average human player.
In the game "Montezuma's Revenge", the AI developed by Ecoffet and colleagues even set a world record after learning from human players.
In eleven games for the Atari 2600, Go-Explore performed better than average human gamers and as state-of-the-art AI systems.
According to the researchers, their approach can also be helpful in robotics.
They applied their algorithms to a robotic arm that was supposed to learn to place an object one after the other in four compartments, two of which were locked.
In contrast to other AI systems, Go-Explore never forgot when it had already opened the compartments.
This enabled the AI to place the item quickly and reliably in the four compartments.
The scientists name language comprehension and the development of new active ingredients as further possible applications of Go-Explore.
Jan Peters from the Max Planck Institute for Intelligent Systems in Stuttgart describes the study as a breakthrough.
"Beating human experts on so many problems is an impressive achievement." However, he considers the application potential in robotics to be limited.
The approach could bring about breakthroughs in medicine, autonomous driving and other safety-critical applications.
With regard to the Go-Explore system, Claus Horn from the Zurich University of Applied Sciences said: "It will enable us to solve more complex problems that require a longer sequence of decisions to be solved."
Icon: The mirror
joe / dpa