DeepMind Creates Imaginative Game Visual AI Without Coding

DeepMind introduces GameNGen, an engine that simulates games realistically like playing real games, but internally it is a model created with Stable Diffusion images trained with game images and various controls. This model relies on creating an agent that plays real games, captures images and actions of players in the game (walking, shooting, running) continuously. Then the images and actions are used to train a generative model to predict the next frame from the starting frame, resulting in a model that can represent game engines, controllable and playable. The model has the ability to count energy or the number of bullets. The team found that during training, it is necessary to insert garbage images in between to make the model work smoother.

This approach, if further developed and put into practical use, may open up avenues for developing new types of games. Developers may insert images along the way in the game to experiment on how the game being developed will look.

Source: DeepMind

TLDR: DeepMind introduces GameNGen, a model that simulates games realistically by training with game images and player actions to predict the next frame, potentially leading to the development of new types of games.