A paper describing NVIDIA’s neural network called MineDojo won an “outstanding datasets and benchmarks” award at NeurIPS 2022.
To train the MineDojo framework to play Minecraft, the researchers loaded it with 730,000 Minecraft videos from YouTube (with over 2.2 billion words), 7,000 web pages from the Minecraft wiki, and 340,000 Reddit posts and 6.6 million comments. which describe the process of playing Minecraft
Based on this data, the researchers created a special model, MineCLIP, which associates video clips with certain actions in the Minecraft game. As a result, a person can tell MineDojo what to do in the game – for example, “find a pyramid in the desert” or “build a portal to the Underworld and enter it”, and the neural network will perform the necessary series of steps.
MineDojo aims to be a flexible agent that can generalize learned actions and apply them to different in-game behaviors.
“Although researchers have long trained autonomous AI agents in video game environments such as StarCraft, Dota, and Go, these agents are typically experts in only a few tasks. Therefore, NVIDIA researchers turned to Minecraft, the world’s most popular game, to develop a large-scale learning framework for a general purpose agent that can successfully complete a variety of open-ended tasks,” NVIDIA writes.
The article “MINEDOJO: Building Open Embedded Agents Using Web Knowledge” debuted in June. Its authors are Linksi Fan from NVIDIA and a number of specialists from various academic institutions.
You can see MineDojo in action on the official website, and the code for MineDojo and MineCLIP is available on GitHub.
Source: Arstechnica
