A new book by Sebastian Mallaby, "The Infinity Machine," about Demis Hassabis and DeepMind has recently been released. Today, several publications published an exclusive excerpt from it, revealing a very interesting story from almost a decade ago.

In 2014, after Google acquired DeepMind, Demis Hassabis and the team began working on Project Mario. The goal was to understand how to properly control AGI and how to prevent a single corporation from gaining absolute power over such technology.

They explored governance structures, options to limit corporate power through financial methods, forms of independent bodies, etc. During the project, they realized that none of this worked, and that if AGI emerged at DeepMind, curbing Google's power over it would be impossible, and it could all lead to a huge threat to humanity.

And then… a secret hedge fund team appeared at DeepMind, trying to beat Renaissance Technologies, i.e., to create a system that predicts markets better than the best. They wanted to train a model similar to AlphaGo for this purpose.

Thus, they hoped to earn their own capital to maintain control over AGI themselves, separating from Google if necessary.

The irony is that they all believed so much in AGI based on their AlphaGo and AlphaZero models, and were so engrossed in Project Mario and the hedge fund, that they literally missed the significance of the transformers invented by their colleagues in 2017.

And Project Mario, by the way, ended with only one idea left at DeepMind: control through people, not through the system. That is, through trust in specific founders who make decisions based on universal human values.

colossus.com/article/project-ma…is-deepmind-mallaby/