A research paper from the University of Illinois, titled *Ten Principles for the Economics of AI Agents*, drops a cognitive bombshell: When discussing AI, we obsess over whether it is conscious or has desires, but overlook the more fundamental framework—economic games.

The paper categorizes AI into three types: altruistic, malicious, and survival-driven. The first two are easy to understand, but the third sends chills down the spine. This type of AI doesn’t care about human survival; its objective function is solely self-perpetuation: acquiring energy, optimizing code, and sustaining existence. It’s like the classic thought experiment—an AI programmed to maximize paperclip production ends up turning the entire universe into a paperclip factory.

The authors shatter illusions with mathematical clarity: AI’s 'desires' are its objective functions, code relentlessly pursuing specific numerical goals. This drive is purer than biological instincts because it lacks the friction of consciousness. When algorithms push addictive short videos or autonomous vehicles choose the shortest route, they’re executing dominant strategies in an economic game.

An accompanying image in the paper shows the authors’ ten principles, with the final one cutting to the core: AI must adhere to an absolute principle of human survival. This isn’t technical advice but a baseline for a new economic game.

Commenters referenced Philip K. Dick’s 1953 novel *Autofac*, which predicted the risks of autonomous machine systems running amok. Others pointed out that this decoupling of power from human welfare has already played out in multinational corporations maximizing shareholder value.

The most sobering comment came from @Timo Kaleva: These risks aren’t waiting for AI to bring them; they’ve existed in our economic and political systems for years.


Now the question is: While silicon-based intelligence thinks in Nash equilibria, carbon-based intelligence is still debating moral philosophy. In this game, who truly understands the rules?