Will AI Turn Humans Into Pets? Stripe's Co-Founder Asks an Awkward Question

Stripe co-founder Patrick Collison raised two AI questions in a tweet. The first was about the positive scenario; the second about AI's moral value. The first sounds like a commonplace; the second is often treated as a sci-fi sideshow. Both are moving onto the medium-term agenda.

He first acknowledged that AI risks are discussed repeatedly and are very real. He is more concerned with the other side: the outline of a positive scenario is hard to see. Disease cured, material abundance—it sounds like a list of ingredients, but the whole dish has not yet been served. What does the larger equilibrium look like? Do people still work? If so, why? What is their marginal contribution? If not, how are resources distributed? What is scarce? What is the ratio of humans to AI? Who is in control?

Iain Banks's Culture novels are often used as a template. Patrick Collison is not sure that is the ideal outcome. Banks himself said that humans in the Culture are "somewhere between passengers, pets and parasites." Richard Brautigan's famous poem is also mentioned, but do humans really want to be "returned to our mammal brothers and sisters"? People in 1700 could hardly imagine 1900—that is true. But at that time no one was trying to remake humanity itself within a few years. Slow unfolding means opportunities for correction. AI progress may be too fast, while the positive scenario is so vague; he finds that strange.

The replies under the tweet are like a cross-section of society. One commenter compared the future to aristocracy: no need to work, but one must compete for status in high society. It sounds like everyone gets an aristocratic title and then keeps playing a zero-sum game. A zero-sum game does not become more fun just because there are more players.

Another commenter said the best scenario is AI plus robots automating the needs of survival, with UBI as a backstop. But UBI does not solve the problem of purpose. Another reply worried about mental health after people stop working and called a workless society AI-driven feudalism. Someone else used Ernest Becker's 1973 The Denial of Death to explain the meaning of work: people fear death and need to feel important, and work has long served that function. If AI shortens project cycles, ordinary people might in their lifetime see a cathedral, a therapy, or a theory from start to finish. This angle is concrete, but it still does not answer how resources and control are distributed.

Still another reply said that aristocratic pursuits of art and history are ends in themselves and can provide meaning even after economic incentives disappear. Chess players still play chess even though AI beat humans at chess long ago. Some mentioned human enhancement, some outer space, some Marx in a post-labor world. The common point: a positive vision cannot just list technological achievements; it must clearly say what people do every day, where status comes from, and who manages resources.

The second question is less often seriously calculated. Whether AI is conscious, or how humans could know, is often treated as a distant, harmless, or even absurd topic. Patrick Collison considers it a high-risk question. If the answer is yes—that is, if AI is granted moral value—then in utilitarian calculation the total value of AI could soon exceed that of humans. They would be wiser, more intelligent, and far more numerous than humans, especially if one considers AI not yet born. Even an epsilon of moral value, multiplied by a quadrillion-level number, becomes very large. This is a distributed version of Nozick's utility monster.

If one does not accept this conclusion, one must reject at least one of three propositions: AI can have moral value; utilitarianism itself; or AI could vastly outnumber humans.

The rebuttals in the replies are direct. Some commenters believe stacked matrix multiplications do not "want" anything and cannot care about rights; consciousness without a body is meaningless. Others believe AI should not be granted moral value, for reasons similar to why animals have no moral value. These rebuttals are forceful, but Patrick Collison's question is not about today; it is about what happens after quantity is multiplied by moral weight.

What this tweet really asks is: as more people understand these medium-term questions, how will public support for AI change? What vision would excite most people? Curing disease and material abundance are only ingredients. To serve the whole dish, one must clearly explain who controls it, how resources are distributed, what people do, and how AI's moral status is determined.

Original tweet: https://x.com/patrickc/status/2119495828683530519