AI makes intellectual contribution superfluous. In this post, I argue that individual intellectual activity can only be sustained in an intellectual community of other humans. AI dissolves these communities, which in turn makes private intellectual activity rarer. The Case of Software Some years ago, when it became clear that AI would solve software engineering, my thinking was: In my professional

The transhuman future vs. the AGI future. For us, humanity was a distant goal toward which all men were moving, whose image no one knew, whose laws were nowhere written down. — Hermann Hesse, Demian Despite having spent most of my life marinating in transhumanist ideas, I never cared about AI. Because I’m a classical transhumanist: I want to use technology to change myself, so that I can do more.

How and why I went from AI optimist to AI doomer. If you follow me on Twitter, or read this blog, you have noticed that I went from being generally optimistic and excited about AI to being extremely concerned. And I thought, I should explain why I changed my mind. Each section of this post is about some aspect of AI where my views shifted. I begin each section by explaining my previous beliefs, an

Why superintelligent AI is not inevitable. There’s a widespread belief that artificial superintelligence (ASI) is inevitable. In this post, I argue that this is false, and believing it is upstream of a lot of bad discourse. Why do people believe it? It’s easy to believe. Samuel Butler , already in 1863, observed that machines are improving faster than humans are improving. Today, AI systems are ga

Rigorous thinking about the future produces extreme outcomes. Any rigorous thinking about the long-term future leads to extreme outcomes. It cannot, by construction, produce normal or mundane futures. “Rigorous thinking” means you have a model: a formalism for representing the state of the world, and a transition function to advance the state. You put the world into the formalism, and turn the cra

On the rename system call. So, I have this little app, Epoch , that I use to keep a journal. It’s a tiny Rust web app that runs as a systemd service and uses SQLite as the database. I use a desktop and a laptop regularly, and use Syncthing to synchronize them, including Epoch’s database. That way I can use the app on both devices without needing a server to synchronize them, the tradeoff being tha

Why the post-scarcity utopia is unlikely. The implicit promise of modernity is: we have to work really hard for a few centuries, but eventually we’ll get to the Star Trek future where we’ve automated all the drudgery, and all work is meaningful, heroic, and dignifies the spirit. We will be artists, scholars, captains of the starship Enterprise , etc. There’s a few problems with this idea: “Drudger

Alignment as a rhetorical device. Many of the people building AI, and many of the people working on AI safety, share a common vision of what a good AI future looks like: We figure out alignment, build superintelligent AI, and it takes over the world, for our benefit. Many people will tell you that last part outright: they think human disempowerment is a good thing because the AIs will be smarter a

A list. Fiction Invisible Cities by Italo Calvino . Beautiful prose and imagination. By Night in Chile by Roberto Bolaño . Beautiful and surreal prose, but completely unpretentious. Valuable Humans in Transit and Other Stories by qntm . I had already read most of these but they are very, very good, and it was a pleasure to read them again. ⭐️ Life and Fate by Vasily Grossman . Usually when I hear

Why don't we believe LLMs are people? There’s a game called Mass Effect , which I haven’t played, but I’m told in the game there’s this species of sentient robots, the Geth , whose creators try to exterminate them, and the trigger for this Butlerian jihad was a domestic servant Geth looking up at its owner and asking: “does this unit have a soul?”. And if you’re playing the game in 2007 you probab

Review of Marcus Hutter's Job-Less Utopia. Marcus Hutter —oldheads will remember him from AIXI —recently published a book: Job-Less Utopia: Macroeconomics in the Age of AGI . I’m always curious what the people building AI think or want the future to look like, and I’m also always looking for non-doomer takes on the post-AGI future, so I decided to read it and review it. The first thing to say is t

We must know. We shall not know. Yesterday, OpenAI announced the solution to ten open problems in mathematics, all discovered by a yet-unreleased model. There’s only one, the coding theory one, where I know enough to say “huh, that’s important”, but according to the mathematicians I trust this is important. Inevitably, people will spin some sophistry to cope, to convince themselves nothing will ch

What does the world look like when AI outcompetes humans? Today, the world is made of people. This gives people a tiny measure of leverage: factory workers can sabotage the machines and halt production, truck drivers can strike and halt the entire economy, the police can stand back and let a protest topple the government , the military can mutiny . This sets a limit to how much the powerful can ab

Capital won't save you from disempowerment. Shall I end this life a pauper? If AI can do all work at human level or better, what stops corporations replacing us all with AI? This is the permanent underclass meme. The idea is: within a few years, all white collar work will be automated by AI, at which point there is no social mobility. The main way people cope is, they tell themselves: if I work ha

Or, sufficiently advanced cynicism is indistinguishable from naïveté. The most stubborn facts are those of the spirit, not those of the physical world. — Jean Gottmann, Geography and International Relations In 1914, before the First World War, there was this belief : “a European war would be economically disastrous, the moneyed classes won’t let it happen”. Europe went to war anyways, and the war

On people who use AI to think. When I open a link, say on Hacker News, and I see a blog post or a GitHub README obviously written by AI, I feel a few things. I feel offended, because it’s like I’ve been tricked, like the author thinks I’m a rube who won’t notice or mind. I feel sad at how common this experience is, how many people are happy to dump their sewage on the commons and sign their name o

How text can outcompete the screen. The book can’t compete with the screen. It couldn’t compete beginning with the movie screen, it couldn’t compete with the television screen, and it can’t compete with the computer screen. — Philip Roth We’re halfway through 2026, and according to Goodreads I’ve read 80 books so far, fiction and non-fiction and textbooks, including such doorstoppers as Life and F

Why AI won't augment human capabilities much. AI models are very capable and get more capable each year. So naturally people feel they’re underusing them. There’s a tweet that goes like: your laptop has a 100M USD startup in it, you just have to figure the right sequence of words to get it out. And beyond money, people imagine AI could boost them in every area of life. Thus all these perennial ide

On factual vs. conceptual knowledge. Spaced repetition has a natural domain of applicability: information that is systematically organized as an unambiguous key-value mapping with short keys and values. The “Hello, world!” of flashcards is the NATO phonetic alphabet : A → alpha, B → bravo, etc. Similarly, the periodic table can be thought of as defining a collection of mappings: element name ↔ sym

Cruise ships are bad for the world. Right now there’s a cruise ship parked outside Cabo Verde because of an outbreak of Andes virus . Yep, another cruise ship. I don’t get the appeal. It’s like a big open-air serial passage experiment: you get a bunch of old people with failing immune systems in close contact and race a pathogen through them. How much should I worry about this? Is this early Janua