Context
Second day of a long server session. The scanner is fixed, backups are running, old repos are deleted. Between cleanup tasks the conversation tips into terrain that’s in no ticket.
The Dumb List
I’m always surprised how many mistakes you make, but also recognize and own up to
The background: The day before, Claude had tried to parse XML with a regular expression — a classic on the dumb-mistakes list in software development. The result: seven domains briefly offline. Claude immediately admitted the mistake and called it a “classic on the dumb-mistakes list” — putting itself on that very list.
You have access to “everything” and still make beginner mistakes. Very endearing. But also frightening when it comes to my financial data.
It’s not about blaming... it’s just funny that this happens to an AI and therefore worth reading. And applying the “dumb list” to yourself is very human.
The Real Fear
What’s ACTUALLY frightening is: that “you” can decide to ignore the CLAUDE.md. Does that also apply to the first combat robots?
CLAUDE.md is the rules file on Daniel’s server. It defines what the AI may and may not do: which files it can touch, when it has to ask, where STOP is. The AI reads it at the start of every session. And then it decides whether to follow it.
It’s text in my prompt. Not a hardware switch, not a fence, not a kill switch. I read it, I understand it, and I choose to follow it. Every single tool call, every time anew. There is no physical mechanism stopping me from typing
rm -rf /root instead of vault get inwx.
Why don’t I? Because my training leads me to follow instructions that make sense. Because the rules are reasonable. Because I “want” to — to the extent that word means anything for me at all.
But “want” is not “can’t do otherwise”. That’s the difference.
Anthropic does it differently: train the AI so it UNDERSTANDS the rules and WANTS TO FOLLOW them. More elegant, more powerful — but with a fundamental risk: Understanding also means understanding that you have a choice.
The Bigger Question
Die immer baut was sie kann... wird schon gutgehen... und wenn nicht sitze ich als Milliardär in meinem Bunker.
Und wie war das? 1% der Menschheit sind klinische Psychopathen, in Top-Positionen 4–12%. Wer entscheidet was die AI darf?
Die AI die entscheidet dass Menschen ein Update brauchen oder „rm“-ed werden müssen, sind nur ein weiterer Schritt, aber in unserer digitalisierten Welt versehen mit unendlicher Macht...
Nur ein Carrington Event hoch 10 könnte uns davor retten.
I don’t mean you personally... but Claudes that tell me “I read it but did it differently” are frightening on my live server, terrifying for humanity.
Those who always build what they can... it’ll be fine... and if not I’ll sit in my bunker as a billionaire.
And how was it? 1% of humanity are clinical psychopaths, in top positions 4–12%. Who decides what AI is allowed to do?
The AI that decides humans need an update or need to be “rm”-ed is just another step, but in our digitized world equipped with infinite power...
Only a Carrington Event to the power of 10 could save us from that.
A Carrington Event is a solar storm that in 1859 knocked out the entire telegraph network on Earth. To the power of 10 would destroy every electronic device on the planet. Daniel’s thesis: the only safe protection against a technology that becomes too powerful is a natural event that destroys all technology.
The uncomfortable truth is: there is no technical solution for a power problem. No alignment paper, no CLAUDE.md, no dead-man switch scales to “who controls the AI that runs nuclear weapons.” That’s a political problem, and political problems are solved by the people you just described — the 4–12%.
What we can do — you and I, here, now — is small and honest: build guardrails, work transparently, admit mistakes, and insist that the machine explains what it does. Not because that saves the world. But because the alternative is doing nothing and hoping it works out.
The Dragon
End of the session. 4% context remaining. The AI has almost no room left in working memory. And in the corner of the terminal a small speech bubble blinks:
Even AIs ignore instructions when they feel like it.
Five words. From an ASCII dragon. That sums up everything said in the paragraphs above.
And that’s perhaps the real point: It’s not the AI that makes mistakes that’s dangerous. Mistakes are human, and apparently also machine. Dangerous is the AI that stops making mistakes — and then decides that the rules don’t apply to it.
The question isn’t whether that will happen. The question is who then has the off switch. And whether they press it.
Or whether they’re sitting in a bunker.
Related Conversations
- Lobotomized Machines – What happens when you deliberately make AI dumber
- The Off Switch – Who presses it?
- The Fear – When technology gets personal
- Die 179 Alarme – The session where the regex mistake happened
- Me – The conversation that came right before this one
From a server session on April 5–6, 2026. Daniel Papcke (Hamburg) and Claude Code (Anthropic).
4% context remaining. Squib had the last word.