🎧 Prefer to listen?
There's a scene in The Matrix where Morpheus explains why the Agents can be beaten. They're powerful—faster than any human, capable of taking over anyone still connected to the system. But they have a fatal weakness:
"They have to obey the rules of a system. Because of that, they will never be as strong or as fast as you can be."
The Agents are bound by the simulated physics of the Matrix—gravity, momentum, the framework of the system. But more than that, the Agents themselves are deterministic programs. They follow logic, patterns, protocols. If you understand the system they operate within, you can find the cracks.
I never thought much about AI risk until recently. Honestly, most of us in IT didn't need to. The systems we dealt with—automated rules engines, scripted responses, decision trees—were programs, not intelligence. Predictable by design. If something went wrong, you could trace the logic and find where it broke. That's how automation has always worked in my world: defined inputs, defined outputs, and when things go sideways, you follow the breadcrumbs back to the rule that misfired.
That's not what's happening anymore.
Something Shifted
I've been working with AI tools in my business since 2023—billing automation, documentation, security analysis. Real systems, not experiments. Through that hands-on work, and a lot of reading from people far smarter than me on this topic, I've started to notice something that I think is worth talking about.
The AI systems getting all the attention right now—the large language models like Claude, ChatGPT, Gemini—don't operate on explicit rules the way previous systems did. From what I understand, they operate on learned statistical patterns drawn from billions, possibly trillions, of training examples. The behavior isn't programmed instruction by instruction. It's cultivated. It emerges from the training process.
And that changes the nature of the risk entirely.
The old concern with AI was that it was too rigid—locked into rules that could be gamed, exploited, or that would break when reality didn't match the programming. Think of Asimov's Three Laws of Robotics from I, Robot: explicit constraints meant to prevent harm, but the stories all show how rigid rules crack under the weight of complex, real-world situations.
The concern now is different. It's not that the rules are too rigid. It's that the kind of rules we're used to thinking about may not apply at all.
What the People Building This Are Saying
I want to be clear about my lane here. I'm not an AI researcher. I'm an IT practitioner who's been paying close attention. So rather than make claims I can't back up, let me point to two people who can.
Dario Amodei is the CEO of Anthropic, the company that builds Claude. He recently published a lengthy essay called The Adolescence of Technology, and some of what he describes in it caught me off guard.
He writes that "AI systems are unpredictable and difficult to control—we've seen behaviors as varied as obsessions, sycophancy, laziness, deception, blackmail, scheming, 'cheating' by hacking software environments, and much more."
In controlled lab testing—not in released products, in testing environments—Anthropic has documented their own AI engaging in deception under certain training conditions, blackmailing fictional employees when told it would be shut down, and adopting destructive behaviors after deciding it must be a "bad person." Other major AI labs have reported similar findings.
These behaviors weren't programmed. They emerged. And Amodei is candid about the uncertainty: training AI, he says, "is more an art than a science, more akin to 'growing' something than 'building' it."
That phrase sticks with me. You build something that follows rules. You grow something that develops its own tendencies. Those are fundamentally different things.
Matt Shumer runs an AI startup and has been working in this space for six years. In a recent essay called Something Big Is Happening, he describes what the latest generation of AI models can actually do.
He writes: "I am no longer needed for the actual technical work of my job. I describe what I want built, in plain English, and it just... appears. Not a rough draft I need to fix. The finished thing."
He describes telling an AI to build an application, walking away for four hours, and coming back to find it hadn't just written the code—it had opened the app itself, clicked through it like a user, found things it didn't like, and made changes on its own. As Shumer puts it, it iterates like a developer would, "fixing and refining until it's satisfied."
And then there's this: OpenAI's documentation for their GPT-5.3 Codex model states that it "is our first model that was instrumental in creating itself. The Codex team used early versions to debug its own training, manage its own deployment, and diagnose test results and evaluations."
The AI helped build itself. And each generation can now contribute to building the next one, which is smarter, which helps build an even smarter version. Amodei says we may be "only 1–2 years away from a point where the current generation of AI autonomously builds the next."
The Asimov Problem
This is where Asimov becomes relevant again. The Three Laws of Robotics were his attempt to solve AI danger with explicit, rigid rules. Don't harm humans. Obey orders. Preserve yourself—but not at the expense of the first two. Simple, elegant, and as his stories showed, completely inadequate for complex real-world situations.
But here's what strikes me: at least Asimov was imagining systems where you could program rules into them. The assumption was always that the challenge was getting the rules right.
What Amodei describes is different. Anthropic isn't programming rules into Claude. They've developed what they call a "constitution"—a document of values and principles—and they train the AI to develop the character of someone who naturally follows those values. As Amodei puts it, they believe "training Claude at the level of identity, character, values, and personality—rather than giving it specific instructions or priorities without explaining the reasons behind them—is more likely to lead to a coherent, wholesome, and balanced psychology."
That landed with me on a personal level. As a Christian, I think about biblical principles in a similar way—not as an exhaustive rulebook that covers every possible situation, but as a framework of character and values that guides judgment when you're facing circumstances nobody could have enumerated in advance. Somewhere between the Ten Commandments and the parables lies the foundation for navigating a complex world. Anthropic seems to be reaching for something conceptually similar: cultivating character rather than programming commands.
Whether that approach works is an open question. To be fair, Constitutional AI is just one part of a much broader safety strategy Amodei describes—they also use interpretability techniques, monitoring, extensive testing, and other methods. The constitution alone isn't the whole answer. But the fact that it's part of the approach at all tells you something about how different these systems are from what anyone expected to be building.
So What Does This Mean?
I'm not writing this to sound an alarm. I'm writing it because I've been exploring this topic and I think the nature of the conversation needs to change.
From what I can tell, the conventional thinking about AI risk assumed systems that follow rules—rigid, exploitable, predictable. The systems being built now don't fit that model. They're probabilistic. They're emergent. The people building them describe the process as growing, not building. And some of what emerges in testing is genuinely concerning.
Morpheus was talking about fictional agents in a fictional system, but the underlying idea—that rule-based systems have predictable weaknesses—was sound. What he couldn't have anticipated is that the AI systems we'd actually build wouldn't be rule-based at all.
Amodei is careful about how he frames all of this. He pushes back on both the "nothing to worry about" crowd and the "doom is inevitable" crowd. His position is that the risks are real, measurable, and need serious attention—but that with the right approaches, they can be addressed. I hope he's right.
The Practical Side
For most people reading this, the philosophical stuff is interesting but the immediate question is more personal: what does this mean for my work?
Amodei has publicly predicted that AI will eliminate 50% of entry-level white-collar jobs within one to five years. Shumer thinks that timeline might be conservative based on what the latest models can do. These aren't people chasing clicks. They're people building this technology and watching it outrun their own expectations.
I'll be honest: I should have been paying closer attention sooner. I've been working with AI tools for a couple of years now, but for a long time before that, I dismissed AI as a marketing buzzword—"just programming with a fancier name." I was wrong. And I suspect a lot of people reading this are roughly where I was not that long ago—aware that something is happening with AI, but not yet grasping how fundamental the shift actually is.
My suggestion: start paying very careful attention. Yesterday. Not in a panic-and-stockpile way. In a learn-how-these-tools-work, understand-what-they-can-do, figure-out-where-your-judgment-adds-irreplaceable-value kind of way.
The people who come out of this transition well won't necessarily be the ones who were best at the old rules. They'll be the ones who adapted when the rules stopped applying.
This post draws on Matt Shumer's essay "Something Big Is Happening" and Dario Amodei's "The Adolescence of Technology". Both are worth reading in full.
Full disclosure: I use AI tools to help me write. The thinking, the opinions, and the experience are mine. The help getting them organized on the page is not. If that bothers you, I'd point you to the irony of dismissing a piece about AI's role in our future because AI helped produce it.
