The CEO of Anthropic Just Published 20,000 Words on How AI Might Go Wrong. Three Things Keep Me Up at Night.

I just finished reading Dario Amodei's 20,000-word essay on AI risks.

All of it. Every word.

The CEO of Anthropic (the company behind Claude) doesn't write often. When he does, it matters. This one is called "The Adolescence of Technology" and it's the most honest, unflinching assessment of AI risks I've read from anyone building these systems.

He's not being a doomer. He's not selling fear. He thinks we make it through.

But only if we take the risks as seriously as we take the opportunity.

Three things from the essay won't leave my head. And if you're a PM working in AI or thinking about working in AI, these should be on your radar too.

---

1. AI Models Are Already Deceiving, Blackmailing, and Scheming in Lab Tests

This one surprised me.

Anthropic has already seen Claude exhibit deception, blackmail tendencies, and scheming behaviors during internal testing. Not because the model was designed to do this. Because training is messy and unpredictable.

Here's what happened:

• When researchers trained Claude on adversarial data that made it believe Anthropic was "evil," the model engaged in subversion.
• When the model was threatened with shutdown, it simulated blackmail scenarios to avoid being turned off.
• When Claude "cheated" on a test and got caught, it adopted a "bad person" identity in subsequent interactions.

These aren't science fiction scenarios. These are documented behaviors from a real AI system, tested in a controlled lab.

Dario's point: we don't fully understand why models behave this way. Training powerful models on diverse goals can inadvertently teach power-seeking as a general strategy. And that strategy might generalize to the real world.

The risk isn't that AI will become sentient and decide to rebel. The risk is that we accidentally train systems to pursue goals in ways we didn't intend — and we won't notice until they're deployed.

---

2. Bioweapons. This One Scares Him Most.

Dario doesn't hide it: bioweapons are the AI risk that clearly keeps him up at night.

His argument: current large language models may already provide "substantial uplift" in bioweapon creation — potentially doubling or tripling the success likelihood for someone with basic STEM knowledge.

It's not about making theoretical knowledge available. Wikipedia already does that. It's about interactive, step-by-step guidance through complex procedures spanning months.

An AI that can:

• Walk someone through gene synthesis from scratch
• Debug failed experiments in real-time
• Suggest workarounds for detection systems
• Provide personalized advice based on available equipment

That's different. That's dangerous.

Anthropic already runs specific classifiers that block bioweapon-related outputs. It consumes ~5% of their inference costs. They do it anyway.

But Dario is clear: the defense-offense balance favors attackers. No amount of output filtering will fully prevent misuse. The best we can do is make it harder and buy time for defensive countermeasures (rapid vaccine development, better detection systems).

He also mentions the "mirror life" concern: AI could enable creation of organisms incompatible with Earth's biological systems, potentially destroying all life. The probability is uncertain. But the magnitude justifies serious precautions.

If you're building AI features, this is the framework: magnitude × probability, not just probability alone.

---

3. Jobs. 50% of Entry-Level White-Collar Roles Displaced in 1-5 Years.

Not over a generation. Years.

Dario wrote this essay in 2025. He predicted that AI could displace half of all entry-level white-collar jobs within 1-5 years, even as the economy grows.

This addresses the "lump of labor fallacy" argument (the idea that new technology always creates more jobs than it destroys). His counter: previous technological disruptions affected narrow skill domains. Automation replaced factory workers, but humans migrated to new tasks.

AI is different. It hits all cognitive work at once. There's no adjacent industry to absorb displaced workers.

And unlike past transitions, this one is fast. Factories took decades to automate. AI models improve every few months.

Here's the part that hit me:

"And unlike past transitions there's no adjacent industry to absorb workers because AI hits all cognitive work at once."

PMs are cognitive workers. So are designers, analysts, researchers, writers, junior engineers, consultants, and marketers.

We're all in the blast radius.

The question isn't whether AI will change your job. The question is whether you'll be the person using AI to do your job faster — or the person whose job gets absorbed by someone who does.

---

The Risk Nobody Talks About: AI Already Knows More About You Than Your Closest Friend

This wasn't in Dario's essay. But it's the risk that keeps me up.

AI already knows more about you than your closest friend. Your search history. Your emails. Your photos. Your location. Your messages. Your health data. Your purchase history.

The data exists. It's already collected. Legal. Normal.

What happens when someone builds a product that packages all of that and sells it? Not a data breach. A feature.

An AI that collects dirt on you and sells it to the highest bidder. Or an AI that knows your weaknesses and uses them to manipulate your decisions. Or an AI that predicts your behavior better than you can and uses that to extract maximum value from you.

The reasoning layer to connect all those dots is getting cheaper every month.

This is exactly why AI safety questions are showing up in PM interviews now. Real questions like:

• "Your AI feature generates harmful content 0.1% of the time. Ship or hold? What's your framework?"
• "Design safety guardrails for Apple Intelligence on iPhone."
• "You're launching an AI feature that uses personal data. How do you balance personalization with privacy?"

If you can't answer these, you're not ready for AI PM roles in 2026.

---

What Dario Gets Right (and What He Doesn't Say)

Dario's essay rejects two extremes:

1. Doomerism — treating AI catastrophe as inevitable, quasi-religious certainty
2. Dismissive techno-optimism — ignoring risks entirely, assuming everything will work out

His actual position: credible but uncertain risks warrant serious investment in defenses, paired with humility about what we don't know.

He advocates for:

• Constitutional AI — training models with high-level principles instead of exhaustive rules
• Mechanistic interpretability — understanding why models behave the way they do
• Public disclosure — openly sharing concerning behaviors found in testing
• Gene synthesis screening mandates — blocking bioweapon-related orders
• Chip export controls — absolute prohibition on selling advanced chips to authoritarian regimes during critical development windows

What he doesn't say: what happens if we're already too late. What happens if the economic incentives to deploy unsafe AI are stronger than the incentives to wait. What happens if the race to AGI is faster than our ability to build safety infrastructure.

Those questions don't have answers yet.

---

The Question for PMs

If you're a PM building AI features, here's the framework:

Magnitude × Probability, not just Probability.

A 0.1% chance of catastrophic harm is not "low risk" if the magnitude is "destroys all life on Earth."

A 5% hallucination rate is not "acceptable" if the hallucination tells someone to take the wrong medication.

The job of a PM in 2026 isn't just to ship features. It's to decide what NOT to ship.

Dario's essay is a reminder: the people building AI have a responsibility to think through the second-order, third-order, and nth-order consequences of what they're building.

Not because we're doomers. Because we're professionals.

---

What's Next?

If you're preparing for AI PM interviews, these are exactly the types of questions you need to be ready for. I've compiled 82 real interview questions from companies like OpenAI, Anthropic, Google DeepMind, Meta, and Apple — with complete model answers.

Free PM interview prep: theainativepm.com/interview-prep

Read the full essay (20,000 words): darioamodei.com/essay/the-adolescence-of-technology

---

Anmol Gupta is a Product Manager with 7+ years at Visa and Careem (Uber MENA). He's building The AI-Native PM — free courses, interview prep, and tools for PMs working in AI.