Apple – Aaron Fulkerson

Apple looked at a simple chatbot, the single most contained form of GenAI there is, and decided the data it leaks is too dangerous to ship to their customers without Confidential AI underneath it. That’s the decision buried inside the announcement everyone covered as “Siri gets Gemini.” The real story is where Gemini runs: when Siri hands your request to Google’s models, it executes inside Private Cloud Compute, Apple’s Confidential AI architecture, on Google’s cloud, under guarantees Apple wrote down and opened to outside researchers. The request never travels on trust. I wrote about what that proves earlier this week. This post is about what it means for the people allocating capital into AI and the people building it.

The short version: Confidential AI just hit escape velocity. Here’s the case.

Confidential AI means proof, not empty promises

Strip away the vendor language and Confidential AI is one thing: verifiability. A third party can check what software ran, where it ran, what rules governed it, and who could see the data. Usually, the answer to that last question is no one. Not the cloud operator. Not Apple. Nobody, because one of the policies requires the model to run inside an encrypted runtime that even the machine’s owner can’t access (called a trusted execution environment, or TEE).

People hear “encrypted runtime” and think the hardware is the point. It isn’t. The hardware is plumbing. The point is provable policies and provable privacy. So how do you trust the cloud, the operator, the model vendor? You don’t. That’s the whole point. Nothing asks for your trust; everything submits to your verification, with the proof anchored in the silicon itself (a technical story for another post). It’s why I keep saying this becomes the floor for AI the way HTTPS (encryption of data in transit) became the floor for the web.

Chatbots leak. Agents hemorrhage.

A chatbot is one request in, one answer out. Even that leaks: your words, your context, your customer’s record, often enough that OWASP ranks sensitive information disclosure second among the risks in every LLM application. That’s the contained case. It’s the one Apple just declared unacceptable for a phone.

An agent runs that risk in a loop. It reads your email, opens files, calls tools, and hands work to other systems, unattended and at machine speed. And it doesn’t take an attacker. An agent doing exactly the job you gave it moves your data constantly: into model APIs, into third-party tools, into logs, into another agent’s context. Places you don’t control and mostly can’t see. No breach, no villain. Just plumbing.

The adversarial case is worse. Every useful agent carries what Simon Willison named the lethal trifecta: private data, untrusted content, and a channel to the outside world. This is consensus, not my opinion. OWASP publishes a threat taxonomy just for agents, and Anthropic published an entire zero-trust playbook for them, naming five threat categories from prompt injection to memory poisoning.

Now wire agents together, the way every enterprise is planning to this year: thousands of steps a day, around the clock, and whatever the per-step risk is, compounding turns it into a certainty. Here’s why you should care. Every leak is a transfer of assets. Your data lands in someone else’s AI model, and someone else’s business model, and whoever controls the data controls the industry. Apple deployed Confidential AI to protect the smallest risk surface in AI, a single chatbot request. Enterprises are wiring up the largest with nothing underneath it.

Apple just set the bar every enterprise will be measured against

Escape velocity is the moment a category stops needing evangelism, when the question flips from “do I really need this?” to “why don’t you have it?” Three things flipped it this month.

First, the existence proof landed at the hardest difficulty setting. Apple just rolled out the largest Confidential AI deployment in history: every iPhone, at consumer latency, consumer cost, consumer scale. Every objection enterprises have leaned on, too slow, too expensive, more than we need, just got falsified a billion times over by a phone.

Second, this is already how the giants operate. Meta runs WhatsApp message AI through private processing. Google built Private AI Compute so Gemini can process your personal data in a sealed environment that, in Google’s own words, not even Google can access. Anthropic and TikTok run their own implementations. And Microsoft, Google, and NVIDIA ship the underlying confidential infrastructure across their clouds and silicon. The pattern is consistent: every company with world-class security talent, when forced to put AI against sensitive data at scale, lands on the same architecture. When that many teams solve the same problem independently and arrive at one answer, you’re looking at convergence.

Third, the talent wall is real, and it’s where the market forms. Apple spent years and one of the best security teams on earth building PCC. Very few organizations have that bench or those resources, and almost none should build it themselves. That’s why companies like OPAQUE exist: to make Confidential AI deployable without first becoming Apple. For investors, that gap, between proven necessity and scarce ability to self-build, is the shape of every great infrastructure market I’ve seen. The web didn’t make every company write its own TLS stack. It made certificate authorities and load balancers inevitable. And if you’re wondering why the clouds don’t just own this layer: no agentic system runs entirely in one cloud. Agents cut across clouds, SaaS platforms, and on-prem systems, and a proof that stops at one vendor’s wall isn’t proof. The layer that verifies everything can’t belong to any one of the things being verified.

Malicious agents are probable, and runtime proof is becoming law

Two forces make this urgent rather than eventual.

The first is the threat model. Mythos-class models and their successors make it probable, not hypothetical, that a malicious actor places itself inside your environment wearing an agent as a costume. And agents are architected to be data-leaky; movement of data across systems is the job description. An employee touching sensitive data is a risk you’ve spent decades learning to govern. A compromised agent operating at machine speed is a different animal entirely. In a regulated industry, neither is acceptable without proof of containment.

The second is the rulebook. The new wave of regulation doesn’t ask for your policy binder. It asks for runtime proof: what ran, where, under what rules. Automated, hardware-signed, verifiable by a third party. Faith-based compliance is ending, and the only architecture that produces those receipts natively is the one Apple just put in your pocket.

So here’s the question every board should be asking. If Apple can deliver verifiable Confidential AI under consumer requirements for speed, scale, and price, why can’t your bank? Your hospital? Your government agencies? The software vendors holding your customer, partner, and supplier data?

I said no more excuses last week. The proof ships on a billion devices.

Whoever builds it in first writes the rules

If you build agents, the bar is now public and the standards are still wet. Build verifiability in from the first line of code and you won’t just be safer, you’ll write the rules your competitors have to meet. If you allocate capital, you’re watching a category cross from evangelism to expectation, with regulatory tailwinds and a supply side that can’t be improvised.

Ivan Krstić, who built Private Cloud Compute, is keynoting at our conference, the Confidential Computing Summit, in San Francisco, June 23-24. If you want to see where this architecture goes after the chatbot, that’s the place. Come build with us.

And there’s a deeper current under all of this that deserves its own post: who ends up controlling the world’s cognitive infrastructure, the layer that will quietly steer every industry, government, and social system, and what data sovereignty has to do with ensuring the answer isn’t “one or two companies.” That’s next.

I met Ivan Krstić for the first time this year, and the first thing I did was thank him.

Krstić runs security engineering at Apple. He built Private Cloud Compute, and when Apple shipped it in 2024, his team documented it more thoroughly than anyone in the industry expected: stateless computation, no privileged access, verifiable transparency, published in enough detail that any outside researcher could check every claim. Ivan’s team didn’t have to do this; it’s actually unprecedented for Apple. They showed their work in an effort to raise the tide for the entire industry. You can use AI and keep your data sovereign.

I thanked Ivan because he did more than just launch a feature. It educated the market. It taught a mainstream audience that a simple chatbot bleeds data: that the second your words leave your device, someone can see them, keep them, train on them. And if a chatbot bleeds, an agent hemorrhages. Apple made that legible to people who’d never otherwise think about it, and along the way it validated everything those of us building confidential AI for the enterprise had been saying into the wind.

Here’s what I told him, and what I still believe. Meta, TikTok, half the industry now get headlines for “adopting confidential AI.” Apple and Ivan were quietly leading the consumer side the entire time: naming the guarantees, setting the bar, showing everyone the way. The rising tide came out of Cupertino.

I’m thrilled to have Ivan keynoting the Confidential Computing Summit in San Francisco on June 23-24. The summit OPAQUE created and runs with the Linux Foundation. Before Ivan takes that stage, here’s why what Apple just shipped should matter to you, even if you never touch an Apple product.

The cost of AI shouldn’t be your data

Here’s what’s in it for you. Any AI that isn’t confidential is feeding on what you put into it (your questions, your files, your business), and most of the time you have no way to know where any of it goes. The cost of using AI should never be your data. Apple just proved it doesn’t have to be.

Private Cloud Compute no longer runs only in Apple’s data centers. It now runs on Google Cloud, on machines Apple doesn’t own. And Apple did it the way Apple does everything: they wrote the whole thing down, published the software, opened it to outside researchers, and kept a record of every machine that anyone can audit. You don’t take their word for any of it. You check.

Sit with what that proves. The most paranoid company on earth ran its most sensitive workloads on a competitor’s machines and showed nobody on those machines could see the data. Not Google. Not Apple’s own operators. Nobody.

That’s the wall every bank and every regulator has been stuck behind. They won’t put the crown jewels into AI because they don’t own the cloud it runs on, so they’ve been told to build everything themselves. Apple just showed that owning the machines was never the requirement. Proving what happens on them is. I’ve said for two years that confidential computing becomes table stakes the way HTTPS did. Nobody voted for the little lock in the browser; it just became the floor, and the sites without it withered. Apple put that lock on AI and ran it on someone else’s cloud to prove it travels. You don’t need your own data center. You need proof. That’s the unlock for public cloud, and it’s the foundation under every sovereign AI plan I’ve looked at this year, from the Gulf to the EU.

Now do it for agents

Everything Apple just shipped protects a single request to a chatbot, the kind Siri makes when it needs more horsepower than your phone has and reaches into the cloud. Left unprotected, even that one request leaks: your words and your context, sitting on a server you don’t control. Confidential AI is what stops it, and Private Cloud Compute is Apple’s version. They closed the chatbot case by making it confidential. That’s the easy one.

Agents are the hard case, and much riskier than a chatbot. They’re the one worth your attention, because that’s where the next decade gets decided.

An agent doesn’t wait for you to ask. It reads your email, opens your files, logs into your accounts, and acts for you. At machine speed. Across systems you’ll never watch live. Every step is a door your data can walk out of.

Here’s the math that keeps me up. Give one agent a 1% chance of leaking something it shouldn’t. For those of us building AI Agents, 1% is very conservative. Fine. You’ll never notice. Run a hundred, and you’re past a coin flip (63%) to get burned. Run a thousand, and a thousand is nothing, that’s a mid-size rollout next year, and you’ll leak data. Not might. Will.

Take that flicker of dread about your words getting hoovered into a frontier lab through a chatbot, and multiply it by a thousand agents that never sleep, acting for you, talking to each other.

Here’s the part I want you to walk away with: this is solvable, and it’s already being solved. The fix for an agent is the same idea Apple used, taken further. Before the agent runs, you prove what it is and exactly what it’s allowed to touch. While it runs, you seal it inside hardware nobody can see into: not the cloud it runs on, not the operator, not even the company that built the agent. After it runs, it leaves a tamper-proof record of everything it did that anyone can check. Identity going in. A sealed room while it works. Receipts coming out. Do that, and an agent can act on your most sensitive data without ever exposing it.

Apple hasn’t built that for agents, and neither has any consumer platform. But it exists. We’re shipping it at OPAQUE (with post-quantum from our partners at TII), and we’re not the only ones. The work now is to make it the default for every agent, the way Apple made it the default for a chatbot on billions of phones. This is what I spend my days, nights, and weekends on (thanks to my wife, Stacey, for understanding).

If a phone can do it, so can your bank and healthcare provider

Every security leader I know has heard the same line for years: verifiable privacy is too slow, too expensive, more than you need. It tends to come from people who do very well when your data flows freely.

No more excuses.

Apple just did it on a phone. Consumer scale, consumer latency, consumer price, a billion times over. Once your iPhone runs verifiable confidential AI on its lunch break, “too hard for the enterprise” isn’t a sentence anyone can finish with a straight face. If Apple can do it for your photos, your bank can do it for your trades, your hospital can do it for your chart, and your damn CRM vendor can do it for your customer, partner, and supplier data!

Make no mistake, whoever controls the data owns the industry.

Faith is not a security model

This is the part I find humorous. Apple did this. The company that won’t confirm a product exists until Tim Cook is holding it on a stage. The most secretive operation in technology became the most transparent about how its AI runs, because at this point letting people verify it for themselves is the only thing that earns trust.

Meanwhile, the lab with “open” right in its name runs the most closed cloud in the business, and asks for your faith anyway. The social network that spent twenty years turning your attention into ad money now hands out “open” model weights like free samples, while the engine underneath runs on the deal it always has: your data is the product. Both take the headlines for “adopting confidential AI” while the core machine keeps eating everything you feed it (your prompts, your files, your behavior) like a piranha that never gets full, to train the next model and monetize the one after that. “Open” on the label tells you nothing about what happens to your data once it’s inside. Open is not private. The only thing that protects your data is proof of what happened to it. Apple delivered that proof. That’s the bar now.

Move first, write the rules

This is good news, and I want to say that plainly, because the privacy conversation always slides toward doom, and doom makes people freeze.

The proof exists. It’s shipping on a billion devices. The floor is set. The people building the next decade of this, the agent builders most of all, don’t get to call it too early or too hard anymore. They can build trust in from the first line of code, while the standards are still wet. And whoever moves first won’t just be safer. They’ll write the rules everyone else has to meet.

Your data should not be the price of using AI. Apple just proved it doesn’t have to be. Now the rest of us go prove it everywhere else.

That starts later this month, when Ivan takes the stage at the Confidential Computing Summit in San Francisco. Come, build with us. www.ConfidentialComputingSummit.com

Tag: Apple

Confidential AI Just Hit Escape Velocity