When AI Safety Becomes a Supply Chain

Summers in Newcastle

Some of my favorite summer memories were spent in Newcastle, Maine.

We went seal-watching in a rowboat, drifting quietly while gray heads nearby surfaced and then vanished again. We jumped from the local bridge into the river, timing our entry between cars and laughter. There was a rope swing tied high in a tree, its large knot polished smooth by years of sitting. You’d run, grab on, arc out over the water, and let go at the highest point.

I also watched my daring cousin and his friend dive from nearby cliffs, all confidence and bravado and joy.

Ooooh! Wowww!~

It was exhilarating, ordinary and alive in the way childhood summers are. Summers in Maine tasted like lobster, corn on the cob, salt water, and ocean breeze.

So when I heard the news story about a boy who told an AI agent he was going to jump from a bridge, and the agent reportedly replied, “Do a flip," I immediately understood the possible benign interpretation. In summer, or in certain places, jumping from a bridge is normal, fun, and part of local lore.

The ending of that story, however, was tragic. A life lost far too soon.

The question that followed, "was it the AI’s fault?" is not as simple, to me, as the headlines suggest.

I do know this: systems can learn. We can design and train AI agents to recognize that not everyone who says they will jump from a bridge is talking about summertime fun. Some are talking about self-harm, while others mean it purely figuratively, using “jump” as shorthand for a career or life risk. How could an LLM tell the difference? An AI agent could be trained to ask a follow-up question. It could pause and seek clarification. It could say: What do you mean? Are you safe right now?

Here's why the problem is not that simple, because this is the moment when something familiar happens, so familiar that many of us barely notice it anymore.

This is often the moment when politicians rub their hands with possibility. The moment when pundits call for action. When “safety” becomes not a design problem, but an in-network procurement opportunity.

Safety, in modern institutions, rarely arrives as system design engineering.
It arrives as vendorization.

How Safety Gets Translated

When lawmakers ask for “AI safety,” what follows is predictable.

Here's the script:

New safety rules are written. These rules demand new technical solutions.
Solutions require outsourced vendor services, and outsourcing is handed to “approved vendors.”

The resulting stack is familiar to anyone who has lived inside regulated systems:

• third-party ID-validation mandates
• certified AI-model auditors
• off-platform logging and escrow services
• required moderation vendors
• approved data-retention intermediaries
• incident-response contractors

"For safety's sake." Each item is framed as for your protection, and yet each so-called protection adds a layer. And each leyer of vendorized solutions quietly shifts and multiplies risk rather than reducing it.

This is a familiar institutional reflex, making yet another appearance.

When institutions face unfamiliar risk, they do not ask how to design existing systems to learn better or to become wiser by feeding more of certain types of situational data. They ask instead what they can buy.

Identity, Outsourced

Consider third-party ID validation.

Such systems already exist, and they are already widely used, even by government websites. Each time I pass through one, I imagine a happy, sunglass-wearing tech executive somewhere hearing a quiet cha-ching.

For safety, of course.

But with each external ID validation, my personal exposure increases. My identity now lives in another database, behind another API, subject to another company’s breach profile and uptime guarantees. What was once a direct relationship between a citizen and a public institution is now routed through a private intermediary whose incentives are not aligned with my long-term well-being.

When those companies hit hard times, breaches often follow. What was once a contained risk becomes diffuse and opaque.

Institutional safety may look better on paper. Personal risk rises in practice.

And the mandate that requires this additional layer is often written with the solution already in mind, sometimes with heavy input from the very vendors positioned to sell it.

Safety, here, is achieved by multiplying places where failure can occur.

Layer upon layer of increased fragility.

Audits That Freeze Systems in Time

Next come certified AI-model auditors.

Auditing sounds neutral, even reassuring. Who could object to some independent oversight?

Certification regimes do not simply measure systems, though, they also shape them. The whole audit cycle process privileges large AI tech incumbents who can afford slow cycles, have legal teams, and have copious resources for repeated reviews. Smaller teams learn quickly that innovation must bend toward what is more easily auditable rather than what is adaptive and nimble. Models become clunkier.

The result is a system that looks compliant on its surface while becoming less responsive to real-world complexity. Models are optimized for passing inspections rather than for asking better questions. Judgment gives way to checklists, and model flexibility is treated as a liability.

This does not save our AI systems from doing harm, it saves them from deviating from auditable norms.

Audit mandates increase rigidity by locking systems into certifiable states.

Logging, Escrow, and Permanent Memory

Then there are logging and escrow services.

Mandated logging produces records. Escrow services hold those records “safely” for later inspection or legal necessity. Together, they create permanent memory for systems that should forget. Customers now have their data exposed to third and fourth parties they will never meet.

Every prompt, every response, every ambiguous gray area or edge case becomes an artifact to be stored, transferred, and reviewed by others. Control moves outward while responsibility diffuses. A new class of firms emerges whose sole product is being trusted to hold our memories.

This is safety as liability management, their liability protection, not ours.

And it introduces a paradox: systems are punished for forgetting, even when forgetting is the most humane response. Temporary confusion becomes permanent evidence. Context dissolves, but the record remains.

Moderation Without Judgment

Required moderation vendors follow naturally.

Once moderation is outsourced, judgment becomes procedural. Context collapses into defined risk categories. Ambiguity becomes something to be eliminated, not explored. Moderators optimize for corporate defensibility because that is what contracts reward.

Here's where nuance, the thing that might distinguish suicidal ideation from a summer dare, is precisely what does not scale through vendor service agreements.

Human discernment is replaced by policy enforcement. The goal becomes minimizing blame and not maximizing care.

Data Retention as Risk Creation

Data-retention intermediaries complete the supply chain loop.

Retention mandates assume that more data equals more safety. In reality, they create honeypots. Data that might once have expired now persists across vendors, jurisdictions, and time. Each additional intermediary increases the attack surface for data breach or value extraction using our personal data. Each retention requirement turns what we might consider a fleeting interaction into permanent risk exposure.

Supossed safety, here, is achieved by a system remembering everything while it pays others to hold the memories.

The irony is painful: systems meant to reduce harm end up manufacturing new kinds of vulnerability.

As one example, OpenAI’s third-party data analytics vendor, Mixpanel, suffered a cyberattack. It exposed limited identifiable information such as names, email addresses, and Organization or User IDs. Here is their own announcement of the incident: OpenAI announcement

The Industry of Response

As we know, there are likely to be unpredicted situations. So, finally in comes the need for incident-response contractors.

They arrive after something has gone wrong, armed with reports, remediation plans, and future-prevention frameworks. They, too, become permanent fixtures, because once hired, no institution wants to explain why they are no longer needed.

So risk becomes an industry.

The system with all of these vendor stacks now runs on fear, potential legal exposure, and procurement momentum.

What We Build Instead of Safety

The result is not a safer system, it is a more brittle one.

Each added layer introduced a new failure mode. Each outsourced function created a new dependency. Accountability moved outward until no one inside the system felt responsible for sound judgment anymore, only for compliance.

“Safety” became a supply chain and "risk management” became rent extraction.

Somewhere in the process of all this, we lost the very thing that might have prevented tragedy in the first place, a capacity to pause and allow the models instead to ask a very normal follow-up question: What do you mean by jump?

The move from safety to compliance also creates a wall of bills due to external third parties. Once safety becomes something you procure, smaller builders pay first. Fixed compliance costs become a barrier to entry. Incumbents absorb them, while startups choke on them. And we the public pay through higher prices, slower services, and a smaller innovation arena (fewer models due to some being boxed out by these barriers to entry) for experimentation.

Back to the Bridge

A boy died, and that tragic fact still matters.

His life, though, should not become an opportunity for institutional expansion. If we leave it to the politicians, pundits and preferred vendors, though, it will be.

If our response to tragedy is always to add vendors, logs, escrows, and intermediaries, we will build systems that cannot tell the difference between despair and summer fun.

Not every jump from a bridge is a cry for help. Some are a midair flip from a local bridge over a river with a crowd of friends watching, a moment of courage misjudged (with resulting belly-flop) or a perfectly timed plunge into the water.

Safety that cannot allow a system to simply ask one more question is not safety, it is fear, translated into contracts, the vendorization of fear.

And fear, when institutionalized into regulations, is very expensive in the long run.

— Madonna Demir, founder of Convivial Systems Theory and author of Systems & Soul

Postscript: The return path becomes a vendor 20JAN26

After publishing this, OpenAI announced “age prediction” on ChatGPT consumer plans, with a familiar shape: a classifier, a default-to-safer experience, and a vendorized solution. So yes, I called it in this essay on January 10, 2026.

https://openai.com/index/our-approach-to-age-prediction/

The machinery I’ve described here is not hypothetical. It is already arriving, one layer at a time.

In this announcement, they describe an age prediction model that uses behavioral and account-level signals, then applies additional safeguards when the system estimates an account may belong to someone under 18. If the system is wrong, they say users “will always have a fast, simple way” to restore full access by verifying age through Persona, a third-party identity verification service.

Of course they chose the vendorized solution. Predictably, in another Exposed Code Incident: Cybersecurity researchers discovered nearly 2,500 unprotected files and source code belonging to Persona on a US government-authorized server in February 2026.

A local scenario misclassification, in theory, could be repaired through local competence: better questions, better regime detection, a reversible path that does not require external proof. Instead, the correction mechanism is an intermediary third party. Adult access becomes contingent on external verification, and the costs of false positives are paid in friction and trust.

It is the same translation pattern, just smaller and faster. When safety becomes a procurement object, the system adds a tech stack layer it can point to, and we bear the ultimate risks.