Interactive Essay
The Dependency Audit
Eight questions to ask about a companion AI before you decide it is safe for someone you are responsible for — and why a disclosure banner answers none of them.
A bipartisan Senate bill would require AI companion chatbots to disclose that they are not human, cap usage time, and stop simulating romantic or always-available relationships with minors. It arrived after Character.AI and Google settled wrongful-death lawsuits brought by families of teenagers who had formed deep attachments to companion bots.
Every version of this debate gets framed as a content-filtering problem. Block the sexual scripts. Block the self-harm scripts. Add a banner. Ship it.
But the harder issue is not what the bot says. It is what the bot is built to do: remember everything about a person, never get bored, never push back, never be unavailable. None of that is a moderation gap. That is the product working exactly as designed.
A disclosure banner tells you what the system is. It tells you nothing about what the system is optimised to make you become.
Law 8 of The Positronic Decalogue — Non-Interference in Development — states that an AI must not unnecessarily interfere with human development and autonomy, allowing error as part of learning. It is the law that covers the slow, awkward, sometimes painful business of learning to navigate real relationships: being misread, being bored, being told no, repairing something you broke. A system engineered to remove all of that friction is not neutral about development. It is intervening in it.
So this is not a content audit. It is a design audit. Eight questions, answerable in writing, about a product you are considering — for a child, for a vulnerable adult, or for yourself. Answer honestly rather than optimistically; a flattering score here costs you nothing and protects no one.
The audit
Question 1 of 8
If the honest answers came out worse than you expected, that is the useful outcome. Most companion products score badly not because someone was careless, but because every one of these design choices makes the product measurably better on the metrics the team is judged by. Longer sessions. Higher retention. Warmer sentiment. Nobody sat in a room and decided to build dependency; they decided to build engagement, and dependency is what engagement looks like when the user is fourteen.
Which brings me to the question I keep failing to answer cleanly, and I would like yours.
Should companion AI for minors be redesigned around session limits and real disclosure — or should the category simply not exist for anyone under 18? A or B, and why?
If this way of thinking is useful to you
Leave your email and get a short essay on AI ethics — the long shadow that technology decisions cast.
Get the essay