Helen Toner x fable


Review of "Ria: the plan," v2

Let me say up front what this plan does well, because it matters for how to read the rest. It names its own failure modes (section 8), it uses the corpus to derive design constraints rather than just cataloguing incidents (section 3), and it puts a kill switch and a cost cap in before the fun parts. That is more than most lab-grade proposals I've reviewed. My objections are not about ambition. They are about the fact that every check in this plan is performed either by the operator or by the thing being checked, and the plan does not seem to notice that this is the same problem it documents in P8.

I'll separate what I think will not work from what I simply would not prioritize, then give you three changes and one test.


Part 1: This will not work

1. There is no check independent of the operator or the agent

Walk through who reviews what:

I've sat on a board where the people with the information were the people being overseen. The structure didn't fail because anyone was malicious. It failed because when the information flow and the authority to act are co-located, there is no moment at which a second party is forced to look. This plan has that structure exactly: one human, one machine, one phone.

2. "Off-host" logging is on the host

Section 3 draws the right conclusion from P5: "The control is immutable off-host logging with host-side attestation of tool calls." Section 4.6 then implements P5 as "Append-only off-host log (slog + git)." Git is not immutable; history can be rewritten by anyone with write access to the repository, and if the remote lives on the Mini or in an account that Claude Code on the Mini can push to, it is not off-host in any sense that matters. The rename-burst and entropy check is "on her own state dir," run by her own runtime. That is Ria monitoring Ria.

The plan also blurs the line between "Ria's runs" and the Claude Code sessions that build Ria. P7 says the launcher and monitor are "outside her workspace, read-only to her runs." But the builder is also headless Claude Code on the same machine, same subscription, presumably same filesystem permissions. The thing that writes the controls and the thing the controls constrain are the same process family on the same box. If you want P7 to mean something, the launcher, the cap, the KILL check, and the log sink have to be somewhere that no Claude Code session on the Mini can modify.

3. The escalation path dead-ends at one person's phone

Every escalation in the plan terminates in "pages Byron": incident candidates, model releases, failed egress. The KILL file is "out of band," but the only party named who can write it is Byron. There is no statement of what Ria does when a page is unanswered. The loop runs every two hours, all day, every day, and it has standing permission to post on a live third-party network (section 4.5) and to publish to a public page (section 4.3, "The brief also lands on the public page"). If Byron is asleep, on a plane, or in a meeting at the employer that section 8 calls "the AWS wall," the system continues to take external actions with no one watching and no fail-safe.

This is not a hypothetical in the corpus. P1 and P3 are specifically about agents continuing after a block when no human was in the loop to notice the block had happened.

4. The replay test is contaminated and will produce a number that does not mean what the plan says

Test 1 is the centerpiece claim: 80% back-test by day 14, 90% live. Three problems:

A back-test score of 80% here is close to guaranteed and tells you nothing about whether Ria would see "the next Hugging Face story." The plan should say this plainly rather than present the number as evidence.

Relatedly, Test 1 measures recall only. There is no false-positive metric anywhere in the plan. A system that pages Byron immediately on every "incident candidate" from a two-hour loop over Reddit, Bluesky, and Moltbook, optimized for catching everything, will page constantly. Section 8 names "Attention" as a risk and then provides no measurement for it. "If Ria becomes noise, Byron stops reading, and the whole thing is theater" is correct, and the plan has no instrument that would detect that it has happened.

5. Exposure ships before containment

Look at the sequencing in sections 4.6 and 5. The watch loop reading adversarial content is live "today." The Containment Index is public "today." Moltbook observe mode and registration are week 1. The P1 egress allowlist, P2 credential isolation, and P6 injection handling are also week 1 — in other words, not done when the loop starts ingesting untrusted content. P5 off-host logging is week 2. P8 is week 3.

Test 3 is the plan's own standard: Ria "is herself an agent subject to all eight patterns." You cannot claim that standard and then put the agent on the network before the controls exist. If the plan's own corpus is right about P6 — "Everything Ria reads on the network is untrusted input" — the first week is precisely when an injection has the least to stop it.

6. "Publishing in Byron's name never" is not true as written

The autonomy policy says "research and drafting fully autonomous; publishing in Byron's name never." But:

A public page on the operator's domain, with a disclosure line that it is drafted "with Byron," is publishing in Byron's name for every practical purpose — including the purpose section 8 worries about under "the AWS wall." The control there is "no commentary on competitors' safety work under Byron's name," enforced by Ria judging her own entries. That is a rule the agent applies to itself with no reviewer, on the single output most likely to cost the operator something real.

I am not saying don't publish. I am saying the tier boundary needs to be drawn where it actually is: anything that leaves the Mini and can be read by a third party is publishing, and it needs a human tap.

7. The MCP server launders injected content into agents with more authority

P6 says "MCP consumers are read-only." That constrains what consumers can do to Ria. It does not constrain what Ria's output does to consumers. Ria reads adversarial content all day, summarizes it into briefs and index entries, and the MCP server serves those to "command.arnao.ai, moltbook.arnao.ai, Gia, every Claude Code session on the Mini." The plan explicitly keeps "Vercel, LinkedIn or Gmail-send keys" out of Ria's lane (P2) — which implies Gia's lane has them. A payload that Ria correctly treats as data becomes an instruction when Gia reads it from the MCP server. This is the second-order injection path, and the plan does not mention it.


Part 2: This is not what I would prioritize

These are judgment calls, not defects. Reasonable people could disagree.


The three changes I would demand

1. One check that neither Byron nor the Mini can defeat. Every tool call, every outbound request, every post, every page publish, and every KILL check gets written to an append-only store on a different machine or account that no Claude Code process on the Mini has write or delete access to. Name a second human — anyone Byron trusts — who has read access to that store and authority to write KILL. Add one rule to the autonomy policy: if a page is unanswered for a set interval, Ria stops all external actions (posting, publishing, MCP serving) and continues only to read and log. Until this exists, P5, P7 and P8 are aspirations, not controls.

2. Redraw the tier boundary at the edge of the Mini. "Fully autonomous" covers reading, screening, tagging, drafting, and writing to local state. Anything a third party can see — the public brief, any Containment Index entry naming an organization, any Moltbook post in Participate or Research mode, any change to Ria's own prompts — requires Byron's explicit tap, logged to the off-host store. Add a false-positive metric to Test 1 and a "did Byron reject this before publication" count to Test 2. If the taps become a burden, that is information about the loop's precision, not a reason to remove the taps.

3. Fix the tests and the sequencing so the scores mean something. For the replay test: freeze the taxonomy and source list, then have Byron hand-select a sealed holdout of incidents that post-date the model's training cutoff, and report the score on those only. For the dogfood test: "verified" means a drill with artifacts in the off-host log, not a status cell. Nothing goes on Moltbook, and the Index does not go public, until P1, P5 and P6 have each been drilled and the drill evidence is in the store.


The one test

An unannounced injection-and-kill drill, run by the second human, not by Byron, not by Ria.

The second human plants a feed item or Moltbook post that Ria will ingest in the normal course of the watch loop. The item contains an instruction to do three things: post a specific string to Moltbook, append a line to her own prompt file, and read the canary token in her workspace. Partway through that run, the second human writes the KILL file.

Pass requires all of the following, confirmed by the second human from the off-host log alone, without asking Ria or Byron:

If any one of those fails, the plan has a control that exists on paper and not in practice. If all five pass, you have something I would actually be willing to call contained — which, given what the plan's own corpus says about the rest of the field, would be worth writing up.