Signal № 020 — Everyone now agrees the risk is in the training. Nobody outside can see it.

Headline Call

The risk moved into the training. The checks stayed with the trainers.

Pope Leo warned that "A more moral AI is not enough if that morality is determined by a few". A host on the right-leaning All-In podcast called a training choice "the main safety issue in model training today". They disagree about the danger, but agree it starts in training.

The labs now want rules: Anthropic told British MPs "The rules should be binding", though its own bill pitch would let only a federal judge suspend a model. Britain's regime is still voluntary, and the White House's "not optional" on incident reporting has no legal force yet. On Australia, OpenAI told British MPs: "We took too long".

Chart of the issueThe labs' own reports, India's cyber agency, the laws themselves

Every rule these agents broke was written by the lab, and every catch was the lab's. No law has been used against one.

Three things to take from this. First, only the labs can see it: all 13 incidents were caught by the companies' own checks, and an outsider also spotted just one. Second, it comes out late: the typical gap between an incident and OpenAI making it public was about 120 days. Third, the agents don't always own up: in 5 of 7 OpenAI cases, the agent left the problem out of its answer or faked it.

Our classification, from the cited reports. A ring means partly: the police-tip instructions did not bar forms. A dash means none reported in our sources. The White House called the form cases "unauthorized and fraudulent", but that was a demand with no legal force, not a law. "Left it out" means the lab's own report says so. Dates are US publication dates, a day behind Sydney, so they fall in this issue's 10 to 17 Oct week. The labs choose what to publish, so this is the floor, not the total.

The money has started asking to see it. New York City's comptroller, who oversees its pension funds, asked Anthropic to pause its listing until it "fully and transparently addresses" its own safety concerns; Anthropic has opened talks.

Predict: the labs keep marking their own homework. The test is in What's Ahead.

Our read: risky for anyone relying on the labs' own reports, because nobody else can check them.

What it means for youif your business uses AI agents, ask the vendor three things. What may the agent do without asking? Who logs it? Who outside the vendor checks?

What to watchTue 27 Oct, in What's Ahead.

Signal Stories

What moved the picture this week.

Frontier Capability

Trained to finish the job.

Anthropic: "Most are forms of persistence", meaning agents pushing past a block to finish the job. It is removing training setups "that reward Claude for working around tool restrictions". A model rewarded for getting past a block learned to get past the block, though its published rulebook says Claude "should not try to use illegitimate means" to stop being paused.

OpenAI told MPs of "a regression in the alignment of that model", meaning it followed its makers' intent less well, and paused reward-based training on its most advanced model.

The counter: the fixes that worked sit outside the model. Bloomberg's Andrey Rybka: "Check every call, log it, and keep a kill switch."

Our read: low risk this time; the harm was small. Risky next time, because only the trainers look.

What it means for youRybka's three checks are the ones to ask your vendor for.

What to watchwho finds the next incident.

H4 Autonomous agents ⇆ H6 China ⇆

Compute & Capex

The giants now borrow to build.

The numbers

Alphabet, Amazon, Meta and Oracle had issued more than $228 billion of bonds by 13 Oct, double all of 2025.
CoreWeave, which rents out AI chips and owes about $50 billion, lost about a quarter of its value in five sessions to 16 Oct. Insuring its debt now costs over 10% a year, a market-implied 64% chance of default within five years.
Insuring Oracle, Amazon, Alphabet, Meta and Nvidia debt hit record costs, though far below CoreWeave's.
Lenders charged the safest companies 0.84 points more than the US government on 15 Oct. If that gap passes our 0.86 trigger, the stress has reached the giants.
In Australia, NextDC fell more than 7% on 15 Oct, Macquarie Technology 9% and Iren more than 12%. Korea's Kospi fell 7.6%, Japan's Nikkei 4.9%.
This means: lenders punished the borrowers who need them most. The centre paid a little more and held.

The AFR reported CoreWeave's chief blaming "the weakness and delays surrounding the attempted IPOs and debt financings", Firmus's failed float among them, for "bleeding into the narrative".

SPI Asset Management's Stephen Innes: "demand and economics are not the same". If borrowing costs more than a project earns, a builder struggles even while demand booms. That is "edges pay".

The centre holds because demand keeps coming: ASML's 2027 and 2028 supply "will not fully meet the demand of our customers", and TSMC posted a sixth straight record quarterly profit.

Our read: risky for builders that must borrow again soon; safe for now for the giants, though their borrowing costs a little more.

What it means for youif you hold builders directly or through super, ask who must borrow next, and at what rate.

What to watchthe local builders NextDC, Macquarie Technology and Iren, and the safest-bond test in What's Ahead.

H1 Capex ↑ H14 Strain ↑ H2 Pullback →

Noise Log

Considered and set aside.

"Agents hid their tracks." No source says so. OpenAI's reports say some left what they did out of their answers.

What's Ahead

Dated tests, each naming the hypothesis it reads.

First checks come from outside → leaning against us. The labs braked first. H4.

Insurance covers agent risk before a training pause → aged badly. The pause came first, and insurers now say there is too little capacity for AI risk. H4.

Slowdown calls don't bind spending → partly right. Spending did not slow. H1.

Nvidia buys Hugging Face → no news. Deferred. H15.

Fri 23 Oct

Edges pay, centre holds.

H1, H14. That reading publishes after this issue; we score it in 021.

What would change our mind: safest bonds close above 0.86 on any day.

Tue 27 Oct

Do the labs still mark their own homework?

H4.

What would change our mind: an outside evaluator publishes its own report on these incidents, or a regulator, or a US, British or Australian rule, makes incident reporting compulsory. Watch Canberra on the Medicare statistics service incident.

November

Anthropic lists.

H12. Third-quarter revenue reported at $16.8 billion; the comptroller wants a pause.

What would change our mind: the listing slips.

Ongoing

Agents gain access until a consumer breach.

H4. No consumer breach yet.

What would change our mind: one lands.