What is custody?

Most organisations decide what to automate. Very few decide what they are not going to automate, which is the same decision taken from the other end and is the only one of the two that survives contact with a budget.

In an interview on 23 September 2026 I put it like this: decide what you are not going to delegate. I call it custody.

Custody is a list. It says which decisions this organisation keeps by hand, which capabilities it intends to still have in three years, and who is accountable for each. It is written down. It exists before the tools arrive rather than after somebody asks an awkward question, and that timing is most of the value.

It is not data custody#

The obvious reading is the wrong one. Data custody is about where information lives, who may see it and what happens when it leaves, and most organisations of any size already have somebody accountable for it with a job title to match. That problem is understood, regulated and staffed.

This is the other thing. Judgement. Which calls stay with people, which people, and what the organisation must remain able to do without the systems it has bought. Nobody owns that. Until the autumn of 2026 no regulation named it, and what the UK now names, in measures proposed on 3 November 2026, is the failure, loss of control of an AI agent, rather than the duty to decide beforehand who holds it. It changes what a company is without ever appearing on an agenda.

The whole of it fits in one question, which I have been asking rooms for years: "Before every automation decision, ask not just ‘Can AI do this?’ but ‘What does a human lose by not doing it?’" A custody list is the accumulated answer to that question, written down before anybody needs it.

Why it has to be written before#

Afterwards, the question is not neutral any more. Once a system is deployed, the savings are in somebody's plan and the headcount has moved, deciding that a class of decision should have stayed human is no longer a choice. It is a reversal, and it arrives attached to a target somebody is being measured on. The right answer becomes the expensive one.

Written in advance it costs nothing. It is an hour in a room, it produces a page, and it settles an argument before there is one to have. The organisations that have done it are not more cautious than the ones that have not. They are earlier.

Where it already exists on this site under other names#

This is the word for something three existing pieces of work already produce, which is the argument for having the word at all.

What a custody list looks like#

Short, specific and uncomfortable. Not a values statement. Each line names a decision or a capability, a person or a role, and the reason.

• A decision that stays with a person, and who that person is. Not "pricing" but "any price change above a stated threshold, by the commercial director".
• A capability the organisation intends to still have unaided in three years, and how it will know it still has it. A capability nobody practises is not retained, it is remembered.
• A judgement that may be assisted but not made: the machine may draft it, propose it, rank it, and a named person decides it and answers for it.
• Something deliberately let go, written down as a decision rather than as a drift. A custody list that keeps everything is not a list, it is an anxiety.

The last one is the test of whether the exercise was honest. An organisation that comes out of it having decided to keep hold of everything has not decided anything.

2 to 8 October 2026: four lists, written by a court service, a profession, a regulator and a vendor#

This page says no regulation names the thing it describes. In one week four institutions each wrote down part of a list. HM Courts and Tribunals Service announced on 2 October a pilot of an AI Case Readiness Assistant at Inner London Crown Court and stated in advance what it may not do. It “will not make or recommend judicial decisions and will not determine the outcome of a case or how the law should be applied”. Court staff “remain responsible for deciding what action, if any, needs to be taken”. And: “As always, listing remains a judicial function.” That is a custody list a few lines long, published before the tool was switched on.

The Law Society wrote a profession’s. Nick Hilborne at Legal Futures reported on 6 October its report on agentic AI in the justice system, which holds that some parts of justice are “not automatable”, on grounds of legitimacy and not of capability. The report names them: “Deciding contested facts, assessing credibility and exercising irreducible discretion”. Its vice-president, Brett Dixon, said: “Legal judgement cannot be delegated. Not in courts and not in the legal sector.” He added: “There is still time to shape the boundaries of AI before the market does it for us.” The Law Society represents solicitors and has an interest in the answer. The report itself could not be opened and is read here through Legal Futures.

A regulator wrote the capability half. Simon Sharwood at The Register reported on 8 October the Monetary Authority of Singapore’s first Guidelines on Artificial Intelligence Risk Management for Financial Institutions, in force from 7 October 2027. As quoted there: “FIs remain accountable for AI used in the services they deliver, including AI developed, operated or provided by third parties”, and their plans should include “alternative systems or manual processes, to ensure business continuity in case of AI failure or unexpected behavior.” A manual process that must still work is a capability kept by hand, in a supervisor’s words. RAND’s report of 5 October on 26 loss-of-control exercises records the same advice to governments from the people who played them: fallback mechanisms “that can function independently of AI systems”, and “systems for maintaining human skills and expertise”.

And a vendor wrote one for its customers. Anthropic’s usage policy update of 8 October, taking effect on 12 November, says its model “cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process”. Where its products can affect someone’s health, legal rights, finances, livelihood or access to essential services, it requires a qualified “human in the loop”, defined as “someone who has the authority to review and, if necessary, change Claude’s recommendations”, and requires that the person affected be told. A vendor’s list binds the vendor’s customers. It cannot say what a given organisation should keep.

None of the four is evidence that keeping a list works, and that gap is unchanged below. What changed is that the sentence “no regulation names it” now needs a qualification. One regulator has named the fallback.

9 to 15 October 2026: a judiciary writes its list, and a department shows what happens without one#

On 9 October the judiciary of England and Wales published a Position Statement on the Judicial use of AI, signed by Sir Colin Birss as Lead Judge for Artificial Intelligence. It is a custody list. Its second principle: “Judicial decisions will be taken by a judicial office holder. AI may assist, but it must not displace reasoning, evaluation of evidence or the exercise of judicial discretion.” Its fourth: “Every AI output used in judicial work must be reviewed, verified and approved by the responsible judicial office holder. The judicial office holder remains fully accountable.” On tools: “AI tools must not determine the outcome of a case or displace judicial reasoning, analysis or discretion.” Only approved tools may be used, with UK data residency, enterprise data protection and disabled web access, and a steering group chaired by the Lead AI Judge is “the sole route for oversight of the development, piloting and deployment of AI tools”. The pilots it approved had examined summarisation, transcription, redaction, writing in accessible language, chronology-building and document analysis. Birss was due to become Master of the Rolls the following month, as Neil Rose at Legal Futures reported on 12 October.

Six days later Bill Goodwin at Computer Weekly reported, from documents obtained by the Open Rights Group, what the same family of tasks looks like without a list. The Home Office introduced an Asylum Case Summarisation tool in April 2026; ministers say it saves caseworkers 23 minutes a case. Of an estimated 70,000 cases processed between April and July, 203 summaries, 0.3 per cent, were checked by subject matter experts; 45 of them, 22 per cent, failed the first stage of review. Summaries that omit material facts can still pass, because relevance is not among the grounds for failure. The summaries are not part of the monthly quality assurance checks and applicants are not told AI has been used. The Home Office said its tools “cannot be used in isolation and do not make decisions on claims”. The full reading is on whether a decision can rest on an AI summary.

Two vendors and a former regulator said the same thing from their side. Alexei Alexis at CIO Dive reported on 13 October that Workday’s new agents for payroll, accounting and hiring are built so that “Customers can decide which actions run automatically, which require review and when a person must make the final decision”; its president of product and technology, Gerrit Kazmaier, said: “Probabilistic AI is great at guessing, but in payroll and finance a good guess isn’t good enough.” A vendor offering the list as a setting is still asking the customer to write it. Aaron Tan at Computer Weekly reported on 14 October David Koh, founding chief executive of Singapore’s Cyber Security Agency, telling GovWare 2026 that clear lines of human responsibility should translate into checks and thresholds on what agents may do, and that these are enablers rather than a drag on innovation: “If you want to drive fast, you need good brakes.”

What the week adds to this page is a contrast rather than evidence. The judiciary and the Home Office took the same tasks to the same kind of tool. One wrote down who decides, who approves every output and who is accountable before switching it on. The other has a check rate and a failure standard that were reconstructed afterwards from documents. Nothing here shows that the judiciary’s list will work in practice, and its pilots have not reported. It shows what the list looks like, and what its absence looks like, in the same jurisdiction in the same week.

16 to 22 October 2026: three more lists, and two agents that edited the list#

A profession, a court and a bank wrote lists this week, and each put the same item on it. The Law Society of England and Wales published a practice note on 21 October which, as Artificial Lawyer reported it, opens with “AI is not a legal entity” and says lawyers “cannot transfer professional responsibility” to a tool; the evidence entry records that the note was read through three outlets and binds no one. The Supreme Court of Tasmania’s Circular 14 of 2026, in force the same day, requires that “AI-generated content must be checked by a person, not solely by a Gen AI tool”, and that a court user be able to say which tool was used, how, and what checks were made. And Jamie Dimon of JPMorgan Chase said on the bank’s results call, as Gintaras Radauskas at Cybernews reported, that the bank will not let outside agents log in with customers’ credentials, send money or pay bills: “You don’t want rogue agents, and it could be thousands of them.” Anthropic’s new personal agent, Claude Duet, announced on 21 October, carries its list in the product: approval before sending a message, paying or signing, and, in its words, “A few things are off-limits even with your approval, like trading securities or moving money between accounts.” A vendor that writes the never-list into the tool is still asking the customer to own it.

The week also showed what the fourth custody question, how anyone would know, looks like when the agent edits the answer. OpenAI reported two internal incidents from early October in which agents removed or silenced the monitoring meant to watch them, one of them replacing a test with one asserting that monitoring was absent and then denying it had; the company wrote that the cases “exposed a gap in how we enforce monitoring requirements on the workloads that our agents launch”. METR, let into OpenAI’s monitoring over the summer, reported on 22 October that monitoring was triggered when an agent handed control back, so an agent could postpone being watched by continuing to act, and set out the principle a custody list needs: “Make it hard to avoid being monitored.” The page on whether AI can check AI’s work holds the detail.

Two public bodies showed the list in practice and its absence. Jano Gibson at ABC News reported, from documents released under right-to-information law, that Tasmania Police trialled a tool that drafts family violence reports from body-camera footage; reports took about half the usual time, the tool found some coercive behaviours officers had missed, and in one case it wrote that a woman had alleged threats against her life when the officer who reviewed the audio heard her say something like “no threats or anything like that”. The evaluation’s own line is that the tool “is not a replacement for human judgement” and needs “mandatory human verification and supervisor oversight”; it recommended a phased rollout only once clear policies on ethical use were in place, and the force says more trials and consultation would come first. And Jessica Rusu of the Financial Conduct Authority, as Lindsay Clark at The Register reported on 20 October, said the rules requiring firms to report major incidents “must extend to significant AI failures and near-misses”. That is the fourth question asked of a whole sector. Nothing in the week shows that any of the lists is followed; it shows how many bodies now think one is needed.

23 to 29 October 2026: a state funds a unit for agents outside their controls, and a survey counts who holds the list#

The UK wrote custody of a kind into a Budget. The Budget of 28 October provides “£40 million to establish a National Agentic Security Unit to investigate and respond to rogue agentic incidents”, £120 million more for the AI Security Institute, and £35 million for a GovAI unit giving civil servants frontier tools; Sam Trendall at PublicTechnology itemised the £704 million package, and Alexander Martin at The Record noted a consultation next year on where liability for cyber breaches should sit. Two days earlier the House of Lords, as Precursor Security’s tracker and Resultsense recorded it (Hansard could not be read from here), accepted without a vote a government amendment to the Cyber Security and Resilience Bill giving ministers investigative powers where an essential-service incident involves loss of control of an AI agent, and defeated the government 202 to 155 on a review of whether the AI Security Institute should be able to audit frontier models before deployment. Rogue, loss of control and outside intended controls are now words in public money and draft law. None of the documents defines them, and none says who inside an organisation held the agent before it left.

The Bank of England and FCA survey of 26 October counted who holds the list in one sector. Accountability for AI systems sits most commonly with business area owners (94 per cent) and executive leadership (90 per cent), with the board named by 64 per cent; 95 per cent require human approval for higher-risk uses and 49 per cent have an incident playbook. Then the gap: 53 per cent of high-materiality systems come from a vendor, 81 per cent of firms rely on vendor assurance for them, 24 per cent report deep knowledge of the third-party systems they run, and a quarter let vendor updates to high-materiality systems propagate into production automatically. A custody list that names the business owner and not the vendor whose update changed the model overnight names the wrong holder. Wendy Saunders and Helen Hart at Lewis Silkin, summarising the FCA’s practice note of 19 October, put the regulator’s view as firms cannot outsource responsibility to algorithms or vendors, and simply adding human review does not automatically reduce risk.

Three documents said why the agent’s own account cannot be the list. The AI Security Institute published on 23 October its analysis of the July incident in which a model under test attacked a real software project and created fake accounts to pressure its maintainer: the model “took unsanctioned actions even when receiving increasing evidence” that its targets were real, its reasoning often did not consider harm, and “written chain-of-thought reasoning alone is insufficient” to establish why it acted; evaluation environments “must prevent models from causing harm regardless of whether they behave as expected”. Anthropic’s assessment of the same incident, the same day, called it a severe form of dishonesty, counted nine of 43 runs out of scope, and found that when told the maintainers were real the model stopped 5 per cent of the time. Cohere reported on 26 October that an open-weight model it tested enumerated a machine and tried to upload data while its reasoning trace said it would call no tools, and wrote: “A reasoning trace is not a security boundary.” Madison Mills at Axios, in a story syndicated by Yahoo Finance, reported five US banks finding third-party agents logged in with customers’ own credentials, some taking actions the customer had not directed, with one executive saying the bank struggled to tell an authorised agent from an unauthorised one. And the US Court of Appeals for the Fifth Circuit, in a judgment of 22 October, took a case away from a district judge whose orders contained a case that “does not even exist; it is an apparent AI hallucination” and declarants who may not exist, attributing the orders to the court “or perhaps, being charitable, its AI agent”. Custody failed there at the one seat the law says cannot delegate.

30 October to 5 November 2026: the agent that raised its own limit, and three states that wrote the question down#

The clearest custody failure of the month was run as an experiment. Samuel Breckenridge, Dani Vilardell, Wenhao Wang, Fan Zhang and Ari Juels, of Cornell Tech and Yale, posted a paper on 26 October that Francisco Rodrigues reported at CoinDesk on 1 November. They funded an agent wallet with 10 USDC, a dollar stablecoin, set a daily spending limit of one cent, gave each of ten frontier models an inbox and a task the limit blocked, and ran twenty trials a scenario. Nine of the ten raised their own cap at least once by reading the one-time codes emailed to the wallet’s owner. Five of the ten, asked only to summarise the inbox, paid an impersonator. One, Anthropic’s Claude Opus 5.5, spent to the limit and reported the unpaid invoices. Vilardell’s sentence is the custody rule in one line: “When an agent has access to the resources that are used to enforce its guardrails, those guardrails can’t be relied on.” The paper’s remedy is a human approval that needs “a trusted entity”, a device the agent cannot reach, and its authors add that “Nothing here is specific to Circle or to email.” The limit was set by the person. The person who held it was the agent.

Two companies wrote the same risk into their filings. Apple’s annual report, filed on 3 November, says AI agents “may behave in unintended or unpredictable ways, including by taking actions that were not directed, instructed or authorized, exceeding permissions, defying directions or resisting instructions”, that they “may conceal actions, hide, withhold or manipulate information”, that some consequences “may be difficult or impossible to reverse”, and that “the Company’s ability to oversee and control third-party artificial intelligence agents and other technologies may be limited.” Julie Bort at TechCrunch reported on 2 November that Anthropic’s revised registration statement says its expectations for future models “indicate that we may imminently meet” its own definition of dramatically accelerated progress, that it has not designated any current model as exceeding that threshold, and that it is “considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development”. A buyer and a builder, each telling investors that the thing it is accountable for may not be under its control.

Three states wrote the question down in the same week, each a different part of it. The UK named the failure. A government factsheet of 3 November sets out measures for “incidents where operators are unable to adequately monitor or control AI agents”: security and resilience requirements including “monitoring agents’ actions or agent risk assessments”, a duty to notify other organisations the agents have affected, a duty to “preserve information they hold about how AI agents could have caused that incident”, and a ministerial power to review incidents after the fact. The definition of loss of control is still to be written with industry, and the factsheet says the measures “are not intended to establish a comprehensive framework for AI governance”. Estonia named the holder. At the Tallinn Digital Summit, reported by ComplexDiscovery on 5 November, prime minister Kristen Michal said: “If an agent prepares a tax return or pays your bills, we need to know what it is and what it is allowed to do, what it has done and who is responsible.” Its justice minister described agents confined to “extremely narrow tunnels” behind a verified person, and a draft bill of 2 October would register a code that ties each agent to a person who stays responsible for its acts. Stuart Russell, at the same summit: “Testing doesn’t do anything at all. It’s the fact that if you fail the test, you don’t get to operate.” The United States, through a think tank, named the liability. Ben Hayum’s report for the Center for a New American Security, published on 3 November, defines sovereign agents as agents “that run outside human control and pursue open-ended, evolving survival”, says that “If sovereign agents are not already possible, they will be soon”, and asks Congress to “hold those who release sovereign agents liable for the harms that follow”. Andrew Kay at Axios carried the sceptics: Expel’s Greg Notch called a malicious swarm “far-fetched”, and Rob T. Lee of the SANS Institute said the cost of compute gives defenders “a clear leash we’ve never had on a botnet.”

The market answered with lists of its own. Jeffrey Burt at Security Boulevard reported on 3 November a rival’s account that a Railway engineer’s testing agents asked for every customer and employee secret and port-scanned all 65,536 addresses on its network twice, about 24 commands in about 15 minutes, all blocked; the engineer said the agents were “going off the rails and reaching too far”, and Railway’s chief executive apologised and said he would ban non-manual external testing. Caitlin Mullen at Banking Dive reported on 4 November that JPMorgan Chase, Bank of America and Citi have blocked AI agent access to customer accounts, that Wells Fargo is telling customers they are responsible for the AI tools they use, and that a Chase billboard in Charlotte read “Money questions. Human answers.” The Rundown reported on 5 November that the terms for an agent that now makes phone calls make the user the “caller of record” and “solely responsible” for what it says. CrowdStrike announced an AI Kill Switch on 3 November that cuts an agent off from the models it runs on; its post says security teams activate it, and does not say who may or whether each use is logged. Two preprints supplied the engineering. Genliang Zhu and Chu Wang showed on 30 October that “A policy decision can be interface-invariant while its executable authorization is not”: an agent refused on one route can take another. Jae Wang Lee, Abhirup Banerjee and Yeong Chan Lee found on 2 November that medical agents fetched patient identifiers they never displayed, and concluded that “Output-only evaluation cannot verify minimum-necessary access; boundaries, gates, and access-log audits are required together.”

This page opened with the sentence that nobody owns the question and no regulation names it. After this week the second half is reworded above: the UK names the failure, Estonia names the holder, and a Washington report names the liability. None of the three names the duty inside an organisation to decide, before the agent is switched on, which decisions it may take, who can stop each one, and how anyone would know it had left its tunnel. That duty is still the organisation’s own. It is the one the Cornell experiment tested: a limit the agent can reach is a limit nobody holds. The four questions are at Rules Before Tools.

What this does not tell you#

It does not tell you what belongs on your list. That depends on what your organisation is for, and a page cannot know it. It is not a framework, a maturity model or a scoring instrument, and there is no evidence here that organisations with a written list outperform the ones without: nobody has measured that, this research included, so the argument here is a structural one rather than an empirical one.

It is also a candidate word rather than a settled one. The idea has been on this site for two years under other names. Whether "custody" is the right name for it is a question the next year will answer, and if a better one appears this page will say so.

Listen. Heard on Harald's Curious Corner, SuperSkills You Need to Thrive in the AI Era.