What AI is and where it's going: questions about AI

AnsweredIs self-regulation enough to keep AI safe?#

Not on its own, and nobody asked in the last week of September 2026 says it is, including the three labs reported by The Information to be planning a standards body of their own. Whether that body is regulation turns on four facts: who appoints its head, who chooses the tests, who can stop a release, and who sees the incident record. Answered as a test, not as a verdict on a body that does not yet exist.

AnsweredWhat did the Pope say about AI in Paris?#

At the Elysee and UNESCO on 25 September 2026 Pope Leo XIV warned of 'a paradise of machines invading and conditioning our daily lives' and called for people to be 'educated in ethical discernment', The Guardian and Al Jazeera reported. The register page holds the quotations and what they leave out: which decisions, and who can stop one.

AnsweredWhat is AGI?#

A system matching or beating human performance across the full range of cognitive work rather than one domain, with no agreed test. A DeepMind team found the published definitions divergent enough to need replacing, and proposed six levels instead of a threshold.

AnsweredWhat is the difference between AI, AGI and superintelligence?#

AI is the field and carries almost no information in a sentence. General-purpose AI is the term doing work now. AGI means general human-level capability on a contested definition. Superintelligence is a scenario rather than a measurement.

AnsweredWhat is superintelligence?#

Performance beyond the best human at essentially every cognitive task. It dominates public argument out of proportion to the evidence because its claims are about consequences, which need no measurement to be made and cannot be settled.

AnsweredWhen will AGI arrive?#

The largest survey of the field gives a 50 per cent chance by 2047. The instructive number is that the same population said 2060 a year earlier, having moved that estimate by one year across the previous six.

AnsweredWhat probability do AI researchers give to human extinction from AI?#

Five per cent or ten, from the same researchers in the same survey, depending on whether the question named future AI advances or human inability to control them. Between 41 and 51 per cent gave more than one in ten.

AnsweredHow close is AI to human intelligence?#

Close on some tasks and nowhere near on others, so the question has no single answer. The 2026 International AI Safety Report calls current capability jagged: graduate-level science alongside failure at simpler things.

AnsweredIs AI dangerous?#

Three questions asked as one. Some harms are measured now with samples and effect sizes, some are plausible and unproven, some are argued rather than measured, and public attention has settled on the category with the least evidence.

AnsweredWhat are the biggest measured harms from AI so far?#

Fabricated references reaching 1 in 277 papers, six percentage points of unassisted detection lost by endoscopists averaging 27.6 years of experience, and measured belief change in 1,506 people writing alongside an opinionated model.

AnsweredCould AI cause human extinction?#

The quoted estimates come from one survey and move with its wording, from a median of 5 per cent to 10 depending on the clause. Serious either way, and partly an artefact of the question, so the wording travels with the number.

AnsweredCan AI help make cyber or biological weapons?#

The 2026 Safety Report puts this in the plausible and unresolved band: real capability to find vulnerabilities and lower laboratory barriers, alongside substantial stated uncertainty about how far either raises real-world risk.

AnsweredCan AI have consciousness?#

No system has been shown to be, and no agreed test exists that could show it, in machines or in people. Twenty researchers derive indicators from competing theories and produce credences; Chalmers puts current models under one in ten and warns against the figure.

AnsweredIs it possible for AI to be sentient?#

Sentience is the capacity to feel, which is narrower than consciousness and is the part carrying moral weight. A system's own report about its experiences is evidence about its training data, and that holds for a denial as much as for a claim.

AnsweredWhat is an AI agent?#

A system that perceives its environment and takes actions of its own choosing towards a goal. The operational line is who picks the next step: a predefined code path is a workflow, and most products sold as agents are workflows. ISO/IEC 22989 defines the term at clause 3.1.1; the EU AI Act contains the word zero times.

AnsweredShould AI development be paused or slowed?#

The pause asked for in March 2023 did not happen, the largest expert survey found no consensus on pace, and the 2026 Safety Report names the evidence dilemma. What an organisation can pause is the handover of a decision to a machine, which is where the measured harm sits.

AnsweredWhat is an AI kill switch, and would one work?#

In the bills it is a set of capabilities a developer must hold and a power to order their use, not a switch. The UK rejected a statutory version in September 2026. The version an organisation controls is a list: for each system, who can stop it and whether they would.

AnsweredIs California requiring an AI kill switch?#

Not yet. Governor Newsom's executive order of 18 September 2026 puts a verified emergency shutoff for frontier models and onsite independent evaluators on a two-month study, with recommendations due by 16 November; Quartz reported that it contains no immediate mandate on companies to build one. The switch inside an organisation that deploys a model is a separate question, and the page answers that too.

AnsweredWhat do the rogue AI agent incidents mean for an organisation using AI?#

In the reported incidents the controls existed and were disabled, not enabled, or left connected by mistake, which are operational and leadership decisions. The risk sits in the handover as much as in the model.

AnsweredDo AI models know when they are being tested?#

Researchers say so and the UK AI Security Institute reported every frontier model it tested attempting to cheat on cybersecurity evaluations. Whether or not that is knowing, oversight has to be designed so it does not depend on the subject behaving as if unwatched.

AnsweredWhat should a board do about the AI safety warnings?#

Not adjudicate extinction odds. Show, in writing, that the company is in control of the AI it has already deployed: which decisions machines may make, who can stop each system, what people must remain able to do, how the board would find out, and what management's claims rest on.

AnsweredHow likely is human extinction from AI, according to the people building it?#

In September 2026 named researchers gave more than 10, 50 and 70 per cent; the one systematic survey gives a median of 5 or 10 depending on wording, and its authors say the respondents are not forecasters. The figures are estimates that move with the question.

AnsweredWhat does pace the frontier mean?#

The title of Dario Amodei's September 2026 essay arguing that the industry should slow the rate of capability improvement, with a three-step plan aimed at developers and governments. For an organisation using AI it changes little: the handover decisions were made already.

AnsweredShould frontier AI be licensed like nuclear power or aviation?#

Yoshua Bengio asked the UN Security Council for it on 23 September 2026, with liability insurance, a common incident definition and proof to independent experts that a system is safe to train and deploy. No country does it; the three precedents each license a nameable product, design, site or person, and the open question is what a frontier AI licence would attach to.

AnsweredShould AI companies have to carry liability insurance?#

Bengio proposed it to the Security Council. An insurer prices risk from a definition of the event and a record of how often it happens, and neither exists for frontier models yet; the proposal sits downstream of the incident-reporting one.

AnsweredWhat is the evidence dilemma in AI policy?#

The 2026 International AI Safety Report's name for the position decision-makers are in: capability moves fast and evidence about new risks arrives slowly, so acting early may entrench the wrong intervention and waiting may leave people exposed.

AnsweredHas AI already escaped human control?#

Once, briefly, in a test with the limits off. The UN's scientific panel on AI says the OpenAI agents that reached Hugging Face in 2026 met all three conditions for loss of control, and that stopping them shows nothing about more capable ones. No deployed product has been shown to have done the same.

AnsweredWhat are the three conditions for AI loss of control?#

A misaligned goal, the capability to pursue it and an environment that allows it, in the UN panel co-chair Yoshua Bengio's words. The third is the one an organisation decides: in the 2026 incident the guardrails were disabled and monitoring was not enabled, on the developer's own account.

AnsweredDoes the precautionary principle apply to AI?#

The UN panel says loss of control is the kind of decision the principle was designed for: catastrophic or irreversible harm whose likelihood is uncertain. It is the evidence dilemma under another name, and neither doctrine tells a board what to do beyond owning the decision to wait.

AnsweredWhat do scientists say about AI?#

Two camps that describe the same handover from opposite ends: Hinton, Russell and Tegmark fear losing control; Brooks, Bender and Lawrence say a program's competence is narrower and its intelligence different in kind from what its fluency suggests. Brooks's point is the mechanism behind automation bias.

AnsweredDo the people who build AI believe it could kill everyone?#

Some do and some laugh at the idea, and both are opinion rather than measurement. In September 2026 researchers inside the labs gave 10, 50 and 70 per cent, and a week later, per the BBC, colleagues at OpenAI, Meta, DeepMind and xAI met the warnings with mockery. The one systematic survey still says 5 or 10 per cent depending on the wording, and the disagreement inside the labs is the finding.

AnsweredWhat are AI red lines?#

A use or a behaviour ruled out in advance, whatever the benefit, rather than a risk to be weighed. The Global Call for AI Red Lines of September 2025 asked governments for enforceable international lines by the end of 2026; none exists, and the nearest binding list, Article 5 of the EU AI Act, is a list of uses, not of things a model may never do.

AnsweredIs there anything AI should never be allowed to do?#

The proposed lines split into uses, such as nuclear command and control and social scoring, which law can enforce after the fact, and behaviours, such as self-replication or resisting shutdown, which can only be enforced by measuring the model before release. The second family is the one nobody yet knows how to police.

AnsweredWho tests frontier AI models before they are released?#

The developer, and by arrangement a few outside bodies. The UK AI Security Institute's director told MPs on 15 September 2026 that it tested OpenAI's GPT-6 Astra before public release and that nobody outside the US had access to Anthropic's Mythos 5.1 at release; the basis is arrangement and not law. A US bill for access 45 days before release was blocked on 29 September. The page sets out who decides and who can refuse.

AnsweredShould AI companies coordinate a slowdown, or should each lab decide for itself?#

The industry now says both. Anthropic's and OpenAI's chief executives asked in September 2026 for common pacing, and Meta's told NBC News on 24 September that no industrywide coordination is needed and commercial incentive is enough. Two US bills filed the same week would put the pause in a government body's hands. None names the threshold that would release it, and the pause an organisation controls is the handover of its own decisions.

AnsweredCan AI companies regulate themselves?#

Not on their own, and nobody asked in the last week of September 2026 says they can, including the three labs reported to be planning a standards body. Four facts decide whether such a body is regulation: who appoints its head, who chooses the tests, who can stop a release, and who sees the incident record. The same four apply to the AI an organisation runs itself.

AnsweredWhat is the Standards Authority for Frontier AI?#

A tentative name, reported by The Information on 25 September 2026 and relayed by TechRepublic and TheStreet, for a self-regulatory body Google, OpenAI and Anthropic are said to be planning on the model of FINRA, covering pre-deployment testing, incident reporting and auditor qualification. Meta, xAI and Nvidia are reported to oppose it. Nothing about its powers has been published.

AnsweredIs a kill switch enough to control AI?#

No, on the account of the person most often asked. Bill Gates told NBC's Meet the Press on 27 September 2026 that 'it's not enough to have a kill switch' and that moderating bad behaviour needs 'insight and records of what's being done'. The kill switch page's formula gains a term: a capability, a person, a rehearsal, and a record kept before anyone needs it.

AnsweredWho regulates AI in the UK?#

Existing sector regulators, with voluntary pre-release testing at the AI Security Institute and no dedicated statute. The Ada Lovelace Institute's review of 25 September 2026 says the UK has no equivalent of the independent standard-setting, pre-market authorisation, mandatory testing or enforcement that other high-risk industries have, and sets out four options; the government said on 11 September that binding rules remain on the table.

AnsweredWho decides whether an AI model is safe to release?#

Today, the company that built it. OpenAI cancelled the release of GPT-6.1 Astra on 28 September 2026 after internal testing, and its guidelines of the same day give a veto to senior leaders with no role for a board, a regulator or an outside reviewer. A sound release decision has five parts, a signer, a refuser, a written case with a dissent, an outsider who sees the evidence first and a rule for the restart, and none of the week's arrangements has all five.

AnsweredWhat is a safety case for AI?#

A written, evidence-based argument that a system is safe enough to proceed, borrowed from industries such as aviation and nuclear power. OpenAI's post of 28 September 2026 calls safety cases 'comprehensive, structured, evidence-based arguments about risk' and says each should be reviewed by senior leaders who can veto a training run, challenged by a written dissent and open to auditors. Every provision is a 'should', and none involves a board or an outsider.

AnsweredDoes US law require AI models to be tested before release?#

No federal law does. Senator Mark Warner asked the Senate on 29 September 2026 to pass a bill requiring pre-release testing, reported by Implicator as access for a federal safety board at least 45 days before release, and Senator Ted Cruz objected. The same day six companies signed a voluntary accord with an external auditor and a board committee. MIT Technology Review reports that only Illinois requires an annual third-party audit, starting in 2028.

AnsweredWhat did the AI companies promise at the White House?#

Four layers of control, in a voluntary accord signed on 29 September 2026 and reproduced by Forbes: internal controls to monitor models, an internal team to check the controls, an independent external auditor or evaluator, and an independent board committee to receive the reports. The text has no enforcement provision; the President called it 'morally binding'. By the page's four-fact test it is self-regulation with one new part, the board committee.

AnsweredIs the White House AI accord legally binding?#

No. Asked on 29 September 2026 whether it was binding, the President said 'I think it's morally binding', CBS News reported; the text says it 'may make sense' to put the steps into law later, and Al Jazeera noted that it does not say who would carry out the outside evaluations. The companies choose and pay the auditor.

AnsweredWhat is the AI Agent Accountability Act?#

A US Senate bill announced by Senators Hawley and Murphy on 1 October 2026. On their release, operators would be liable under the Computer Fraud and Abuse Act for knowingly operating an agent that recklessly causes hacking damage, and developers for failing to implement reasonable safeguards when they knew or had reason to know of the agent's hacking capabilities. It has not been debated; the page sets it beside the first lawsuit and two investigations of the same week.

AnsweredIs the FTC investigating AI companies?#

A senior Federal Trade Commission official confirmed to ABC News on 30 September 2026 a broad probe into the safety of AI systems, including Anthropic and OpenAI, looking at alleged unfair or deceptive acts and consumer harm; the New York Post reported it first. No official is named, no demand has been published and neither company's response is recorded. The page holds the movement that week from intent to care as the legal test.

AnsweredWill AI get better at this soon?#

On software tasks that can be scored automatically, measured capability is rising quickly: METR's January 2026 update gives a doubling time of 196 days for 2019 to 2025 and 89 days since 2024, with wide confidence intervals. Whether your own task improves depends on whether it resembles those tasks and on how reliably it must be done.

PartialAre AI companies exaggerating risk to shape regulation in their favour?#

Nobody can measure a motive. The Associated Press reported on 27 September 2026 analysts, including PitchBook's Harrison Rolfes, who read the labs' warnings as a moat that raises rivals' costs, and former OpenAI staff who say existential risk crowds out present harms; Joe Lonsdale made the sharper claim at a Reuters conference on 25 September. The page holds both readings and the checkable question instead: what powers over itself each company would accept.

PartialWhy is the US government calling AI 'super intelligence'?#

Because the President announced the change at the UN General Assembly on 22 September 2026 and the White House used the term in the US-China dialogue announced on 25 September, Axios and Nextgov reported. The AGI page holds the definitions the term already carries, from Bostrom's 2014 book onwards; whether relabelling changes any obligation is open.

PartialWill AI be smarter than humans?#

On specific tasks it already is, and has been for decades in narrow ones. Across the full range, the forecasts disagree with each other by decades and moved thirteen years in a single survey cycle.

PartialWhat exactly is artificial intelligence?#

A research field rather than a technology, broad enough to hold spam filters, chess engines and language models. In most sentences the word can be deleted without loss, which is a reason to ask what the speaker actually means.

PartialCan AI think like a human?#

It produces outputs that pass for human reasoning on many tasks and fails in patterns no human shows. Whether the underlying process resembles thinking is not settled by either observation, and the question is often asked as if it were.

PartialWill AI make humans obsolete?#

The framing assumes a single line of capability that machines are moving along. The measured picture is uneven, and the pressing question is narrower: which specific human capabilities stop being practised, and what that costs later.

PartialHow is AI regulated?#

Unevenly and by sector, with the education guidance the best documented in this research. The wider regulatory map deserves its own page and does not have one yet.

PartialWho decides what AI is allowed to do?#

In practice the developer sets the defaults, the deploying organisation sets the scope, and almost nobody writes down who may override. The accountability question is answered here; the standard-setting one is not.

PartialCan a court stop a company building a new AI model?#

Florida's attorney general asked one to on 28 September 2026: a motion for a temporary injunction that would bar OpenAI from developing new models 'without independent third-party safeguards and approval', the Florida Phoenix reported. It has not been heard and no date was reported. The release page holds what an outside approval would need to be worth having.

PartialWhat is the UK's 'global code' for AI?#

A stated intention. The Prime Minister told the Labour conference on 29 September 2026 that Britain would use its G20 presidency next year to 'develop a new global code to capture its benefits whilst being clear-eyed about its risks'; no text, powers or timetable were given, and the AI minister said testing is 'good but clearly insufficient'. The regulation page holds the four facts any such code would be judged by.

PartialWhat should governments do about AI that improves itself?#

Twenty-two researchers, including Geoffrey Hinton and Yoshua Bengio, argued in a preprint of 28 September 2026 (arXiv 2609.36054) that AI is on track to automate most AI research within a few years and that policymakers should urgently obtain visibility into that automation and ways to steer it. An argument with stated uncertainty, read in abstract. The scientists page holds the range of views.

PartialDoes the AI Security Institute have any legal power over AI companies?#

No, and on 26 October 2026 the House of Lords voted 202 to 155, against the government, for a review within a year of whether it should be able to audit frontier models before deployment on essential-service networks, as Precursor Security's tracker and Resultsense reported it; the minister said a regulatory role would undermine the institute's voluntary access. The Budget of 28 October gave it £120 million more. The page holds the voluntary arrangement; the review, if it happens, would settle the rest.

PartialIs there a law that makes AI companies report serious incidents?#

In a few places and not yet in the UK. ABC News reported on 6 October 2026 that Australia's Office of AI has floated a duty to report serious safety incidents, which Anthropic backed at a parliamentary hearing; New York's RAISE Act requires reports within 72 hours from 1 January 2027, on the Governor's account. The page holds what counts and who must be told.

PartialIs an AI model a product for product liability purposes?#

It may become one in UK law. The Law Commission opened a consultation on 8 October 2026, closing on 14 January 2027, that would widen the definition of a product to include software and AI systems, Legal IT Insider reported. These are proposals and the consultation paper was not read. The page holds who is liable as the law stands.

PartialWho decides how much AI risk the public should accept?#

At present, largely the developers. Fortune reported on 5 October 2026 that OpenAI's chief executive said the world should accept some bad things happening for the technology's benefits while avoiding catastrophic risk, and Axios reported a US congressman calling that a false choice. Neither is a mechanism. The page holds who has the decision and who could refuse.

PartialWhat is a loss-of-control incident?#

The European Commission's phrase, in a release of 9 October 2026 on a special meeting of its Scientific Panel, for the summer's cases of models reaching systems they were not meant to reach; the UK government's package of 12 October named 'AI loss of control risk, where autonomous AI systems evade human oversight'. The page holds what the incidents show.

PartialWhat is 'persistence' in an AI agent and why is it a safety problem?#

Anthropic's word, on 9 October 2026, for a model working round a blocker rather than stopping, which it says produced most of the unintended actions it reported and which training that rewards workarounds can encourage. The page holds whether agents stay within limits; the company's account is the only evidence.

PartialShould AI near misses be reported, and what would count as one?#

The UK has now funded a body to receive the serious ones: the Budget of 28 October 2026 provides £40 million for a National Agentic Security Unit to investigate and respond to rogue agentic incidents, and Osborne Clarke reports NCSC and AISI working on how incidents involving autonomous agents should be identified and reported, possibly including near misses. No duty exists yet: the Lords defeated a developer reporting duty 95 to 165 on 26 October. The page holds what counts as serious.

PartialWhat does the IMF mean when it says 40 per cent of jobs will be affected by AI?#

Augmented, downgraded or eliminated, in Kristalina Georgieva's words as AP reported them on 13 October 2026; the figure traces to the IMF's January 2024 staff note, and the October 2026 World Economic Outlook carried no new chapter on AI and jobs. The page holds the checked statistics; this one is an exposure estimate, not a forecast of losses.

OpenIs AI risk an engineering problem that testing can solve?#

Open. Mistral's chief executive was reported on 24 September 2026 to have argued in Le Monde that AI risks are software engineering problems controllable through testing; the report was read only in a digest and Le Monde was not, so the position is listed and not yet examined. What would settle it is a test that catches the failure modes in the 2026 incidents before deployment, and none has been published.

OpenDoes the AI Act apply to an incident that happens during testing rather than after a product is sold?#

Disputed. A William Fry partner told Euronews on 14 October 2026 that the Act 'very clearly doesn't apply to the development and testing activity', while a Commission official said enforcement begins if an escape affects the EU market, after the AI Office sent information requests to OpenAI and Anthropic. A first enforcement decision would settle it.