What should I never put into AI?

Three kinds of thing: information belonging to someone who has not agreed to share it, anything that would cause you a problem if it surfaced, and the part of your work you are supposed to learn from. The first two follow from how chatbots handle what is typed into them, and the answer shifts with the plan you are on. The third follows from what the learning evidence shows, and no setting fixes it.

A typed question is visible to the company that runs the model#

The UK National Cyber Security Centre set out the basics on 14 March 2023. A public chatbot does not learn from your query in real time, so what you type does not enter the model that answers the next person. The query is nonetheless visible to the organisation that provides the service, it is stored, and the authors say it will almost certainly be used in developing later versions. They also name three risks that sit outside the model: stored queries could be hacked, leaked or accidentally exposed; the provider could be bought by a company with different privacy practices; and queries from the same login can be combined, so a series of small questions can reveal more than any one of them.

The centre's advice is to leave sensitive information out of public chatbots and to avoid any query that would cause problems if made public. Its example of sensitivity is about the person and the moment as much as the data: a chief executive found to have asked how best to lay off an employee. Health and relationship questions it flags as revealing for the same reason. This is 2023 guidance, and provider settings have moved since.

The training default depends on the plan, and the plan is usually your choice#

OpenAI's help page, read on 19 October 2026, says content from its free, Plus and Pro plans may be used to train its models unless the user opts out. The switch is "Improve the model for everyone" under Data controls, or a request through its Privacy Portal. Temporary Chats are not used for training while they remain temporary, and saving one turns it back into an ordinary chat. Business, Enterprise and Education plans and the API are excluded by default. That is one provider on one date. Other services have their own settings, and a page that gives no date of its own should be re-read before you rely on it.

Employers add a second layer. Whether a manager can read your conversations on a work account is a separate question, covered at can my employer see what I put into AI.

Other people's information, secrets and the sentence you would regret#

• Information about clients, patients, colleagues or pupils. It is theirs rather than yours, and your employer's rules and data protection law may apply. This page does not assess either. Cisco's 2024 survey of 2,600 privacy and security professionals found 48 per cent admitting they had entered non-public company information into generative AI tools, which shows how routine the habit is. It was self-reported and the survey was run by a vendor.
• Passwords, keys and tokens. There is no reason for one to appear in a prompt. This is a precaution drawn from the centre's general advice about sensitive information, and it comes from no measurement.
• Anything that would be a problem if it surfaced. Apply the centre's test to the question as well as the data, because who asked and when can be the revealing part.
• Health, legal or relationship details tied to your name on a personal account with training switched on. The setting above is the control, and checking it takes under a minute.

The work you are meant to learn from#

The last item is a different kind of never. In a field experiment with nearly 1,000 high-school students, Bastani and colleagues gave one group unrestricted GPT-4 for practice. Their practice grades rose 48 per cent, and once access was taken away they scored 17 per cent lower than students who had never had it. A tutor version limited to hints raised practice grades 127 per cent and largely removed the harm. The result is about school mathematics over a bounded period, so it supports caution rather than a rule. For anything you are trying to become able to do, the exercise that builds the ability is the one to keep, and does using AI stop you learning sets out where the line seems to fall.

What is left open#

• Harm from stored prompts is not measured here. The sources describe what can happen to a query and what a provider says it does. None counts incidents.
• Settings change. The NCSC post is more than three years old and the OpenAI page is undated.
• One provider is described in detail. The same questions apply to any chatbot, but the answers differ by service and plan.
• The legal position on other people's data is not covered. It varies by country, sector and contract.

Key research and primary sources

• Chismon, D. and Paul J. (2023). ChatGPT and large language models: what's the risk? National Cyber Security Centre, 14 March 2023.
• OpenAI. How your data is used to improve model performance. OpenAI Help Centre, read 19 October 2026.
• Cisco Systems (2024). 2024 Data Privacy Benchmark Study. Release of 25 January 2024.
• Bastani, H. et al. (2025). Generative AI Without Guardrails Can Harm Learning. PNAS, 122(26).

Related SuperSkills research#

On work accounts, what an employer can see. On the learning side, whether using AI stops you learning and whether to let it summarise everything you read. On sensitive uses, AI for therapy or advice and financial advice from a chatbot.

About this research#

Rahim Hirji is the author of SuperSkills (Kogan Page, 2026), keynote speaker on AI and human capability, and founder of The SuperSkills Intelligence Company. The National Cyber Security Centre post and the OpenAI help page were read at their own addresses on 19 October 2026. The advice on passwords and keys is a precaution and is labelled as one. This page is information, not legal advice, and does not assess data protection obligations.

How this research works  ·  Reviewed quarterly  ·  Found an error? Tell me and it is corrected on the page.