Responsible AI use in retail doesn’t begin with an ethics policy. It begins with a moment most professionals have experienced and quickly moved past. A confident-looking output that sounded plausible, and nearly made it into a presentation before someone thought to check the source.

That gap between “this reads well” and “I can stand behind this” is where professional credibility starts to erode.

The tools are useful. AI is usually fine for drafting and summarising.

But some tasks are high-risk.

They’re often the ones you’ll be asked to defend to senior leaders. Open-ended claims. Statistics. Regulatory detail. Anything framed as “what the research shows.”

Why Familiarity Creates the Risk, Not the Safety

The standard advice when starting with AI at work is to verify outputs before using them. That advice is sound and works well during the early phase, when the tool feels unfamiliar, and every output gets scrutinised. The problem is what happens once that phase ends.

Automation bias is the documented tendency to trust confident-looking computer output more than the evidence warrants, even when you know enough to be sceptical.

It shows up in aviation, in medical diagnosis, in any environment where humans and automated systems share decision-making. The cause isn’t carelessness. It’s that if something looks professional and structured, the brain relaxes. AI amplifies this because AI output is fluent and carries no visible hesitation. When a model doesn’t know something, it tends not to say so. It fills the gap with something plausible.

The Stanford AI Index Report 2026 captures this with a specific finding. When questions are built around false premises, where an incorrect assumption is embedded in the question itself, models tend to confirm the framing rather than challenge it.

In one test, GPT-4o’s accuracy dropped from 98% to 64% solely because of the question’s structure. Same tool, same type of task. The question smuggled in a false belief, and the model built a confident answer on top of it. For CX teams, this means a prompt that carries a built-in assumption about customer behaviour is likely to return a well-structured answer that reflects that assumption back, regardless of whether the assumption is accurate.

The Two Tasks That Define Responsible AI Use in Retail

Understanding where AI is reliable and where it isn’t is the practical foundation of responsible AI use in retail.

In practice, the key is knowing what you’re actually asking the tool to do.

Here are two different types of tasks we ask AI to do.

In the first, the tool works with the material you provide. It could be a document, a set of verbatim feedback, or a draft to rework. The AI is doing a restructuring job. It has something solid to work from, and the output can be checked against what was given. According to the Stanford AI Index, GPT-4o’s hallucination rate on grounded summarisation tasks like this, where the answer is contained in the source material, sits at around 1.5%.

In the second, the tool is given nothing to work with. You might ask it to generate: “What does the research say about returns behaviour in UK retail?” or “What are typical satisfaction scores for fashion?”

The model draws on its training data, which has a cutoff date and may be limited or skewed. It cannot always distinguish between something it knows with confidence and something it is essentially filling in. On open-ended factual tasks where the model is reaching beyond provided material, accuracy can fall well below 40% by the same benchmarks.

The output looks identical either way. Nothing in the language signals which type applies.

Customer Data, AI Tools, and What the ICO Actually Says

A separate risk surfaces when professionals paste customer data into AI tools to speed up analysis. Verbatim complaint details, names, email addresses, and store location codes. These feel like raw material for a useful summary. But you should know before pasting what happens to that data.

Most consumer AI tools (on standard pricing tiers) can use user inputs to improve their models. That means verbatim customer feedback submitted for quick analysis may not remain within a controlled environment.

The ICO’s guidance on AI and data protection is explicit: customer data processed through an AI tool remains personal data. The lawful basis for its collection, the purpose for which it was collected, and the protections it requires follow the data regardless of which tool it passes through.

The Data (Use and Access) Act 2025 further tightened these requirements. Retailers must be able to demonstrate meaningful human review of automated processes and define specific, limited purposes for any AI data processing. For most retail teams, the issue isn’t intent; it’s that a clear, practical conversation about what can and cannot go into a consumer AI tool hasn’t happened. The ICO guidance is there. But the translation into day-to-day habits often isn’t yet.

Three Questions for Responsible AI Use in Retail

The retailers approaching this most thoughtfully are those building the right questions into their deployment decisions rather than leaving verification entirely to individual judgment.

M&S distributed 11,000 Microsoft 365 Copilot licences to every store manager and support centre colleague earlier this year, alongside a structured training programme covering how the tools should be used and what they shouldn’t be used for. Chief executive Stuart Machin described the aim as freeing colleagues to spend more time with customers. Access came with explicit guidance on scope, not just the technology.

John Lewis Partnership took a different but equally deliberate position. At the Retail Technology Show in April 2025, chief data and insight officer Barry Panayi described holding back on customer-facing AI as a very deliberate organisational decision. The reason was that customer-facing use cases rely on customer data. So AI deployment has stayed focused on supply chain, routing, and internal finance, areas that don’t raise the same data obligations. That’s an organisation that asked what data goes in before deciding where to deploy.

Both approaches share a common logic: the question of where and how AI is used gets asked early, before anything goes live.

You could apply the same logic at the individual level by asking three questions before opening the tool.

1. What kind of task is this? If the AI has been given material to work from, the output can be checked against it. If it’s being asked to generate without source material, the verification burden is higher and the error rate differs. The language in the response won’t signal which category applies. The task type does.

2. What data is going in? For most everyday tasks, this isn’t a concern. For anything involving customer names, contact details, complaint specifics, or any identifiable information, the ICO’s guidance applies, regardless of the tool used. It’s good practice to remove personal data from analysis and always use an enterprise AI tool. It’s best to check your organisation’s policy for the finer details of what’s permitted.

3. What am I putting my name on? A slide going into a leadership meeting, a written recommendation, a briefing document — these are professional outputs. The tool generated the draft. The conclusions belong to the person who submitted them.

---

Key takeaways

• Automation bias accumulates with familiarity, not inexperience. The riskiest phase isn’t the beginning, when every output gets checked; it’s later, when the tool feels reliable, and verification habits relax
• AI accuracy varies significantly by task type. The same model that performs reliably on summarisation tasks can drop substantially on open-ended generation, and nothing in the output signals the difference
• Customer data doesn’t lose its legal status when it passes through an AI tool. The ICO is clear on this, and the Data (Use and Access) Act 2025 has strengthened the obligations further
• The most considered approach to responsible AI use in retail isn’t individual vigilance alone. It’s organisations asking the right questions at the point of deployment, as M&S and John Lewis Partnership have done in contrasting ways
• Three questions asked before opening a tool (what kind of task is this, what data is going in, what am I putting my name on) can shift the relationship with AI from passive consumption to active judgment

---

Other episodes about AI in retail:

• Episode 27: The AI Skills Gap in Retail Is a Prompting Problem — Not a Tool Problem — The first episode in this mini-series; covers why AI productivity in UK retail stalls at access rather than capability
• Episode 5: Boost Your AI Skills – AI Literacy for Retail Professionals – Helps you get started with the basics

---

If this episode resonated with you, I’d love to hear your thoughts. What’s one thing you’re planning to put into practice or one learning you took from today’s episode?

Let’s connect – Find me on LinkedIn (https://www.linkedin.com/in/jo-williams-ccxp/)

Found this valuable? Please leave me a review. It helps other retail professionals discover these conversations and honestly means the world to me.