When AI Hallucinations Meet the Courtroom

By Naz Keceli, Co-Founder & CEO at Khiliad Ltd

On 17 November 2025, a three-judge panel of the Upper Tribunal handed down one of the most significant UK judicial decisions yet on artificial intelligence in professional practice: UK and R (on the application of Munir) v SSHD [2026] UKUT 81 (IAC).

The case concerned fictitious legal authorities cited in tribunal proceedings, almost certainly generated or influenced by AI. But the judgment reaches well beyond immigration law. It sets out firm principles on professional responsibility, supervision, verification, and data protection in an age where generative AI is woven into everyday workflows, often without people realising it.

Critically, the decision is not anti-AI. It is anti-carelessness. The Tribunal explicitly endorsed specialist legal AI tools and described them as beneficial. What it condemned was the uncritical use of general-purpose AI and the professional failures that follow.

What Happened: Two Firms, One Problem

TMF Immigration Lawyers

Mr Tahir Mohammed submitted grounds of appeal citing Horleston v SSHD [2007] EWCA Civ 654. The case does not exist. The citation belonged to an unrelated equal pay case. When the Tribunal queried Google's AI features about the fictitious case, it generated plausible but entirely fabricated judicial compositions, with different panels appearing depending on how the question was phrased.

Mr Mohammed initially denied using AI but later conceded he could offer no other explanation for how a non-existent case with a real but unrelated citation ended up in his submissions. He also admitted uploading Home Office decision letters and client correspondence into ChatGPT for summarisation and redrafting, raising a separate data protection concern.

He self-reported to the IAA and SRA before the hearing. That mattered significantly to the outcome.

City Law Practice

The second case involved Mr Zubair Rasheed, a senior solicitor and Compliance Officer for Legal Practice (COLP) at City Law Practice. He had delegated the drafting of grounds for judicial review to his brother, described variously as a "part-time trainee lawyer" and a "very junior caseworker."

Four false authorities were cited. Upper Tribunal decisions were given High Court citations, a first-instance decision was attributed to the Court of Appeal, and one case simply did not exist. When UTJ Blundell attempted to verify the citations, it became what the judgment memorably describes as a "fool's errand."

The subsequent hearing exposed a catalogue of wider problems: inadequate supervision, poor record-keeping, contradictory evidence about when files were deleted, and a troubling lack of awareness about how embedded AI already is in tools as commonplace as Google search. Mr Rasheed was referred to the SRA.

The contrast between the two cases sends a clear message. Early admission and genuine remorse mitigate. Evasion and a lack of insight aggravate.

The Tribunal's Key Principles

1. The Duty Not to Mislead Is Absolute

It does not matter whether a false citation was generated by ChatGPT, surfaced by Google AI, copied from a flawed precedent template, or simply made up by a careless trainee. If it appears in a submission bearing your name, you are responsible. The Tribunal was unequivocal on this point: AI is not a defence, and delegation is not insulation.

2. Supervisors Bear Greater Responsibility

The Tribunal held that a supervisor who fails to check a junior fee-earner's work is more culpable than a lawyer who fails to check their own. The logic is straightforward: the individual lawyer who submits unchecked work fails the tribunal, the public, and their client. The supervisor who allows it also fails in their duty to develop junior lawyers and maintain the standards of the profession. Given that junior staff are often the earliest adopters of generative AI tools, this principle has real teeth.

3. Specialist AI Is Welcome; Uncritical Use Is Not

The Tribunal explicitly endorsed specialist legal AI platforms, describing them as "of enormous benefit" and "a step forward in legal practice." It drew a clear line, however, against relying on general-purpose tools like ChatGPT or Google AI for legal research without rigorous verification. The court is not resisting innovation. It is demanding that innovation be accompanied by professional discipline.

4. New Statement of Truth Requirements

The Upper Tribunal has amended its claim forms to require representatives to confirm that every cited authority exists, can be located using the citation provided, and supports the legal proposition relied upon. False certification will ordinarily result in regulatory referral. Other courts and tribunals are likely to follow.

5. Confidential Data and Consumer AI

The Tribunal stated that uploading confidential documents to general-purpose AI tools amounts to placing information in the public domain, thereby breaching client confidentiality and waiving legal privilege. It directed that such conduct should be referred to the Information Commissioner's Office.

The underlying warning is valid and important. But the technical framing needs refinement, which brings us to the most interesting part of the judgment.

The ChatGPT vs Copilot Distinction: Right Instinct, Wrong Detail

At paragraph 21, the Tribunal contrasted "open source AI tools such as ChatGPT" with "closed source AI tools which do not place information in the public domain, such as Microsoft Copilot." It suggested the latter could be used for summarisation tasks "without these risks."

The instinct behind this distinction is sound: not all AI tools handle data in the same way. But the specific claims are technically inaccurate in several respects, and it matters because practitioners will read this judgment and draw practical conclusions from it.

ChatGPT is not open source. It is a proprietary product owned by OpenAI. "Open source" has a specific meaning in software: it refers to code that is freely available for inspection and modification. ChatGPT's models are not open source. What the Tribunal meant was that ChatGPT is publicly accessible, which is a different concept entirely.

Microsoft Copilot is not inherently safe. Microsoft uses the "Copilot" brand across multiple products with very different data handling characteristics. The free consumer version is publicly accessible and does not offer the same protections as Microsoft 365 Copilot, the enterprise product integrated into Word, Outlook, and Teams. Even the enterprise version routes data through Microsoft's cloud infrastructure, meaning data leaves the organisation's immediate control. Meanwhile, ChatGPT offers its own enterprise tier with SOC 2 compliance, data encryption, and contractual commitments not to use customer data for model training. A firm using ChatGPT Enterprise with a signed Data Processing Agreement may well have stronger data protection safeguards than one using the free consumer version of Copilot.

Uploading to ChatGPT does not place data "in the public domain." When you upload a document to ChatGPT, it is transmitted to OpenAI's servers. Under certain tiers and unless the user opts out, the content may be used to improve models. That is a real risk. But it is not publication. The document is not indexed by search engines or made accessible to other users. The more precise concern is loss of control over confidential data and potential non-compliance with UK GDPR, not public disclosure in the traditional sense.

Why this matters practically: if practitioners read this judgment and conclude that simply switching from ChatGPT to Copilot solves their compliance concerns, they will be wrong. The real distinction is not between product brands. It is between consumer-grade AI tools of any brand and enterprise deployments configured with appropriate data processing agreements, access controls, and contractual safeguards. The question is not "Which AI brand do we use?" but "What are the data processing terms, where does our data go, and does this arrangement meet our regulatory obligations?"

Lessons Beyond Law

Healthcare

The parallels are immediate. A clinician who relies on an AI-generated diagnosis or treatment suggestion without verifying it against clinical evidence is in exactly the same position as the lawyer citing a hallucinated case, except that the consequences in a clinical setting can be irreversible. Patient data is among the most sensitive categories under UK GDPR, with additional protections under Article 9 for health data as a special category. The supervisory principle applies with equal force: a consultant who knows or suspects that junior colleagues are using unvalidated AI tools bears a heightened responsibility for the quality and safety of the resulting work.

Financial Services, Accountancy, and Other Regulated Professions

The same structural risks apply wherever professional duty of care exists. A financial adviser who uses AI to generate client recommendations, an accountant relying on AI-drafted tax advice, or an engineer using AI-generated calculations must all apply the same discipline: verify the output against authoritative sources before acting on it. Professional judgement cannot be outsourced to probabilistic text generation, however polished the output appears.

Practical Recommendations

Drawing on both the Tribunal's guidance and the technical realities of AI in 2025 to 2026, any regulated organisation should consider the following.

1. Develop a formal AI usage policy that distinguishes between approved specialist tools and general-purpose platforms, specifying permitted use cases and mandatory verification steps.
2. Conduct a Data Protection Impact Assessment before deploying any AI tool that will process personal or confidential data, and ensure appropriate Data Processing Agreements are in place.
3. Train all staff on hallucination risk, not once, but continuously. As AI becomes embedded in everyday tools including search engines, the risk surface keeps expanding.
4. Embed verification workflows for any AI-assisted output that will be relied upon professionally. In law, that means checking every cited authority. In healthcare, validating against established clinical guidelines.
5. Strengthen supervisory practices. This judgment makes clear that supervision means ensuring the people you supervise understand the tools they are using, the risks those tools present, and the checks required.
6. Do not treat brand names as compliance strategies. Evaluate the deployment model, data processing terms, and governance framework of every AI tool against your regulatory obligations.

Conclusion

[2026] UKUT 81 (IAC) is a judgment that every regulated professional should read. The Tribunal was right to enforce professional standards, right to insist on verification and supervision, and right to treat confidentiality breaches with seriousness.

Where it lacked precision was in its technical framing of how different AI tools handle data. But that gap reflects a broader reality: most institutions, judicial and otherwise, are still building fluency in AI architecture. The irony is that this only strengthens the judgment's central point. If you are going to use AI in professional practice, you need to understand what you are actually using.

The organisations that come through this transition well will be those that embrace AI deliberately, with governance, clear policies, robust verification, and an unwavering commitment to the professional standards that existed long before anyone had heard of a large language model. The alternative, as these two cases demonstrate with uncomfortable clarity, is reputational damage, regulatory referral, and a fundamental failure of the duty owed to clients and the public.

Case reference: UK and R (on the application of Munir) v SSHD (AI hallucinations; supervision; Hamid) [2026] UKUT 81 (IAC), promulgated 17 November 2025.