Reasoning outsourced: the limits of artificial intelligence in the law

Welcome to the era of reasoning outsourced to Artificial Intelligence (AI). Outsourcing reasoning to AI is just a change of technique. It leaves the lawyer’s obligations exactly where they were, and it makes those obligations easier to breach.

The claim that AI will replace lawyers is half right. It will dismantle the junior law associate as a unit of cheap, typed drafting output, which is how many chambers still price juniors. It will not remove the need for a seasoned lawyer – an accountable human between the model and the court file, and Pakistani law provides no mechanism to make a machine accountable.

The fundamentals are legal, not sentimental. The profession’s core duties are written into the legal system’s structure. The precedent system depends on authenticity. Article 189 of the Constitution makes decisions of the Federal Constitutional Court and the Supreme Court of Pakistan on points of law binding on other courts, and Article 201 makes High Court decisions on points of law binding on subordinate courts. A cited authority that does not exist, or says something it does not, corrupts the machinery that makes those articles work. Article 10A, which guarantees a fair trial and due process, is undermined whenever a bench decides on invented law. Responsibility sits squarely with the human signatory.

Under Order VI Rule 15 of the Code of Civil Procedure, pleadings are verified by the party, but they are drafted and filed through counsel. Section 35-A empowers the court to award compensatory costs for false or vexatious claims and defences. The Legal Practitioners and Bar Councils Act of 1973 and the rules framed under it give Bar Councils disciplinary jurisdiction over professional misconduct. None of these provisions carves out an exception for “the software drafted it.”

Criminal exposure is narrower, and it is worth being precise about it. Sections 193 and 196 of the Pakistan Penal Code, on false evidence and using evidence known to be false, hinge entirely on intent. A lawyer who honestly files a hallucinated citation out of simple oversight may be negligent, but is not guilty of perjury. However, a lawyer who knows generative tools routinely invent cases, deliberately chooses not to verify them, and files them anyway crosses from mere negligence into reckless misrepresentation before the court.

Contempt under Article 204 remains a further risk if a bench concludes the court was knowingly misled. Hallucination bites harder here because Pakistani law reporting is fragmented across PLD, SCMR, CLC, YLR, PCrLJ, and others, and a model can produce a plausible-looking citation in any of those formats without any case behind it.

Statutes are amended and the amendments are unevenly covered in AI training datasets. The Phalia episode shows this in practice. In 2023, an Additional Sessions Judge in Phalia, Muhammad Amir Munir, experimentally consulted ChatGPT-4 while deciding pre-arrest bail for a 13-year-old accused, after the court had already decided to grant bail on the facts, as reported by Gulf News. According to news reports, the judge put 18 questions to the tool and it gave wrong answers twice, requiring him to manually correct it on the amended statutory age provisions connected to Section 83 of the Pakistan Penal Code. A judge who already knew the law caught the errors; a tired junior working from the tool’s output at 11pm before a morning cause list may not.

What the courts have actually said begins with the Supreme Court of Pakistan. In Ishfaq Ahmed v. Mushtaq Ahmed, decided on March 13, 2025 (C.P.L.A. No. 1010-L/2022, reported at 2026 SCLR 41, 2025 SCP 112), Justice Syed Mansoor Ali Shah and Justice Aqeel Ahmed Abbasi used a rent dispute to address AI in the judiciary. While these observations are framed as recommendations within a rent appeal – meaning they serve as persuasive guidance (obiter dicta) rather than a formal rule bound under Article 189 – they set a clear judicial direction.

The judgment treats AI as an auxiliary resource only, insists that identifiable humans retain ultimate responsibility, and describes hallucination as fabricated output produced by statistical pattern-matching rather than fact-checking, citing a study where GPT-4 fabricated legal references in 58% of cases.

It strongly recommended that national judicial bodies consider developing comprehensive guidelines on AI usage. National guidelines of the kind that judgment called for now exist. The National Judicial (Policy Making) Committee approved the National Guidelines for Use of Artificial Intelligence in Judicial Institutions of Pakistan at its 57th meeting in February 2026, published by the Law and Justice Commission.

Refined by the National Judicial Automation Committee chaired by Justice Muhammad Ali Mazhar, their core is human oversight. Under Clause 4.1, AI may assist judicial functions but not replace the judge’s final decision, requiring human validation checkpoints at which the judge records independent reasoning. Clause 4.2 requires users to identify portions of a document generated by AI and explain how that material was verified. Clause 4.3 protects confidential data, Clause 4.4 provides for training, and Clause 5.5 prefers locally developed systems trained on Pakistani legal data. Clause 5 outlines permitted uses including legal research, initial summaries, sample drafts, case management, and translation, while Clause 2 makes guidelines applicable across superior and subordinate courts and tribunals.

The gap is on the other side of the bar. While these guidelines govern judicial institutions, they do not directly bind advocates or Bar Councils. Yet the test they set for judges - to identify what the machine produced and explain how it was verified - is the very standard lawyers lack, even though counsel supply most of the authorities a court relies on. It is a ready-made template for a Bar Council rule. Comparative authority shows that courts sharing our common-law heritage are moving quickly. In Pooja Ramesh Singh v. J&K Bank (2026 INSC 668), decided July 2, 2026, India’s Supreme Court adopted zero tolerance for producing or citing unverified AI-generated precedents, asking the Bar Council to frame strict disciplinary norms. Because both jurisdictions share identical procedural and statutory foundations, its reasoning carries strong persuasive weight: citing fake legal authority without verification amounts to professional misconduct, and a judicial decision resting on hallucinated material is no decision at all.

The same reasoning appears in Mata v. Avianca in the United States, where attorneys were sanctioned for fake ChatGPT citations, and in the English Divisional Court decision Ayinde v. Haringey LBC (2025 EWHC 1383 Admin), which held that freely available generative AI tools cannot conduct reliable legal research. As technique changes, verification becomes the primary skill in legal practice. Every authority gets pulled from the actual report, through the Supreme Court’s website, High Court portals, or a verified subscription database. The lawyer must read the passage, confirm it supports the proposition, and confirm it has not been overruled or distinguished.

This was always the professional standard. It now has to be explicitly built into daily practice, because the temptation to skip it has never been greater. Evidentiary standards are shifting as well. Article 164 of the Qanun-e-Shahadat Order, 1984 and the Electronic Transactions Ordinance, 2002 recognise electronic records and modern-device evidence, but they were not drafted for synthetic media. A lawyer who cannot interrogate the provenance and integrity of an audio clip, screenshot or AI-enhanced document will inevitably lose to a counterpart who can.

Data confidentiality requires a concrete policy. Professional communications between advocate and client enjoy statutory privilege under the Qanun-e-Shahadat Order. Pasting a client’s facts or case details into a public AI tool risks breaching that privilege and compromising client trust. While the judiciary’s own guidelines under Clause 4.3 require litigants’ confidential data to be secured against unauthorised access, law chambers must meet the same ethical standard.

The duty arises directly from advocate-client privilege and legal ethics, regardless of separate data protection legislation. Chambers should establish written policies specifying which digital tools are permitted to touch client material. Local knowledge becomes the real asset. A model gives generic answers on criminal bail or commercial disputes. A lawyer who knows the Supreme Court’s approach from Tariq Bashir v. The State (PLD 1995 SC 34) onward, the “further inquiry” doctrine governing bail under Section 497(2) of the Code of Criminal Procedure, or how a specific bench treats severe offences, adds what an algorithm cannot. The same applies to leave-to-defend practice under Section 10 of the Financial Institutions Ordinance, 2001, or provincial land-revenue procedure.

The blunt reality regarding junior associates is that the tasks traditionally handed to entry-level lawyers are the ones language models perform fastest: drafting initial plaints, bail applications, legal notices, summary applications, chronologies, and routine research memos. Chambers billing clients primarily for those manual typing hours will face pressure, and the traditional volume of drafting-only junior positions will shrink.

The profession should state this honestly instead of offering vague reassurances. However, the associate role is undergoing an evolutionary shift rather than an extinction, for three reasons. First, legal accountability cannot be delegated to software. Courts, regulatory bodies, and clients require a human who can be questioned, trusted, and sanctioned. That human will increasingly be a junior associate whose primary responsibility evolves from typing first drafts to auditing machine outputs, checking original sources, and verifying facts.

Second, the legal apprenticeship pipeline must remain intact. Pupilage under a senior advocate is both a legal requirement for enrolment and an indispensable custom, because legal judgment is learned by doing routine work under correction. A chamber that automates away its junior lawyers today will face a severe shortage of qualified seniors in ten years. Smart chambers will redesign the apprenticeship so that juniors check AI output against original law reports, handle court appearances and client contact earlier, and participate directly in case strategy.

Third, language models are notoriously weak precisely where Pakistani practice is most dense: unreported judgments, registry requirements, bench-specific habits, and recent statutory amendments. The national guidelines’ preference under Clause 5.5 for locally developed systems trained on Pakistani legal data points the same way.

Associates who master localised, procedural nuances retain a decisive advantage over those who merely offer speed in basic drafting. AI has altered how a first draft is produced while leaving untouched who answers for the final product.

The Supreme Court of Pakistan and the National Judicial (Policy Making) Committee have affirmed that human judgment and accountability cannot be surrendered to a machine, while comparative common-law precedents demonstrate that submitting unverified AI authority constitutes misconduct. Lawyers should treat generative tools as fast, fallible assistants – never trusted unchecked and never permitted to sign a pleading.

Junior associates will not vanish, but the role of the associate as a high-volume typing engine is over. The practitioners who understand this early will be the ones who maintain their standing.

The article does not necessarily reflect the opinion of Business Recorder or its owners.