AI could break legal systems by finding previously hidden loopholes
If AI can find the holes in programming code, it can do the same with legal code, paving the way for chaos. Felix Ringe, a visiting fellow at Stanford Law School’s liftlab and legal scholar, suggests large language models (LLMs) have become potent enough that they could break legal systems by identifying exploitable inconsistencies at scale. "Legal systems encode the foundations of social order in natural language," Ringe and collaborator Linus Folkerts argued in a blog post last week. "Similar to computer systems, the written law can contain vulnerabilities that invite exploitation." Ringe contends that while courts tend to pave over these text vulnerabilities through statutory interpretation, the ability to discover adversarially useful inconsistencies in the law in an automated way risks overwhelming the legal system’s ability to bridge the gaps. To buttress his thesis, he points to recent work by other legal scholars that frames laws as vulnerable code, such as " Legal Zero-Days : A Novel Risk Vector for Advanced AI Systems" and " Hardware and Software Bugs in the Legal System " [PDF]. The analogy dates back at least to 1999, with Lawrence Lessig’s Code and Other Laws of Cyberspace [PDF] and Code Is Law: On Liberty in Cyberspace . Lessig credits William J. Mitchell, in City of Bits: Space, Place, and the Infobahn (1995), with the idea that code is cyberspace’s law, adding that he "drew the metaphor from others as well." Ringe has turned this on its head by proposing that law is code. The analogy between programming code and legal code works reasonably well, up to a point. Law and code both have their share of formatting pedantry and ritual. The mapping breaks down when courts get cast as a runtime environment – non-deterministic trial outcomes don’t work for production applications. Discontinuities aside, Ringe’s point is that AI models can surface statutory gaps at a scale that may bring the chaos currently seen in the software security ecosystem to the legal world. To demonstrate that, Ringe and Folkerts cite several potential gaps in German law that they uncovered with the help of LLMs, noting that they’re only sharing legal exploits they believe could not cause serious harm. For example, an amendment to Germany’s Occupational Diseases Regulation (BKV) is subject to a constitutional requirement to cite its statutory authority. But Ringe and Folkerts argue that the amendment omits a required reference…