Large Language Models for Legal Reasoning and Legislation Support

The first part of the presentation presents the state of knowledge of what we know about the ability of large language models to engage in ‘authentic’ legal reasoning. The narrative will cover relevant technical basics, benchmarks, and the role of so-called ‘thinking models’, leading up to experimental results on a synthetic case-based reasoning task and dataset, where even thinking models produce mixed results. The second part of the talk reports on the state of the art and proof-of-concept implementations using LLMs for legislation support, specifically the drafting of legislative amendments from natural language policy objectives in tax law, and the automatic consolidation of statute versions. The concluding discussion will feature lessons learned about the challenges faced with working with publicly available data cutting across different source types towards effectively supporting government roles with AI technology.

Matthias Grabmair is a tenure-track Assistant Professor of Legal Tech in the Department of Informatics at the Technical University of Munich. After a German state exam in law and an American LLM he researched and taught at Carnegie Mellon University’s language Technologies Institute until 2019. He then spent a year as a Legal Data Scientist at SINC before joining the CIT School at TUM in 2021. He is also an academic co-director of the Legal Tech Colab. His research focuses on how natural language processing on legal text, knowledge representation, and computational models of argumentation can be leveraged to support and transform legal practice, in particular the judicial and legislative branches of government.