From Promise to Practice. Implementing AI in Legal Environments

Pim Betist's presentation about generative AI in the legal sector at Lexpo.com '24

1. From Promise to Practice Implementing AI in Legal Environments Lexpo ’24

2. Source: https://www.artificiallawyer.com/2024/05/28/the-primary-purpose-of-legal-tech-is/ What is the primary purpose of legal tech? 6% 20% 62% d) Reduce cost of legal services c) Give lawyers more convenience b) Provide jobs for ex-lawyers ;-) a) Help lawyers make more money 12% e)To create value for your clients

3. “Don’t miss the boat”

4. Scale Research Experiment

5. $200B Source: https://www.goldmansachs.com/intelligence/pages/ai-investment-forecast-to-approach-200-billion-globally-by-2025.html Investment in AI

6. $20B https://www.economist.com/business/2024/03/17/just-how-rich-are-businesses-getting-in-the-ai-gold-rush Growth in revenue

7. $2400B Growth in Market Cap Tech Titans Bron: https://www.accel.com/noteworthy/accel-2023-euroscape-generation-ai

8. $2400B/$20B=120 Multiple ???

9. Are we having a déjà vu?

10. Are we having a déjà vu?

11. Initial b oom and u nrealistic e xpectations with e-commerce B2C to B2B Shift Infrastructure i nvestment Market c rash ( and r ecovery ) A brief history of the dot-com bubble

12. https://www.morningstar.hk/hk/news/243779/nvidia-2023-vs-cisco-1999-will-history-repeat.aspx Is NVIDIA the Cisco of 2024?

13. Wishful thinking or future reality? Bron : https:// www.gspublishing.com /content/research/ en /reports/2023/03/27/d64e052b-0f6e-45d7-967b-d7be35fabd16.html

14. March 2023

15. Feb ’23 results skewed by repeat test takers Source: https:// link.springer.com /article/10.1007/s10506-024-09396-9 Januari 30 2024

16. If you use it there is trouble The Catch-22 of AI If you don’t it will be double

17. Bringing more legal tasks in-house has generated immediate benefits in terms of cost (60-80%), quality, engagement and speed (25%) Luis-Xavier Hernandez, Unilever

18. “ While technology will not replace lawyers and notaries, the profile will shift towards tech-savvy professionals .“

19. “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.” ― Edsger W. Dijkstra in Texas, 1984

20. Understanding AI Key features and limitations What is RAG and what can we do with it? How is AI being adopted by the legal sector? What can we learn from early adopters ? How can you create value with AI ? What we’ll be covering

21. Understanding AI Rule based Generative Machine learning Recognition AI LLMs

22. LLMs in a nutshell 175 billion artificial neurons (GPT-3.5) 45 terabytes of training data (GPT-3.5) Black box Results are generated based on statistics “Autocomplete on steroids”

23. Dog Meaning Form

24. External Internal Legal research legislation, case law, literature Case data, best practices, precedents Unstructured proprietary data and content Data Knowledge Value AI Legal advice, documents, litigation Creating value with AI in legal

25. Key features Semantic search Text generation (drafting) Text analysis (reviewing) Data extraction Clustering (predictions)

26. Drafting & review Generate text Legal research Formulation Red lining Translation Summarise Contract management Data extraction Workflows Negotiate Legal AI assistent Formulating questions based on the facts Legal research Memo writing with link to sources Litigation Predicting outcomes Structuring documents File search Timeline of events Key applications

27. Key players

28. https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html Key limitations

29. Stanford RegLab & Institute for Human-Centered AI published on 2 Jan 2024 Hallucinations from legal prompts

30. Retrieval Augmented Generation RAG to the rescue! Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

31. Retrieval Augmented Generation RAG to the rescue! Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

32. Retrieval Augmented Generation RAG to the rescue! Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

33. Retrieval Augmented Generation RAG to the rescue! Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

34. RAG, turns out, is no panacea New Stanford study reveals hallucinations and incompleteness Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

35. RAG, turns out, is no panacea What to check when using AI for legal research Groundedness : do key factual propositions make valid references to relevant legal documents? Correctness : is the response factually correct and relevant? Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

36. There is almost never one clear cut answer to legal prompts Document relevance is not based on text alone An LLM can’t synthesize facts while keeping the appropriate legal context in mind RAG limitations Source: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf

37. We are not 68%, but 90% accurate Thomson Reuters’ response https:// www.artificiallawyer.com /2024/06/10/thomson-reuters-contradicts-stanford-genai-study-we-are-90-accurate/ ‘ AI-Assisted Research uses large language models and can occasionally produce inaccuracies , so it should always be used as part of a research process in connection with additional research to fully understand the nuance of the issues and further improve accuracy. ‘We also advise our customers, both in the product and in training, to use AI-Assisted Research to accelerate thorough research, but not to use it as a replacement for thorough research.’

38. Meanwhile, AI is making its way into our workplace Whether we like it or not https://reutersinstitute.politics.ox.ac.uk/sites/default/files/2024-05/Fletcher_and_Nielsen_Generative_AI_and_News_Audiences.pdf

39. Company guidelines for using free LLMs Paid subscription to LLM for (all) employees Private server with general purpose LLM or subscription to Legal LLM with data protection Narrow purpose legal AI assistance or replacement of humans General purpose legal AI assistance The 5 Adoption Levels of Generative AI In Legal Sleepers Explorers Adopters Innovators Leaders

40. Make or buy? 202 4 2020 Building blocks Custom software

41. “Programming is not the same as writing code.” 8 lawyers + office manager Built an in-house platform using no-code solutions Contains reviewing, drafting and transcription features LLM/AI provider agnostic Enables prompt/ knowledge sharing

42. “..the smartest generalist frontier models beat specialized models in specialized topics” Ethan Mollick Associate Professor at The Wharton School. Author of Co-Intelligence

43. Use cases that are safe to do Summaries Translations Transcriptions and next steps Text generation (drafting) First step of legal research With generative AI provided your data is secure

44. AI doesn't always make your results better "The jagged frontier of AI capabilities" AI will make your results better AI makes no difference AI makes your results worse

45. Cyborg Centaur Source: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged

46. 5 ways to create value with AI Educate your team! Invest in knowledge management Consistency in content and lay out is key 80/20 is a universal rule. Find it in your organization and start there. Try things out and expect no more than 10-20% efficiency for now

47. Download this presentation and subscribe on clayrichard.eu Thank you for listening