Using openly licensed Large Language Models (LLMs) to develop services - experiences, insights and speculations

By: Tarmo Toikkanen, Senior Lead, Data and AI, The Finnish Innovation Fund Sitra

Presentation as part of the track program: The Future of Open Source Business

Mindtrek Conference
OpenTech: From the community for the community.
8th of October, 2024 | Tampere, Finland
www.mindtrek.org 

1. #mindtrek #mindtrek Using openly licensed Large Language Models (LLMs) to develop services - experiences, insights and speculations Tarmo Toikkanen Senior Lead , Data and AI Sitra 2024-10-08, MindTrek , Tampere

2. 1 2 3 Basics Types of automated decision making 1. Rule-based automation 2. Traditional AI 3. Generative AI Manually crafted by engineers Learns from high quality data. Requires some computation . Machine learning Neural networks Learns from massive amounts of mediocre data. Requires super computers . Generative Pretrained Transformers ( GPTs ) if city==Tampere then attend ( MindTrek )

3. pilots Two 1. Understanding previous work related to a new EU act ~60k€, 3mo Ministry of Transport and Communication Futurice Interactive chat UI FinGPT 3 2. Summarising public consultation results ~60k€, 3mo Prime Minister’s Office SiloGen AI Batch process , no UI Poro LLM

4. Lessons learned How to train your AI 1 You need access to high quality , managed and up -to- date data 2 You need comprehensive and high quality examples for fine-tuning , and lots of them 3 You need to customize your c hoice of model size and training / fine-tuning methods to suit your case 4 You need to understand the limitations of your chosen LLM

5. Lessons learned Results and next steps 1 2 3 4 Current open LLMs are not capable enough Larger context windows needed There is potential in AI We have a roadmap to get to truly functional specialised AI tools . Both closed and open models have their place Commercial models are often the most capable . But often you don’t need the best . Sometimes you can’t use a commercial non-EU service . Processes and culture should also adapt Adjusting organisational processes to be more AI friendly .

6. Quality and price AI performance 6

7. Transparency Strategic autonomy Cost-effectiveness Confidentiality Open or closed Why use open LLMs or SLMs ?

8. 1 2 3 Step by step Creating a gen -AI service 1. Get a foundation model 2. Prepare your service related data 3. Train your AI Buy a service or Download an open model or Train your own model with data Usually 100s or 1000s of examples of labeled input-output pairs . NB. RAG is not a silver bullet , as quality of content indexing is crucial for quality . Reinforcement learning phase with your own data RAG: Retrieval-Augmented Generation

9. Plug and play? AI architecture 9

10. New Sitra Agile experiments We ideate and experiment with solutions . We build partnerships . Scaling up solutions We scale up solutions in collaboration with our partners . We also work internationally . Foresight and the operative landscape We identify societal innovation needs and the pressing challenges that need to be addressed . We consider both short-term and long- term perspectives . Our operating model

11. Contact More info Tarmo Toikkanen Senior Lead , Data and AI [email protected] [phone]

12. Päätöskalvo sitra.fi @ sitrafund