Using openly licensed Large Language Models (LLMs) to develop services - experiences, insights and speculations

Mindtrek 29 views 12 slides Oct 10, 2024
Slide 1
Slide 1 of 12
Slide 1
1
Slide 2
2
Slide 3
3
Slide 4
4
Slide 5
5
Slide 6
6
Slide 7
7
Slide 8
8
Slide 9
9
Slide 10
10
Slide 11
11
Slide 12
12

About This Presentation

By: Tarmo Toikkanen, Senior Lead, Data and AI, The Finnish Innovation Fund Sitra

Presentation as part of the track program: The Future of Open Source Business

Mindtrek Conference
OpenTech: From the community for the community.
8th of October, 2024 | Tampere, Finland
www.mindtrek.org 


Slide Content

#mindtrek #mindtrek Using openly licensed Large Language Models (LLMs) to develop services - experiences, insights and speculations Tarmo Toikkanen Senior Lead , Data and AI Sitra 2024-10-08, MindTrek , Tampere

1 2 3 Basics Types of automated decision making 1. Rule-based automation 2. Traditional AI 3. Generative AI Manually crafted by engineers Learns from high quality data. Requires some computation . Machine learning Neural networks Learns from massive amounts of mediocre data. Requires super computers . Generative Pretrained Transformers ( GPTs ) if city==Tampere then attend ( MindTrek )

pilots Two 1. Understanding previous work related to a new EU act ~60k€, 3mo Ministry of Transport and Communication Futurice Interactive chat UI FinGPT 3 2. Summarising public consultation results ~60k€, 3mo Prime Minister’s Office SiloGen AI Batch process , no UI Poro LLM

Lessons learned How to train your AI 1 You need access to high quality , managed and up -to- date data 2 You need comprehensive and high quality examples for fine-tuning , and lots of them 3 You need to customize your c hoice of model size and training / fine-tuning methods to suit your case 4 You need to understand the limitations of your chosen LLM

Lessons learned Results and next steps 1 2 3 4 Current open LLMs are not capable enough Larger context windows needed There is potential in AI We have a roadmap to get to truly functional specialised AI tools . Both closed and open models have their place Commercial models are often the most capable . But often you don’t need the best . Sometimes you can’t use a commercial non-EU service . Processes and culture should also adapt Adjusting organisational processes to be more AI friendly .

Quality and price AI performance 6

Transparency Strategic autonomy Cost-effectiveness Confidentiality Open or closed Why use open LLMs or SLMs ?

1 2 3 Step by step Creating a gen -AI service 1. Get a foundation model 2. Prepare your service related data 3. Train your AI Buy a service or Download an open model or Train your own model with data Usually 100s or 1000s of examples of labeled input-output pairs . NB. RAG is not a silver bullet , as quality of content indexing is crucial for quality . Reinforcement learning phase with your own data RAG: Retrieval-Augmented Generation

Plug and play? AI architecture 9

New Sitra Agile experiments We ideate and experiment with solutions . We build partnerships . Scaling up solutions We scale up solutions in collaboration with our partners . We also work internationally . Foresight and the operative landscape We identify societal innovation needs and the pressing challenges that need to be addressed . We consider both short-term and long- term perspectives . Our operating model

Contact More info Tarmo Toikkanen Senior Lead , Data and AI [email protected] +358 294 618 511

Päätöskalvo sitra.fi @ sitrafund
Tags