AI/ML Infra Meetup | Preference Tuning and Fine Tuning LLMs

AI/ML Infra Meetup
Aug. 29, 2024
Organized by Alluxio

For more Alluxio Events: https://www.alluxio.io/events/

Speaker:
- Ankit Khare (Developer Relations, @OpenAI)

This session aims to provide practical insights for AI enthusiasts on effectively customizing and leveraging LLMs in various applications through preference tuning and fine-tuning.

1. Fine-Tuning & Preference Tuning
LLMs

Ankit Khare
Developer Relations, OpenAI

2. Overview
•Mental Model of the landscape of optimizing LLMs
•Intuition on which methods to use when
•You should leave with some level of confidence that
LLM optimization is indeed hard.
•But, no pain no gain folks!!!

3. False Belief on optimization
•Process is linear X
•Process is non linear

4. Actual Flow

5. Steps in the flow

6. Prompt Engineering

7. Example

8. Mental Model - RAG and FT

9. RAG Intuition

10. RAG Eval

11. Fine Tuning Intuition

12. FT Success Story

13. Fun FT Story

14. Fun FT Story

15. Fun FT Story

16. FT Best Practices

17. FT + RAG

18. Preference Tuning

19. Preference Tuning

20. Preference Tuning

21. RLHF

22. DPO

23. Explore More!

24. Slide credits and references
●Huggingface
https://www.youtube.com/watch
?v=QXVCqtAZAn4

●OpenAI
https://platform.openai.com/docs
/guides/fine-tuning

25. Thank you!

Feel free to connect!