Langid with ChatGPT benchmark comarision with fasttext.pdf

LangID with LLM

1. ChatGPT: Jack of all trades,
master of none
Paper Discussion





03/2023
Evgeny BAZAROV, [email protected]

2. arXiv:2302.10724v1 [cs.CL] 21 Feb 2023

5. Let’s go to paper

6. Small After Party
Language identification with Chat-GPT

7. ChatGPT API (gpt-3.5-turbo)
BESEDO

8. Old Benchmark

9. 1.Filter data (1 < n_chars < 15)
2.Sample 10k (stratified) - should be ok to be representative
3.Create prompt
4.Run API with gpt-3.5-turbo
5.Postprocess results
6.Benchmark fasttext vs chatgpt
Steps

10. Prompting
V1: "You are an API to a Language Identification service. I will give you text as
input, you will return to me the ISO 639-1 code of a language in which text was
written. Your response should be in json format."

11. Creativity?

12. Prompting
V2:

13. Result

14. Result

15. Result
5-7 points better than fasttext