LangSmith_and_LLM_Evaluation_Session 1.pptx

LLM Tracking

1. LangSmith & LLM Evaluation

2. Why Evaluate LLMs?

3. What is LLM Evaluation?

4. Understanding Evaluation Dimensions

5. How LLM Evaluation Is Done Methods: Human Evaluation Automated Metrics (BLEU, F1, etc.) LLM-as-a-Judge Pairwise Comparison Task-based Evaluation

6. Challenges in Evaluation

7. Introducing LangSmith

8. LangSmith Core Components

9. LangSmith Evaluation Flow

10. Types of Evaluators

11. Other Evaluation Features

12. Common Metrics in LangSmith

13. When to Use Which

14. LangSmith Workflow Dataset → Run → Evaluator → Metric → Dashboard

15. Advanced Use Cases

16. Best Practices

17. Summary

18. Q&A