Google AI Hackathon: LLM based Evaluator for RAG

Slides accompanying project submission video for Google AI Hackathon. Describes a LCEL and DSPy based evaluation framework inspired by the RAGAS project.

Accompanying video URL: https://youtu.be/yOIU65chc98

1. LLM based Evaluator for RAG Mayank Bhaskar Dave Campbell Sujit Pal

2. question context answer ground truth Retriever LLM

3. question context answer ground truth Retriever LLM Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall

4. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall JSON Snapshot LLM

5. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall LCEL JSON Snapshot LLM

6. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall LCEL JSON Snapshot LLM Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall DSPy predicted scores

7. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall LCEL JSON Snapshot LLM Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall DSPy

8. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall LCEL JSON Snapshot LLM Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall DSPy Manual Evaluation Tool

9. Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall LCEL JSON Snapshot LLM Faithfulness Answer Relevance Context Precision Context Utilization Context Relevance Answer Correctness Answer Similarity Context Recall DSPy Manual Evaluation Tool Synthetic Data Predictive Models

10. https:// github.com / sujitpal / llm -rag-eval 🙏