LLM Evals Course Lesson 6: Complex Pipelines and CI/CD
Notes from lesson 6 of Hamel and Shreya's LLM evaluation course - debugging agentic systems, handling complex data modalities, and implementing CI/CD for production LLM applications.
Evals
LLM Evals Course Lesson 5: How to Evaluate Complex Architectures
Notes from lesson 5 of Hamel and Shreya's LLM evaluation course - evaluating retrieval quality, generation quality, and common pitfalls in RAG systems.
LLM Evals Course Lesson 4: Multi-turn and Collaborative Evaluation
Notes from lesson 4 of Hamel and Shreya's LLM evaluation course - handling multi-turn conversations and building evaluation criteria through collaboration.
LLM Evals Course Lesson 3: Building Automated Evaluators
Notes from lesson 3 of Hamel and Shreya's LLM evaluation course - implementing automated evaluators, building reliable LLM-as-judge systems, and avoiding common pitfalls.
LLM Evals Course: Lesson 2b (office hrs)
A few things from Evals Course office hrs following lesson 2 of Hamel and Shreya's LLM evaluation course.
LLM Evals Lesson 2 Error Analysis
Notes from lesson 2 of Hamel and Shreya's LLM evaluation course - covering error analysis, open and axial coding, and systematic approaches to understanding where AI systems fail.