LLM Evals Course: Complete Course Recap
This is my recap of Hamel and Shreya's LLM evaluation course. I'm hoping I come back here in the future every time I need to remind myself of how to do this the right way.
Machine learning
LLM Evals Lesson 8: Improving LLM Products
Notes from the final lesson of Hamel and Shreya's LLM evaluation course - practical strategies for improving accuracy and reducing costs through prompt refinement, architecture changes, fine-tuning, and model cascades.
LLM Evals Course Lesson 6: Complex Pipelines and CI/CD
Notes from lesson 6 of Hamel and Shreya's LLM evaluation course - debugging agentic systems, handling complex data modalities, and implementing CI/CD for production LLM applications.
LLM Evals Course Lesson 4: Multi-turn and Collaborative Evaluation
Notes from lesson 4 of Hamel and Shreya's LLM evaluation course - handling multi-turn conversations and building evaluation criteria through collaboration.
LLM Evals Course Lesson 3: Building Automated Evaluators
Notes from lesson 3 of Hamel and Shreya's LLM evaluation course - implementing automated evaluators, building reliable LLM-as-judge systems, and avoiding common pitfalls.
LLM Evals Course: Lesson 2b (office hrs)
A few things from Evals Course office hrs following lesson 2 of Hamel and Shreya's LLM evaluation course.