LLM Evals Lesson 2 Error Analysis
Notes from lesson 2 of Hamel and Shreya's LLM evaluation course - covering error analysis, open and axial coding, and systematic approaches to understanding where AI systems fail.
Artificial intelligence
Hamel & Shreya's LLM Evals Course: Lesson 1
Notes from the first lesson of Parlance Lab's Maven course on evaluating LLM applications - covering the Three Gulfs model and why eval is where most people get stuck.
Synthesising a new framework for AI Transformation
I like bits of Brunig's and Mollick's AI frameworks, but neither quite works for me.
Error Analysis for Improving LLM Applications
A systematic approach to analysing and improving large language model applications through error analysis.
Why we need Experiment-based Roadmaps in the AI Product Era
Why evaluation-driven experimentation creates better roadmaps in AI products.
The challenges of mastering LLMs, and their role as cyborg enhancement
Simon Willison was a guest on Logan Kilpatrick's Google podcast. Topics covered: AI as a 'cyborg enhancement', the non-intuitive challenges of mastering LLM use, and the legitimate need for uncensored language models in fields like journalism.