Four real, worked diagnostic scenarios in AI engineering — a RAG chatbot confidently wrong on covered questions, an LLM integration degrading under real traffic, an agent stuck in a tool-calling loop, and a fine-tuned model that fails in production despite a strong eval score.
This module applies the same structured Symptoms → Diagnosis → Testing Steps → Resolution discipline this program uses for any real fault: record the symptom precisely, form a diagnosis consistent with the evidence, run a specific test that confirms or rules it out, then apply the fix that follows from a confirmed cause — not a guess. Each scenario is built around a failure that looks like one thing on the surface and is actually something else underneath: a "generation" failure that's really a retrieval failure, a "reasoning" failure that's really an ambiguous tool contract, and an eval score that never predicted production because the two data distributions quietly diverged.
By the end of this module you should be able to trace each scenario's root cause back to the earlier module it draws on — Module 9's RAG mechanics, Module 12's deployment concerns, Module 7 and 8's agent and tool-calling behavior, and Module 2's overfitting concept — and, more importantly, know which specific piece of evidence to check first before touching a prompt, a model, or a line of code.