An engineering-focused guide to designing, building, deploying, and scaling production-grade AI agents — treating agents as controlled software components, not experimental chat interfaces. Covers prompts as programs, memory systems, tool calling, planning and reasoning loops, deterministic vs. creative model configuration, error handling, and a full production agent build.
Ch 1: What is an AI agent — agents vs chatbots vs automation, the Sense-Think-Act loop, and the five core components (goal, reasoning engine, memory, tools, control logic). Ch 2: LLMs as reasoning engines — tokens, cost, latency, and deterministic vs. creative configuration using temperature and top_p. Ch 3: Agent architecture patterns — event-driven design, tool-calling architecture, stateless vs. stateful agents. Ch 4: Prompts as programs — system vs. user prompts, templates, structured JSON outputs, guardrails. Ch 5: Memory systems — short-term vs. long-term memory, vector databases, and RAG. Ch 6: Tools and function calling — tool interfaces, safe database/file/code-execution patterns, the tool router. Ch 7: Planning techniques — ReAct, Chain-of-Thought, Tree of Thought, and Planner-Executor. Ch 8: Error handling and self-correction — retries, confidence scoring, fallback logic, human-in-the-loop escalation. Ch 9: A full production agent build with a Python orchestrator, tool router, and memory class. Ch 10: Deployment, scaling, and cost control — containers, APIs, webhooks, caching, and token optimization.
The book's central engineering claim: prompts must be treated as programmable artifacts — structured, versioned, and testable, not informal text. A system prompt defines identity and global rules with the highest precedence; a user prompt is untrusted, variable input that can never override system rules. Structured JSON outputs are validated in code, not trusted as raw text, and guardrails must exist at both the prompt level (which only influences behavior) and the code level (which actually enforces it).
A recurring engineering pattern: deterministic settings (temperature 0.0, top_p 1.0) for tool calls and structured outputs where predictability and low hallucination risk matter; creative settings (temperature 0.7, top_p 0.9) for brainstorming, summarization, and planning where diverse ideation is valuable. Production agents often switch configuration across phases of a single run — higher creativity during planning, high determinism during execution, moderate creativity for final output — and that switching is controlled by code, never left to the model.
Among the planning techniques covered (ReAct, Chain-of-Thought, Tree of Thought), the book identifies Planner-Executor as the preferred pattern for production systems: an LLM planner breaks a goal into an ordered list of steps, then a code-driven executor runs each step deterministically against registered tools. This separation delivers predictability, easier debugging, and safer execution compared to interleaved reasoning-and-action loops.
Traditional automation is deterministic and rule-based (if X happens, do Y) with no memory, no tools beyond what is hardwired, and no planning — it breaks when conditions change. A chatbot is LLM-powered but stateless by default, following a simple prompt-response pattern with no long-term memory and no task persistence. An AI agent is stateful and goal-driven: it maintains memory, has access to tools, plans multi-step actions, and self-corrects. The formal definition used throughout the guide: an agent perceives inputs, maintains internal state and memory, reasons using an LLM, takes actions via tools, and iterates until a goal is achieved or terminated.
Use deterministic settings (temperature 0.0, top_p 1.0) whenever the agent is calling tools, generating structured JSON, or performing a repeatable task — this produces predictable outputs, makes testing easier, and lowers hallucination risk. Use creative settings (temperature around 0.7, top_p around 0.9) for brainstorming, summarization, planning, and exploratory reasoning, where diverse responses are valuable. Production agents frequently switch between these configurations across different phases of the same run (higher creativity while planning, high determinism while executing tools), with the switching logic implemented in code rather than left to the model.
Because in production, prompts function like configuration files, interface contracts, and policy definitions — they define the agent's role, operating constraints, decision boundaries, and output format. Treating them as code means keeping them explicit, version-controlled (stored as files, changes tracked and reviewed), and testable, so you can verify output format correctness, tool-call consistency, and safety-rule adherence before deployment. A prompt that cannot be tested should not be deployed.
Memory is what turns a reactive system into a persistent, goal-driven one. Short-term memory holds recent context for the current task (bounded, in-memory, ephemeral) while long-term memory persists durable knowledge across sessions, typically in a database or vector store. Retrieval-Augmented Generation (RAG) combines external knowledge retrieval with LLM generation: a query is embedded, matched against a vector database for the most semantically similar chunks, and those chunks are injected into the prompt so the model answers using grounded, retrieved information rather than fabricating an answer. RAG is most valuable when data is large, changes frequently, or accuracy is critical.
The guide categorizes failures into LLM errors (hallucinated facts, invalid JSON, wrong tool selection), tool errors (API timeouts, permission denials), and system errors (memory overflow, network failures), each requiring a different response. Bounded retries with exponential backoff handle transient failures, but invalid inputs, authorization failures, and safety violations should never be retried. Confidence scoring — computed in code from signals like schema validation success and retry agreement, not guessed by the LLM — determines whether to proceed, trigger a fallback (a simpler prompt, an alternate tool, a cached response), or escalate to a human when confidence is low, errors repeat, or financial/legal impact exists.