A real production AI-agent build, end to end — an agent that ingests federal solicitation PDFs and generates a structured, page-cited compliance matrix. FastAPI backend, PyMuPDF parsing, Pydantic schemas, PostgreSQL persistence, pgvector RAG, async Celery processing, and Docker/JWT deployment.
Ch 1: Why federal RFPs are structured, regulatory-dense documents perfectly suited for AI automation. Ch 2: Defining the exact V1 agent scope — the compliance matrix only, deliberately excluding proposal writing. Ch 3: Python/FastAPI environment setup and first API connectivity test. Ch 4: PDF parsing and text cleaning with PyMuPDF (fitz). Ch 5: Token-aware chunking strategy with overlap for 200+ page documents. Ch 6: Pydantic-validated requirement extraction engine and structured prompts. Ch 7: FAR/DFARS clause detection with a two-layer regex + AI design. Ch 8: Compliance matrix generation with CSV and DOCX export. Ch 9: Production system architecture — frontend, API, job queue, workers, database, storage. Ch 10: Full PostgreSQL schema — 11 tables with foreign keys and indexes. Ch 11: RAG with pgvector for clause-aware regulatory retrieval. Ch 12: Async processing with Celery, token budgeting, and cost guardrails. Ch 13: Docker containerization, managed hosting, JWT authentication, and the production security checklist.
Every code sample in this reader is real, working Python: FastAPI endpoints, Pydantic BaseModel schemas, regex clause detectors (FAR_PATTERN, DFARS_PATTERN), the full chunk-builder algorithm with overlap handling, and the complete PostgreSQL DDL for all 11 production tables (users, rfp_documents, rfp_files, pages, chunks, requirements, clauses, evaluations, processing_jobs, job_events, exports). Nothing is paraphrased into pseudocode — the snippets are shown verbatim so you can copy and adapt them directly.
The case study deliberately defines what the agent will NOT do: write full technical proposals, generate pricing, draft past performance narratives, or provide legal advice. That restraint is a business decision, not a technical limitation — it keeps the system objective, testable, and shippable as a V1, with proposal drafting explicitly deferred to a later phase. This is a pattern worth applying to any AI agent product: automate the structured, high-friction, rule-based slice first.
Rather than adding a separate vector database service, the case study uses pgvector directly inside PostgreSQL — storing FAR/DFARS clause summaries as 1536-dimension embeddings and retrieving the top-3 most similar summaries via the <-> distance operator whenever a clause reference is detected. This grounds clause interpretation in stored regulatory text instead of relying purely on the model's training data, and keeps the entire stack on one database engine.
It ingests a federal solicitation (RFP) PDF — typically 100 to 300+ pages — and automatically produces a structured, page-cited compliance matrix: every mandatory "shall/must" requirement, every referenced FAR and DFARS clause, and a submission checklist, exported as CSV and DOCX. It deliberately does not draft proposal text, generate pricing, or give legal advice; it is a document intelligence system, not a proposal writer.
FastAPI for the backend API, PyMuPDF (fitz) for PDF text extraction, Pydantic for structured schema validation of LLM output, the OpenAI API (gpt-4o-mini for extraction, text-embedding-3-small for embeddings), PostgreSQL for persistence with the pgvector extension for RAG, Celery + Redis for asynchronous background job processing, and Docker for containerized deployment with JWT-based authentication.
Federal RFPs commonly run 80 to 300+ pages, which would exceed practical token limits, spike API cost, and lose page-level traceability if sent in one request. The system splits each document into 800-1,200 token chunks with 50-100 token overlap (to avoid cutting a requirement mid-sentence at a chunk boundary), each tagged with its source page range, chunk ID, and section hint — so every extracted requirement can be traced back to an exact page.
A two-layer design. Layer 1 is fast, cheap regex matching against standardized clause number formats (FAR 52.xxx-xx, DFARS 252.xxx-xxxx) — required for the MVP because these formats are highly consistent. Layer 2 is an optional AI confirmation step that looks at nearby text to extract the clause title when present, with an explicit instruction never to invent a title if one is not clearly stated in the source text.
Detecting a clause number alone (e.g., FAR 52.204-21) does not tell the system what the clause requires. The RAG layer stores FAR/DFARS clause summaries as embeddings in a pgvector-enabled PostgreSQL table, and when a clause is detected, the system embeds the query, retrieves the top-3 most similar stored summaries via vector distance, and injects that regulatory context into the compliance output — grounding the result in version-controlled, deterministic reference text instead of the model's unguided interpretation.