Problem. Support structured report generation, itinerary planning, and grounded conversational Q&A over user documents — with citations, multi-turn follow-ups, and production-grade ingestion.
Architecture. Stateful LangGraph graphs per workflow; Pinecone as the managed retrieval layer; a transactional-outbox ingestion pipeline drained by Inngest workers; LangSmith tracing across every run.
Documents→Outbox→Inngest workers→Pinecone→LangGraph agents→Cited answers
Key decisions
Stateful LangGraph architecture over a simpler sequential chain
Multi-turn follow-ups reuse prior retrieval state instead of re-retrieving each turn, and every response stays grounded in per-run source chunks.
Managed Pinecone over self-hosting a vector store
Retrieval scales as a managed service; operating a vector DB added risk without differentiating the product.
Transactional outbox + Inngest workers for ingestion over fire-and-forget async processing
Idempotent, retry-safe document processing — validated with documents up to 50 pages.
Outcome. A production RAG platform where every answer carries citations to source documents, with observable, replayable ingestion and agent runs.
Problem. Take grounded chat over Markdown and PDF documents beyond a single-script prototype, with citations, streaming, conversation history, corpus controls, and a repeatable security-focused Azure deployment.
Architecture. Azure Blob Storage feeds an Azure AI Search indexing pipeline with chunking and integrated embeddings; FastAPI exposes an Agent Framework AG-UI stream; API Management fronts a private backend; Next.js and CopilotKit provide the user interface; Cosmos DB stores per-user discussion history.
Blob Storage→AI Search indexer→Hybrid retrieval→FastAPI agent→APIM→CopilotKit UI
Key decisions
Hybrid semantic retrieval with integrated vectorization over application-managed embeddings and vector-only search
Azure AI Search owns index-time and query-time vectorization, then combines lexical and vector matching with semantic reranking for stronger grounding context.
AG-UI streaming through Agent Framework over a custom token-streaming protocol
A standard run and message event model connects the FastAPI agent to the CopilotKit interface while keeping the browser-facing runtime simple.
Managed identities and private backend ingress over application API keys and a public backend
APIM validates the caller, applies quotas, and reaches the internal API using managed identity; service access follows least-privilege RBAC.
Outcome. A tested, documented Azure RAG reference with grounded answers and citations, corpus management, persistent discussions, readiness checks, and repeatable Bicep deployment workflows.
