This isn't a system deployed for a client — it's a design exercise I document to show how I reason through the trade-offs when architecting production AI infrastructure.
The reference scenario: an internal support copilot that answers with a citation over runbooks, tickets, and logs. I use that concrete case to tie together six pieces that usually get discussed in isolation: data lake, metadata catalog (DataHub, Amundsen, Unity Catalog), hybrid search (OpenSearch, Elasticsearch), vector database, AI gateway, and LLM-as-judge evaluation.
How the pieces fit
- Data lake (S3). Raw-to-curated storage — bronze/silver/gold — the source of truth every index can be rebuilt from.
- Catalog — DataHub, Amundsen, Unity Catalog. Business metadata: who owns what, where it came from, and its lineage. DataHub is event-driven (Kafka) with column-level lineage; Amundsen is pull-based and simpler to run; Unity Catalog is the native pick when the lake sits on Databricks/Delta.
- Hybrid search — OpenSearch / Elasticsearch. Same technical lineage, different license (Apache-2.0 vs. SSPL). One cluster serves log dashboards and, via k-NN, semantic retrieval.
- Vector database. Embeddings + ANN search — "similar in meaning," not in keywords. On AWS: OpenSearch's k-NN, Aurora + pgvector, or a dedicated engine (Pinecone/Weaviate).
- AI gateway. One front door for every LLM call: auth, rate-limiting, per-team cost, provider failover, semantic caching, and guardrails — owned once, not reimplemented per service.
- LLM-as-judge. A model scores another model's output against a rubric — faithfulness, relevance, safety — offline as a regression gate, online sampling live traffic.
Why this matters to me
I come from three years building anti-fraud platforms for banks — detection, auditable evidence, and AI that has to justify its answer. This exercise translates that same judgment — traceability before speed — into the vocabulary of a modern data platform, applicable to internal support just as much as to fraud investigation and agent automation over operational data.