EVA (RAG Assistant on AWS)
EVA is the RAG (Retrieval-Augmented Generation) assistant living in the bottom-right corner of the portfolio, answering questions about me. Full stack on AWS: embeds the question with Amazon Titan, retrieves relevant fragments of my CV, experience and projects from DynamoDB, and passes them to Claude Haiku 4.5 as context. Designed prioritizing cost control and honesty about retrieval limits (when it does not know, it says so instead of making things up).
Tech Stack
Architecture
# Two-stage retrieval: bi-encoder for recall, cross-encoder for precision
def rerank(query, candidates, top_k=3):
if not candidates:
return []
if not RERANK_ENABLED:
return candidates[:top_k]
model = _load_model()
pairs = [(query, c["text"]) for c in candidates]
scores = model.predict(pairs)
for c, s in zip(candidates, scores):
c["rerank_score"] = float(s)
# DELIBERATE: c["score"] stays as bi-encoder cosine.
# Downstream gates reason in BGE scale.
candidates.sort(key=lambda x: x["rerank_score"], reverse=True)
return candidates[:top_k]Context
I wanted a chatbot that answered specific questions about my CV, experience and projects from the portfolio. The logic from taken decisions was: a LLM would invent biographical details, fine-tuning is expensive and can't be updated without retraining every time my info changes. A KB fits in Haiku's context window, but prompt-stuffing pays for the full context on every query even when the user only asks for my contact.
Goal
Build an assistant that answers accurately about my real information, is honest when it doesn't know something, and has a predictable cost ceiling even when exposed as a public endpoint. FinOps discipline mattered as much as the response provided.
Approach
RAG pipeline with Titan Embeddings + DynamoDB for retrieval, Claude Haiku 4.5 on Bedrock for generation. Added contextual retrieval (metadata embedded) that lifted top-hit scores from 0.30 to 0.66. Then added LangGraph orchestration: routing by question type, conversation memory with gated coref resolution, query reformulation in the weak band, and an eval suite that verifies agent decisions. Last cycle added a second reranking stage with the MS-MARCO cross-encoder that closed the eval's known limitation. Everything sits behind 4-layer FinOps controls: relevance gate, input length cap, API Gateway rate limiting, and staggered budget alarms. Deployed via Terraform, no static credentials thanks to GitHub Actions OIDC.
Impact
Eval suite allows to make immediate tests to check quality responses. Grounded answers over real portfolio information, with the system honestly declining when the KB doesn't cover something in 99% overall. Real cost under $1/month in production and infrastructure reproducible from zero with a single terraform apply, and any repo reviewer can run the full system over a synthetic persona without touching my real data.