AI Engineering, LLM Architecture & Agentic Systems Master (50 Qs)

RAG vs. In-Context Learning vs. Fine-Tuning: Architectural Trade-Offs & Decision Matrix

Document Chunking Strategies: Fixed-Size, Recursive, Semantic & Late Chunking

Embedding Models & Distance Metrics: Cosine Similarity, Dot Product & Euclidean Distance

Vector Database Indexing Internals: HNSW vs. IVF-PQ Trade-Offs

Cross-Encoder Re-Ranking: Architecture, Late Interaction (ColBERT) & Two-Stage Retrieval

Hybrid Search (Dense + Sparse) & Reciprocal Rank Fusion (RRF)

Advanced Query Transformation: HyDE (Hypothetical Document Embeddings) & Multi-Query Decomposition

Context Window Stuffing & The Lost-in-the-Middle Phenomenon

Automated RAG Evaluation: The RAGAS Triad & Synthetic Benchmark Generation

Multi-Tenant Vector DB Architecture: Tenant Isolation, Pre-Filtering & Partition Keys

Autoregressive Generation & The KV Cache: Memory Footprint & Scaling Formulas

PagedAttention & vLLM Architecture: Eliminating KV Cache Memory Fragmentation

Continuous Batching (Iteration-Level Scheduling) vs. Static Batching

Prompt Caching & Prefix Caching: Anthropic, Gemini & vLLM Internal Mechanics

Speculative Decoding: Small Draft Models, Verification & Latency Reduction

Tokenization Mechanics: Byte-Pair Encoding (BPE), Token Healing & Multi-Lingual Compression

Model Quantization Techniques: PTQ vs. QAT, FP16, INT8, INT4, AWQ, GPTQ & GGUF

FlashAttention 1, 2 & 3: Memory Tiling & Eliminating the O(N²) HBM Bottleneck

Streaming Architecture: Server-Sent Events (SSE), WebSockets & The Vercel AI SDK Protocol

LLM SLA Metrics & Capacity Planning: TTFT, ITL, TPS & Concurrency Modeling

The ReAct Pattern (Reason + Act): Thought-Action-Observation Loop & Production Trade-Offs

Structured Outputs & Function Calling: Constrained Decoding (CFG Grammars) vs. Pydantic

Model Context Protocol (MCP): Client-Host-Server Architecture & Dynamic Tool Discovery

Multi-Agent Orchestration & State Machines: LangGraph vs. CrewAI & Cyclical Execution

Plan-and-Solve / Least-to-Most Prompting: Decoupling Planning from Execution

Agent Memory Architecture: Short-Term Scratchpads, Semantic Long-Term & Episodic Memory

Tool Execution Resilience: Timeout Budgets, Exponential Backoff & Circuit Breakers

Human-in-the-Loop (HITL) Workflows: Checkpoints, State Serialization & Resumption Tokens

Autonomous Agent Guardrails: Runaway Loop Prevention, Quotas & Recursion Limits

Code Execution Sandboxes for Coding Agents: gVisor, Firecracker & Docker Isolation

Direct vs. Indirect Prompt Injection: Attack Vectors & Dual-LLM Defense Architecture

Insecure Output Handling: XSS, SQLi & Remote Code Execution from Untrusted Model Outputs

System Prompt Leakage & Extraction: Canary Tokens, Instruction Hierarchy & Reverse Prompting

Supply Chain & Model Weight Security: SafeTensors vs. Pickle RCE & Malicious LoRA Weights

Model Denial of Service (DoS): Context Exhaustion, Recursive Tool Bombs & Sponge Attacks

Overreliance & Hallucination Mitigation: Grounding Citations, Self-Consistency & Confidence Scoring

PII Redaction & Data Sanitization: Presidio, Token Replacement & Differential Privacy

Guardrails Architecture: NeMo Guardrails, Llama Guard & Programmable Safety Rails

Excessive Agency & Privilege Escalation in Autonomous Agents (OWASP LLM08)

LLM Jailbreaking Vectors: GCG Attacks, Many-Shot Exploits & Adversarial Robustness

Parameter-Efficient Fine-Tuning (PEFT) & LoRA: Low-Rank Adaptation Mechanics

QLoRA (Quantized Low-Rank Adaptation): 4-bit NormalFloat (NF4) & Paged Optimizers

RLHF vs. DPO (Direct Preference Optimization): Alignment Mechanics & Loss Formulations

Synthetic Data Generation for Training: Self-Instruct, Evol-Instruct & Quality Filtering

Model Evaluation Benchmarks: MMLU, GSM8K, HumanEval, SWE-bench & Their Failure Modes

LLM-as-a-Judge Design: Mitigating Position, Verbosity & Self-Enhancement Biases

Domain Adaptation & Continued Pre-Training vs. Catastrophic Forgetting

Serving Open-Weights Models: vLLM vs. TGI vs. Ollama & Kubernetes GPU Deployments

GPU Memory Sizing & Sizing Formulas: Weights, KV Cache, Activations & AdamW States

Enterprise AI Gateway Architecture: Semantic Caching, Fallback Routing & Rate Limiting