Q26: Agent Memory Architecture: Short-Term Scratchpads, Semantic Long-Term & Episodic Memory - AI Engineering & LLMs (50 Qs)

1. The Multi-Tiered Memory Hierarchy for AI Agents

Human cognitive systems maintain sensory memory, working memory, and long-term memory. Enterprise AI agents adopt a parallel multi-tiered memory architecture:

java

┌────────────────────────────────────────────────────────────────────────┐
│                        Agent Memory Hierarchy                          │
└────────────────────────────────────────────────────────────────────────┘
  1. Working Memory (Ephemeral Scratchpad)
     • Current prompt, active thought-action loop, immediate tool output.
     • Stored in active context window; discarded upon task completion.
        │
        ├── Syncs key entities & state changes
        ▼
  2. Episodic Memory (Experience & History)
     • Past user interaction logs, previous task outcomes, error post-mortems.
     • Stored in relational / document DB (PostgreSQL, MongoDB).
        │
        ├── Embeds semantic concepts
        ▼
  3. Semantic Long-Term Memory (Knowledge Base)
     • Vector database (Pinecone, Qdrant) storing user preferences, corporate rules,
       and historical solutions across months of interactions.

2. Memory Tier Comparison

Memory Tier	Storage Medium	Eviction Policy	Latency	Example Stored Data
Working (Short-Term)	LLM Context Window (RAM)	Cleared at end of session / FIFO	0ms	Current sub-task tool outputs, active code snippet
Episodic	Relational DB (Postgres/Redis)	LRU or time-based TTL	< 10ms	"User rejected PR #42 yesterday due to missing unit tests"
Semantic (Long-Term)	Vector Store + Embeddings	Cosine similarity top-$K$ lookup	20–50ms	"User prefers TypeScript over JavaScript and strict null checks"

3. Production Entity Extraction for Memory Updates

python

async def extract_and_store_user_preferences(llm, user_chat: str, memory_db):
    prompt = f"""Extract any durable user preferences, tech stack constraints, or 
architectural rules stated in this chat. Output JSON array of strings only.
Chat: {user_chat}"""

    facts = await llm.generate_json(prompt)
    for fact in facts:
        # Upsert into semantic vector store for future session recall
        embedding = await embed_text(fact)
        await memory_db.upsert(vector=embedding, metadata={"fact": fact, "user_id": "usr_123"})