Most "give your agent memory" tutorials pick one tool and stretch it past its limits. Either the whole conversation gets stuffed back into the prompt every turn until costs and latency spiral, or every message gets embedded into a vector store, which is overkill for something as simple as "what did the user say two messages ago."

The fix isn't a smarter model. It's splitting memory into two tiers that do genuinely different jobs — one for what just happened, one for what's worth remembering.

The three ways agent memory breaks

Before the architecture, it's worth being specific about what actually goes wrong:

Stateless amnesia. The agent only sees the current prompt, so it forgets tool calls, stated preferences, and reasoning from three messages ago — inside the same session.

History stuffing. The lazy fix: append the full raw transcript to every call. It works until it doesn't — token costs climb, latency climbs, and long contexts degrade retrieval quality even when the model technically supports the window size.

Cross-session disconnect. The user comes back tomorrow and the agent has no idea who they are or what was decided last time. Everything gets re-explained from scratch.

Each of these has a different fix, and that's the whole argument for two tiers instead of one.

The architecture: fast and dumb, slow and smart

Tier 1 is a Redis sliding-window buffer — the last handful of exchanges, stored verbatim, gone after a day. Tier 2 is Postgres with pgvector — facts and preferences extracted out of the conversation, embedded, and kept permanently. A request pulls from both in parallel, merges them into a compact context block, and only after the response goes out does a background job decide what from this exchange is worth saving long-term.

User Request ──► NestJS Agent Controller
                       │
       ┌───────────────┴───────────────┐
       ▼                               ▼
Tier 1: Redis Buffer          Tier 2: pgvector RAG
- Last 6-10 messages          - Relevant past memories
- Immediate tool state        - User preferences & facts
       │                               │
       └───────────────┬───────────────┘
                       ▼
             Compact Context Prompt
                       │
                       ▼
                  LLM / Agent
                       │
                       ▼
                Agent Response
                       │
        (async, after response)
                       ▼
       Fact Extraction & pgvector Upsert

The extraction step running after the response matters more than it looks. Do it inline and you're paying for an extra embedding call on every single turn, most of which don't contain anything worth remembering. Push it to a background job and the user never waits on it.

Tier 1: the Redis buffer

This tier doesn't need to be clever. It needs to be fast and it needs to expire on its own.

// src/memory/redis-memory.service.ts
import { Injectable } from '@nestjs/common';
import Redis from 'ioredis';

export interface ChatMessage {
  role: 'user' | 'assistant' | 'system' | 'tool';
  content: string;
  timestamp: number;
}

@Injectable()
export class RedisMemoryService {
  private readonly redis = new Redis(process.env.REDIS_URL!);
  private readonly maxWindowSize = 8; // last 8 exchanges
  private readonly ttlSeconds = 60 * 60 * 24; // 24-hour session TTL

  async getRecentHistory(sessionId: string): Promise<ChatMessage[]> {
    const rawMessages = await this.redis.lrange(
      `agent:session:${sessionId}`,
      0,
      this.maxWindowSize - 1,
    );
    return rawMessages.map((msg) => JSON.parse(msg)).reverse();
  }

  async appendMessage(sessionId: string, message: ChatMessage): Promise<void> {
    const key = `agent:session:${sessionId}`;
    await this.redis.lpush(key, JSON.stringify(message));
    await this.redis.ltrim(key, 0, this.maxWindowSize - 1);
    await this.redis.expire(key, this.ttlSeconds);
  }
}

lpush + ltrim on every write keeps the list capped at 8 entries without a separate cleanup job, and expire resets the 24-hour TTL each time someone's active. Eight exchanges is a starting point, not a law — bump it if your agent leans on tool-call chains that run longer than that, but every extra message in this buffer is raw text you're paying to re-send on the next turn, so don't inflate it just in case.

Tier 2: pgvector for the stuff worth keeping

This is where facts survive past the session — user preferences, decisions, anything that should still be true next week.

// src/memory/long-term-memory.service.ts
import { Injectable } from '@nestjs/common';
import { Pool } from 'pg';
import { GoogleGenAI } from '@google/genai';

@Injectable()
export class LongTermMemoryService {
  private readonly pool = new Pool({ connectionString: process.env.DATABASE_URL });
  private readonly ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

  async retrieveRelevantMemories(userId: string, query: string, topK = 3): Promise<string[]> {
    const embeddingResponse = await this.ai.models.embedContent({
      model: 'text-embedding-004',
      contents: [{ text: query }],
    });
    const embedding = embeddingResponse.embedding.values;

    const querySql = `
      SELECT memory_text, (embedding <=> $1::vector) AS distance
      FROM agent_long_term_memories
      WHERE user_id = $2
      ORDER BY distance ASC
      LIMIT $3;
    `;
    const result = await this.pool.query(querySql, [JSON.stringify(embedding), userId, topK]);
    return result.rows.map((row) => row.memory_text);
  }

  async storeFact(userId: string, factText: string): Promise<void> {
    const embeddingResponse = await this.ai.models.embedContent({
      model: 'text-embedding-004',
      contents: [{ text: factText }],
    });
    const embedding = embeddingResponse.embedding.values;

    await this.pool.query(
      `INSERT INTO agent_long_term_memories (user_id, memory_text, embedding) VALUES ($1, $2, $3::vector)`,
      [userId, factText, JSON.stringify(embedding)],
    );
  }
}

text-embedding-004 gives you 768-dimension vectors, and the <=> operator does cosine distance directly in the query — ascending order means closest match first. One thing worth calling out that's easy to miss: this only stays fast if agent_long_term_memories has an actual vector index (ivfflat or hnsw) on the embedding column. Without one, Postgres falls back to a sequential scan the moment the table has any real volume in it, and the 10-25ms latency figure quietly turns into something much worse.

Wiring the two tiers together

The orchestrator's only job is fetching both tiers in parallel and handing the model a context block that fits.

// src/agent/agent-orchestrator.service.ts
@Injectable()
export class AgentOrchestratorService {
  constructor(
    private readonly redisMemory: RedisMemoryService,
    private readonly longTermMemory: LongTermMemoryService,
  ) {}

  async buildAugmentedPrompt(
    userId: string,
    sessionId: string,
    currentPrompt: string,
  ): Promise<string> {
    const [recentExchanges, pastFacts] = await Promise.all([
      this.redisMemory.getRecentHistory(sessionId),
      this.longTermMemory.retrieveRelevantMemories(userId, currentPrompt),
    ]);

    const memoryBlock = pastFacts.length
      ? `\nRELEVANT USER FACTS (Long-Term Memory):\n${pastFacts.map((f) => `- ${f}`).join('\n')}\n`
      : '';

    const historyBlock = recentExchanges.length
      ? `\nRECENT CONVERSATION:\n${recentExchanges.map((m) => `${m.role.toUpperCase()}: ${m.content}`).join('\n')}\n`
      : '';

    return `${memoryBlock}${historyBlock}\nCURRENT USER INPUT: ${currentPrompt}`;
  }
}

Promise.all matters here — Redis and Postgres are independent reads, and there's no reason to pay their latencies sequentially. Run them together and the slower of the two (pgvector, at 10-25ms) is what the request waits on, not the sum of both.

Which tier does what

The archival tier is worth a mention even though there's no code for it above — if you're in a regulated space, you'll want full transcripts somewhere durable and cheap, separate from both the working buffer and the semantic index. Neither Redis nor pgvector is the right home for "keep everything forever, query almost never."

Where this actually gets used

Redis's TTL handles the boring cleanup that a naive implementation would need a cron job for — sessions just expire. But the tier split only pays off if the fact extraction step is disciplined. If every message gets pushed straight into pgvector, you haven't built long-term memory, you've built a slower, more expensive Redis. The extraction job needs to actually decide what's durable — a stated preference, a decision, a correction — and skip the small talk.

That's also the piece most tutorials leave as an exercise for the reader. In practice it's a small LLM call of its own: pass the session's messages, ask for a short list of facts worth persisting, and only call storeFact for what comes back. It's an extra API call per session instead of per turn, which is exactly the tradeoff this whole architecture is built around.