How We integrated TencentDB-Agent-Memory into the SRE K8s Agent

I run the SRE K8s Agent, an AI chatbot built for GreenNode's SRE team. There was one glaring problem: every time someone opened a new tab, the agent basically started from scratch. Users had to keep re-explaining context - which cluster they were watching, what the issue was, and what had already been discussed.

The problem to solve: the agent "forgets" every time you open a new tab

GreenNode's SRE K8s Agent can query metrics, read logs, analyze node health, and run automated runbooks across VKS clusters, running on qwen/qwen3-5-27b via the MaS API. Capability-wise, it's more than capable of answering complex operational questions.

But an LLM's reasoning power doesn't solve one basic problem: the agent has no long-term memory between sessions.

For example, a user might have already told the agent: cluster prod-01 has a memory leak in the payments namespace.

But open a new tab, and that entire context is gone. The user has to re-supply the cluster, the namespace, and the issue all over again.

This is a very real friction point in day-to-day operations work. Users retype "cluster prod-01 has a memory leak issue in the payments namespace" — information the agent already knew yesterday, simply because they opened a new tab.

The question isn't new, but the answer isn't obvious either: is there a way for the agent to remember conversation history across sessions without stuffing the entire chat history into the prompt — which costs tokens and adds latency?

What is TencentDB-Agent-Memory?

TencentDB-Agent-Memory (17.6k stars on GitHub) is a team-level memory hub for AI agents. Instead of storing raw chat logs and re-injecting them into the prompt the way typical RAG does, it distills conversations, documents, and code into four reusable memory asset types: Chat Memory (conversation history, preferences, facts, and decisions across sessions), Skill (reusable procedures like troubleshooting, code review, release checklists), LLM-Wiki (documents, design specs, and runbooks organized into structured pages with a link graph), and CodeGraph (an index of code symbols, files, call relationships, and impact paths).

The core difference from flat RAG lies in a four-layer architecture, with increasing levels of abstraction from raw text to a long-term profile:

By design, the L1→L2→L3 pipeline runs asynchronously and doesn't block the response stream. L1 fires after an idle period, then L2 and L3 continue building context at increasingly higher levels of abstraction. In practice, however, when deployed against Vietnamese-language conversations, this pipeline didn't work as expected — more on that in the gotchas section below.

On effectiveness: the PersonaMem benchmark, which measures how well an agent understands and applies user information across multiple interactions, scored 48% without memory versus 76% with TencentDB-Agent-Memory — a +59% improvement.

Deployment architecture on a GreenNode VM

The entire SRE Agent runs on a single VM, with all services containerized and connected through a Docker bridge network called sre-net. Containers talk to each other by container name as hostname — no static IPs needed. sre-backend calls tdai-memory at http://tdai-memory:3701; Docker's built-in DNS resolves the container name on the same bridge network. Port 3701 is only exposed to 127.0.0.1 on the host — it's never public.

Four endpoints are actually used in production:

The most notable row in that table is the third one: the endpoint that was designed to be the package's flagship feature is the one endpoint excluded from the production flow - the reason is covered in the gotchas section below.

6 steps the AI agent goes through to retrieve and store memory on every request

Every conversation turn goes through four main phases, with the capture phase running in the background so it never blocks the response stream:

1. POST /api/v1/agent-ops/chat - the frontend sends {message, history, session_id}
2. POST /search/conversations - sre-backend runs a BM25 search to fetch related memory (3s timeout)
3. Inject into the system prompt - if there are results, they're appended under a "Related conversation history" section
4. Streaming chat completion - the MaaS LLM is called with the augmented system prompt
5. Render the response - the SSE token stream renders live in the UI
6. POST /capture (background thread) - stores user_content and assistant_content into L0 without blocking the response

The flow can be summarized as: User → Search Memory → Inject Context → LLM → Stream Response → Capture Conversation. Capture runs in the background specifically so it never affects the agent's response stream.

Implementation: the actual code

Step 1: deploy the tdai-memory container

# Create the network if it doesn't exist yet
docker network create sre-net

docker run -d \
  --name tdai-memory \
  --network sre-net \
  -p 127.0.0.1:3701:3701 \
  -v /opt/tdai-data:/data/tdai-memory \
  -e TDAI_LLM_BASE_URL=https://maas-llm-aiplatform-hcm.api.vngcloud.vn/v1 \
  -e TDAI_LLM_API_KEY=<maas-api-key> \
  -e TDAI_LLM_MODEL=qwen/qwen3-5-27b \
  tencentdb-agent-memory:latest

The gateway looks for its config at ./tdai-gateway.yaml (CWD = /opt/tdai-gateway/), not TDAI_DATA_DIR. This is the single most important gotcha — put the file in the wrong place and the config is silently ignored, with no error message at all.

Step 2: tdai-gateway.yaml — the configuration that actually matters

memory:
  recall:
    strategy: keyword   # keyword=BM25; hybrid=requires an EmbeddingService (not used)
    maxResults: 5
    scoreThreshold: 0.1
  extraction:
    enabled: true
    maxMemoriesPerSession: 20
    pipeline:
      everyNConversations: 3
      l1IdleTimeoutSeconds: 30   # reduced from the 600s default
      l2DelayAfterL1Seconds: 30
      l2MinIntervalSeconds: 60
  bm25:
    enabled: true
    language: en   # en, because Vietnamese BM25 isn't well optimized yet

Step 3: Python helper functions — using only the urllib stdlib, no added dependencies:

_TDAI_URL = "http://tdai-memory:3701"

def _tdai_search(query: str, session_key: str, limit: int = 3) -> str:
    try:
        data = json.dumps({
            "query": query, "limit": limit, "session_key": session_key
        }).encode()
        req = urllib.request.Request(
            f"{_TDAI_URL}/search/conversations",
            data=data, headers={"Content-Type": "application/json"}, method="POST"
        )
        with urllib.request.urlopen(req, timeout=3) as resp:
            return json.loads(resp.read().decode()).get("results", "")
    except Exception as e:
        log.debug(f"[tdai] search: {e}")
        return ""

def _tdai_capture_bg(user_content: str, assistant_content: str, session_key: str):
    try:
        data = json.dumps({
            "user_content": user_content,
            "assistant_content": assistant_content,
            "session_key": session_key,
        }).encode()
        req = urllib.request.Request(
            f"{_TDAI_URL}/capture",
            data=data, headers={"Content-Type": "application/json"}, method="POST"
        )
        with urllib.request.urlopen(req, timeout=5) as resp:
            resp.read()
    except Exception as e:
        log.debug(f"[tdai] capture: {e}")

Step 4: wiring it into the agent_chat endpoint

# Search memory before calling the LLM
_mem_context = await loop.run_in_executor(None, _tdai_search, message, session_id)
if _mem_context:
    system_prompt += f"\n\n## Related conversation history\n{_mem_context}"
    log.info(f"[tdai] injected len={len(_mem_context)} session={session_id!r}")

# Capture asynchronously once the agent is done (background thread)
finally:
    if _response_parts:
        threading.Thread(
            target=_tdai_capture_bg,
            args=(message, "".join(_response_parts), session_id),
            daemon=True,
        ).start()

Results in practice

After deployment, sre-backend logs confirmed memory injection and capture were both working as designed:

[tdai] injected memory context len=342 session="prod-session-01"
[tdai] captured session="prod-session-01"

The measured numbers: ~3ms memory search latency (BM25 runs locally, no network round trip), HTTP 200 on every /capture response, 0 added dependencies beyond the urllib stdlib, and capture runs on a background thread so it never affects the response stream.

The result: the agent now has "memory" — a very tangible UX improvement for day-to-day production K8s operations. Backend logs confirm [tdai] injected memory context and [tdai] captured on every request. But the path to that result wasn't a straight line — the six gotchas below were the most time-consuming part.

6 real-world issues integrating TencentDB-Agent-Memory

Gotcha #3 is the one worth dwelling on, because it isn't a configuration mistake — it's a real limitation of the pipeline: L1 distillation runs for 48 seconds, burns through 977 input tokens, and produces only 77 output tokens — with an empty resulting memory list. In other words, the L1→L2→L3 pipeline simply doesn't work with Vietnamese-language conversations in practice. Production memory recall today is nothing more than BM25 keyword matching over raw L0 storage — not the semantic understanding the package was originally designed to deliver. This is a pragmatic compromise: the system still has context, just not real "understanding." A proper fix would require customizing the extraction prompt or switching to a stronger model for Vietnamese.

What's next

• Mount settings.json via a Docker volume so it isn't lost on container rebuilds
• Customize the L1 extraction prompt for Vietnamese and the SRE domain (cluster names, pod names, K8s terminology)
• Try an embedding strategy with a local embedding model for real semantic recall instead of BM25 keyword-only matching
• Implement memory TTL — automatically purge conversations older than 30 days so L0 doesn't grow unbounded
• Add per-cluster memory namespaces so recall can be filtered to a specific VKS cluster
• Evaluate TencentDB-Agent-Memory v2.0+ and its full Memory Hub — with Skill, Wiki, CodeGraph, and team sharing support

Conclusion

The end result isn't the "perfect memory" the package's name promises — it's an acceptable compromise: BM25 keyword search over raw conversation, just enough so the agent is no longer a blank page every time someone opens a new tab, at essentially zero cost — 3ms latency, no added dependencies, no impact on the response stream. The multi-layer LLM distillation behind it is still there as a future direction, not a feature that's actually running today. For a production system operating in Vietnamese, the broader lesson might be this: read the logs carefully before trusting the documentation, and be willing to ship the simplest layer that actually works instead of the most sophisticated one that's advertised.