Related Pages

Related topics: Key Concepts

Getting Started

TraceBase is an institutional memory layer for AI agents that stores, retrieves, and injects past debugging experiences into new sessions. The Getting Started guide walks you through installing the CLI, linking your first project, running diagnostics, and verifying that memory injection is working correctly.

Prerequisites

Before you begin, ensure your environment meets the following requirements:

Sources: www/src/components/dashboard/OverviewView.tsx

Installation

The TraceBase CLI is distributed as an npm package and can be initialized in any project directory.

npx tracebase-ai init

Running this command in a project directory creates the necessary configuration files and links the project into your TraceBase workspace.

Sources: www/src/components/dashboard/OverviewView.tsx:12

Project Initialization Flow

graph TD
    A[npx tracebase-ai init] --> B{Check project root}
    B -->|Valid project| C[Create .tracebase config]
    B -->|No project| D[Error: run in project dir]
    C --> E[Create instruction block in CLAUDE.md/AGENTS.md]
    E --> F[Configure agent hooks]
    F --> G[Verify with tracebase-ai doctor]
    G --> H{All checks pass?}
    H -->|Yes| I[Project linked successfully]
    H -->|No| J[Review doctor output]
    J --> K[Fix issues]
    K --> G

What `init` Configures

The initialization process sets up several components:

1. Project Configuration — Creates .tracebase/config.json with project metadata
2. Instruction Block — Appends a managed section to CLAUDE.md or AGENTS.md that defines the injection protocol
3. Agent Hooks — Configures the appropriate hooks for your agent (e.g., Claude Code hooks via .claude/settings.json)

Sources: src/cli/commands/init.ts Sources: src/cli/commands/doctor.ts:1-30

Diagnostic Check

After initialization, run the diagnostic command to verify your setup:

tracebase-ai doctor

The doctor command performs the following checks:

Instruction File Checks

Sources: src/cli/commands/doctor.ts:20-35

Hook Health Checks

Hook health inspection is specific to Claude Code and is surfaced as a separate check from MCP configuration because the two surfaces are independent. MCP can be perfectly configured while .claude/settings.json has no hook entries, silently degrading the default UX where users lose silent injection and background capture, falling back to the foreground MCP permission prompt.

Sources: src/cli/commands/doctor.ts:45-60

Project Linking

Once initialized, projects appear in the TraceBase dashboard under Installations. Each linked project displays:

• Project Name — The directory name or configured project identifier
• Agent — Which agent is configured (e.g., Claude Code)
• CLI Version — Installed TraceBase CLI version if available
• Timestamps — When the project was linked and last updated

interface Installation {
  id: string;
  projectName: string;
  agent: string;
  cliVersion?: string;
  createdAt: Date;
  updatedAt: Date;
}

Sources: www/src/components/dashboard/InstallationsView.tsx:1-40

Verification Steps

1. Confirm Installation Appears in Dashboard

After running npx tracebase-ai init, navigate to your TraceBase workspace dashboard. The linked project should appear in the Installations list within a few seconds.

2. Run Agent Session with Memory

Start a new agent session in the linked project. When the agent encounters a task, TraceBase:

1. Checks the memory knowledge base for relevant past experiences
2. Retrieves matching blocks (situation, mechanism, dead ends, unlock)
3. Injects high-confidence matches into the agent context

3. Verify Memory Capture

After any session that touched memory, run the dashboard to see the activity log. Successful memory operations appear as Used events in the task funnel.

Sources: www/src/components/dashboard/ImpactView.tsx:10-50

Troubleshooting

Sources: src/cli/commands/doctor.ts:15-25

Next Steps

After completing the Getting Started guide:

1. Connect repositories — Link GitHub repos to pull in issues, PRs, and CI failures for the Engineering Brain
2. Review the whitepaper — Understand the evaluation methodology and benchmark results
3. Configure custom integrations — If using a custom agent runtime, explore the integration options

Related Pages

Related topics: Getting Started, System Architecture

Key Concepts

TraceBase is a memory layer for coding agents that preserves past solutions, file meaning, and doom-loop patterns across runs. It operates as a drop-in MCP (Model Context Protocol) server that enables agents to retain and retrieve institutional knowledge.

Architecture Overview

TraceBase employs a two-stage retrieval architecture with six distinct signals and learned ranking weights.

graph TD
    A[Agent Query] --> B[Stage 1: Fingerprint + BM25]
    B --> C[Candidate Set]
    C --> D[Stage 2: Re-ranking]
    D --> E[Structural Signal]
    D --> F[Cosine Signal]
    D --> G[Freshness Signal]
    E --> H[Thompson Sampling Ranker]
    F --> H
    G --> H
    H --> I[Final Block Injection]

Retrieval Signals:

Weights are learned from outcomes via Thompson sampling. Sources: www/src/app/whitepaper/page.tsx:48-58

Core Data Model: Blocks

Blocks are the fundamental memory units in TraceBase. Each block encapsulates a learned solution pattern with structured metadata.

Block Structure

graph LR
    A[Block] --> B[Trigger]
    A --> C[Body]
    B --> D[situation]
    B --> E[invariants]
    C --> F[mechanism]
    C --> G[deadEnds]
    C --> H[unlock]
    C --> I[verification]
    C --> J[guardrails]

Trigger (situation): The context or problem scenario that activates this memory block.

Invariants: Conditions that must hold true for the block to be relevant.

Body Components:

Prose Rendering

When a block is injected into agent context, it renders as compact prose:

• <Situation, capitalized>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>.

Dead ends append an Avoid: clause when present. Sources: src/core/build-injection-payload.ts:20-38

Imported Blocks

Blocks can be marked as imported with a special XML tag wrapper:

<prior_fix source="imported">…</prior_fix>

This distinction is used for blocks extracted from external knowledge sources versus local agent experience. Sources: src/core/build-injection-payload.ts:12-15

Multi-Round Methodology

TraceBase implements a compound-intelligence effect through iterative knowledge accumulation.

graph LR
    A[Round 0] -->|Empty KB| B[Agent Solves Tasks]
    B -->|Successful patches| C[Traces Created]
    C --> D[Round 1]
    D -->|Warm KB| E[Agent Solves Same Tasks]
    E --> F[Improved Accuracy]

Round 0: Baseline execution with empty knowledge base. Successful solutions become traces.

Round 1: Same tasks executed with populated knowledge base. Both rounds use identical step caps, cost caps, and Docker images—the only variable is KB state.

This simulates production compound-intelligence as agents accumulate institutional knowledge. Sources: www/src/app/whitepaper/page.tsx:105-120

Security: Guardrails

TraceBase implements multiple security patterns to prevent injection attacks and spoofing.

Detection Patterns

The system-spoof pattern includes backtick-neighbour skip logic: documented tags wrapped in inline backticks (e.g., "the <system> block marker") are treated as documentation, not spoofed markers. Sources: src/core/guard.ts:25-40

Guardrail Metadata

Blocks can contain guardrails arrays that constrain their applicability. These are validated during distillation:

if (filtered.length > 0) guardrails = filtered;

Guardrails must be non-empty strings. Sources: src/distillation/llm-distiller.ts:18-24

Agent Integration

MCP Hook Configuration

TraceBase integrates with agent hooks (Claude Code, MCP) for seamless operation. The hook inspection system verifies:

1. MCP server configuration
2. .claude/settings.json hook entries
3. Managed events for each agent type

Missing Stop hooks are a critical regression—MCP can be configured correctly while hooks silently fail. Sources: src/cli/commands/doctor.ts:35-52

CLI Initialization

Projects initialize TraceBase with:

npx tracebase-ai init

This creates the instruction block and configures agent hooks. The doctor command validates installation:

✓ managed section present
✓ hook-events present

Installation Tracking

Installations are tracked with:

Sources: www/src/components/dashboard/InstallationsView.tsx:28-45

Block Serving: XML Rendering

For audit and debugging purposes, blocks can be rendered as structured XML:

<hypothesis id="block-123" calibrated="0.850">
  <situation>Token handling in nested contexts</situation>
  <mechanism>...</mechanism>
  <dead_ends>
    <item>Approach A</item>
    <item>Approach B</item>
  </dead_ends>
  <unlock>...</unlock>
  <verification>...</verification>
  <evidence trace="trace-id-456" role="citation"/>
</hypothesis>

The calibrated attribute shows the probability score, and evidence elements reference source traces. Sources: src/core/block-serving.ts:28-53

Evaluation Framework

TraceBase is evaluated on SWE-bench Verified using mini-swe-agent v2 in Docker containers.

Metrics:

Benchmark results show +25% accuracy improvement and up to 39% token savings on Claude Sonnet 4.6. Sources: www/src/app/whitepaper/page.tsx:70-95

Distillation Pipeline

The LLM distiller extracts blocks from agent sessions with confidence scoring:

Validation Rules:

The extractor uses balanced-brace scanning for nested JSON objects rather than regex, accommodating model-generated nested structures. Sources: src/distillation/llm-distiller.ts:42-60

Impact Funnel

TraceBase tracks memory usage through a funnel:

graph TD
    A[Eligible Runs] --> B[Recalled Runs]
    B --> C[Injected Runs]
    C --> D[Used Runs]
    D --> E[Task Win]

Funnel Stages:

Sources: www/src/components/dashboard/ImpactView.tsx:30-50

Key Design Principles

1. Positive constraints over negative framing — "the bug is X, fix: Y" rather than "do not try A, B, C"
2. Skip-to-fix when prior knowledge is available — plan-and-act, not explore-first
3. Compression for injection — traces stored as three short fields (situation, deadEnds, unlock) for efficient context addition
4. Compound intelligence — patterns that work for one team raise match quality for everyone

Related Pages

Related topics: Block Store, Retrieval System

System Architecture

TraceBase is a compound-intelligence knowledge base system designed to improve AI agent performance by storing, retrieving, and injecting institutional knowledge into agent contexts. The system compounds over time: patterns that work for one team's agents raise match quality for everyone running the same shapes of work.

Overview

TraceBase operates on a retrieval-augmented principle where successful agent solutions are captured as traces and later re-injected into similar future tasks. This creates a compounding effect where the knowledge base grows more valuable as more agents contribute solutions.

The architecture follows a multi-stage pipeline:

graph TD
    A[Agent Session] -->|Completes Task| B[Trace Capture]
    B -->|Stores| C[Knowledge Base]
    C -->|Retrieval| D[Two-Stage Ranker]
    D -->|Fingerprint + BM25| E[Candidate Set]
    E -->|4 Re-rank Signals| F[Scored Blocks]
    F -->|Build Payload| G[Injection Block]
    G -->|Security Guard| H[Validated Payload]
    H -->|Context Injection| I[Future Agent Session]

Key Components:

Retrieval Architecture

Two-Stage Ranking

TraceBase implements a two-stage retrieval ranker for efficiency and accuracy:

Stage 1 - Candidate Narrowing:

• Fingerprint matching: O(1) lookup for exact/similar patterns
• BM25 via FTS5: Full-text search to narrow candidates

Stage 2 - Re-ranking: Four additional signals re-rank the candidate set:

graph LR
    A[Candidates] --> B[Structural Signal]
    A --> C[Cosine Similarity]
    A --> D[Freshness Score]
    A --> E[Unknown Signal]
    B --> F[Weighted Rank]
    C --> F
    D --> F
    E --> F
    F --> G[Final Ranking]

Weights are learned from outcomes via Thompson sampling. Sources: www/src/app/whitepaper/page.tsx

Knowledge Block Structure

Traces are stored as compressed three-field blocks optimized for agent steering:

interface BlockHit {
  block: {
    id: string;
    trigger: {
      situation: string;      // Problem context description
    };
    body: {
      mechanism: string;      // Root cause explanation
      deadEnds: string[];     // Paths to avoid
      unlock: string;         // Solution approach
      verification: string;   // How to confirm fix
    };
  };
  calibratedProb: number;     // Match confidence
  refs: TraceRef[];           // Evidence traces
}

The three-field design prioritizes compression and dead-end steering over verbose documentation. Sources: src/core/block-serving.ts, src/core/build-injection-payload.ts

Block Rendering System

Silent Block Format

The block builder generates compact bullet-format traces:

• <Situation, capitalized>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>.

Dead ends are appended when present:

• <Situation>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>.
Avoid: <deadEnd1>; <deadEnd2>.

Imported traces receive special wrapping:

<prior_fix source="imported">...</prior_fix>

XML Audit Format

For detailed auditing, blocks render to structured XML:

<hypothesis id="block-123" calibrated="0.847">
  <situation>...</situation>
  <mechanism>...</mechanism>
  <dead_ends>
    <item>...</item>
  </dead_ends>
  <unlock>...</unlock>
  <verification>...</verification>
  <evidence trace="trace-id" role="user"/>
</hypothesis>

The calibrated attribute represents the probability that this block matches the current task context. Sources: src/core/block-serving.ts

Security Guard System

The guard module provides input validation against injection attacks:

Guard Patterns

interface GuardPattern {
  name: string;
  re: RegExp;
}

Spoof Categories

System Tag Spoofing:

• Inline <system>…</system> or <assistant> tags faking higher-privilege turns
• Backtick-neighbour skip: documented tags in backticks (e.g., "the <system> block marker") are allowed
• Raw injected tags still match

Delimiter Spoofing:

• Fake TraceBase delimiters like `prior_fix\n…\n` to inherit trust levels
• Patterns include: system, tool_result, prior_fix, file_memory, context_fold

Secret Exfiltration:

• Targets verbose environment variable(s)? form to reduce false positives
• Avoids matching benign code prose like "print env var name"

The guard intentionally avoids matching shorter patterns like env alone, which appear frequently in documentation. Sources: src/core/guard.ts

File Walking System

The file walker indexes project repositories for knowledge base population:

interface WalkOptions {
  baseRoot?: string;      // BFS start point (defaults to root)
  budget?: WalkBudget;    // Time/size constraints
  maxBytes?: number;      // Per-file size cap (default 256 KiB)
  now?: () => number;     // Time override for tests
}

Contract Guarantees

• Relative paths are computed against baseRoot when specified
• BFS starts at root but yields files with relPath = path.relative(baseRoot, abs)
• Caller ensures root is inside baseRoot
• Out-of-base paths surface as ..-prefixed and fail the repo-rel guard

Output Types

interface WalkedFile {
  relPath: string;       // Relative path from base
  content: string;       // File contents
  sizeBytes: number;     // File size
}

interface SkippedFile {
  relPath: string;       // Local-only path
  reason: "binary" | "too-large" | "excluded" | "error";
}

Files larger than maxBytes are skipped with reason 'too-large'. Binary files are detected and excluded automatically. Sources: src/core/file-walker.ts

Agent Integration

TraceBase integrates with AI agents through a multi-step hook system:

Hook Inspection

The CLI doctor command validates agent configuration:

const hookInspection = inspectAgentHookConfig(projectRoot, agent);

Checks include:

• MCP server configuration
• .claude/settings.json hook entries
• Managed events for the current agent

Event Mapping

Managed events vary by agent type:

const managedEvents = hookEventsForAgent(agent);
// e.g., ['on_tool_call', 'on_result', 'on_context_push']

The system differentiates between MCP surface and Claude Code hook surface, catching regressions where MCP is configured but stop hooks are missing. Sources: src/cli/commands/doctor.ts

Configuration System

Instruction File Management

Projects require a managed instruction block:

if (!instruction.present) {
  // Warn: instruction file missing
} else if (!instruction.managed) {
  // Warn: managed section missing
} else {
  // Pass: managed section present
}

Initialization command: npx tracebase-ai init

Initialization Checks

Data Flow Summary

graph TD
    subgraph Indexing
        A1[File Walker] --> A2[Project Files]
        A2 --> A3[Skipped Files]
        A3 -->|Binary/Large| A4[Excluded]
        A2 --> A5[Indexed Files]
        A5 --> A6[Knowledge Store]
    end
    
    subgraph Retrieval
        A6 --> R1[Fingerprint Lookup]
        A6 --> R2[BM25 FTS5]
        R1 --> R3[Candidate Set]
        R2 --> R3
        R3 --> R4[4 Signal Re-ranker]
        R4 --> R5[Ranked Blocks]
    end
    
    subgraph Injection
        R5 --> B1[Block Builder]
        B1 --> B2[Compact Bullet Format]
        B2 --> G1[Security Guard]
        G1 -->|Pass| I1[Context Injection]
        G1 -->|Fail| G2[Validation Error]
    end

Key Design Principles

1. Compression: Traces use three-field format to minimize context overhead
2. Dead-end steering: Explicit path avoidance prevents repeated exploration
3. Trust inheritance prevention: Guard system blocks injection attacks
4. Compound intelligence: Library growth improves match quality across all users
5. Reproducibility: All benchmark data and seeds are version-controlled Sources: www/src/app/whitepaper/page.tsx

Component Interactions

Related Pages

Related topics: System Architecture, Retrieval System

Block Store

Overview

The Block Store is the persistent knowledge repository that captures, stores, and retrieves reasoning patterns extracted from agent behavior. It enables agents to learn from past problem-solving experiences and apply that knowledge to similar future tasks.

Block Store operates as a two-tier system: candidate blocks represent newly extracted patterns awaiting validation, while active blocks represent high-confidence patterns ready for retrieval and injection into agent contexts. Sources: src/server/mcp-v2-helpers.ts:63-75

Data Model

Block Structure

Each block consists of two primary components: a trigger and a body, along with metadata for calibration and provenance tracking.

Block Provenance

Blocks track their origin through the provenance field:

Sources: src/distillation/llm-distiller.ts:89-100

Architecture

graph TD
    subgraph Extraction
        A[Agent Execution] --> B[Outcome Logging]
        B --> C[LLM Distiller]
        C --> D[Candidate Block]
    end
    
    subgraph Storage
        D --> E[SQLite: reasoning_blocks]
        E --> F{Calibration}
        F --> G[Active Block]
        F --> H[Candidate Block]
    end
    
    subgraph Retrieval
        G --> I[Fingerprint + BM25]
        I --> J[Re-ranker]
        J --> K[Top-K Block Hits]
    end
    
    subgraph Injection
        K --> L[build-injection-payload]
        L --> M[Compact Bullet Format]
        M --> N[Agent Context]
    end

Storage Layer

Block Store uses SQLite as its persistence mechanism, storing blocks in the reasoning_blocks table. Sources: src/cli/commands/doctor.ts:89-104

Database Schema

The store file is located at .tracebase/memory.db within the project directory. Blocks are queried using fingerprint matching for O(1) candidate narrowing.

Store Operations

Important: The v2 reasoning_blocks table is distinct from the legacy ReasoningTrace table used by the v1 store MCP tool. Blocks stored via the legacy path are not visible to the v2 retrieval system. Sources: src/server/mcp-v2-helpers.ts:48-62

Retrieval Pipeline

Two-Stage Rank

Block retrieval follows a two-stage ranking approach:

1. Candidate Narrowing (O(1)): Fingerprint matching identifies potential blocks
2. Full-text Search: BM25 via FTS5 provides additional candidate filtering
3. Re-ranking: Four signals re-rank the candidates
4. Thompson Sampling: Weights learned from outcomes Sources: www/src/app/whitepaper/page.tsx:1

Re-ranking Signals

Weights are learned via Thompson sampling from historical outcomes. Sources: www/src/app/whitepaper/page.tsx:1

Injection Format

Compact Bullet Format

When blocks are injected into agent context, they are rendered as compact bullet points to minimize token usage:

• <Situation, capitalized>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>.

Dead ends are appended when present:

• <Situation>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>. Avoid: a; b; c.

XML Audit Format

For debugging and audit purposes, blocks can be rendered as XML:

<hypothesis id="block-123" calibrated="0.847">
  <situation>Null pointer in config initialization</situation>
  <mechanism>Config object not initialized before use</mechanism>
  <dead_ends>
    <item>Retry initialization</item>
    <item>Use default config</item>
  </dead_ends>
  <unlock>Add null check and initialize before first access</unlock>
  <verification>Run config validation test</verification>
  <evidence trace="case-456" role="origin"/>
</hypothesis>

Sources: src/core/block-serving.ts:42-60

Imported Tag Wrapper

Blocks imported from other projects are wrapped with a source tag:

<prior_fix source="imported">• Situation. Mechanism: ...</prior_fix>

This allows the system to track cross-project knowledge transfer. Sources: src/core/build-injection-payload.ts:6-8

Distillation Process

LLM Distiller

The distillation process extracts reasoning patterns from agent execution logs using an LLM. Input validation ensures:

The distiller uses balanced-brace scanning to extract JSON from LLM responses, handling nested objects and arrays robustly. Sources: src/distillation/llm-distiller.ts:70-95

Guardrails

Optional guardrails field can constrain when a block should apply:

if (guardrails !== undefined) {
  // Apply guardrail constraints
}

Security: Delimiter Spoofing

Block Store includes guard mechanisms to prevent injection attacks through fake delimiters:

This prevents attackers from wrapping malicious content in our trusted injection syntax. Sources: src/core/guard.ts:1

CLI Operations

Doctor Command

The tracebase doctor command validates store health:

The command also validates MCP server connectivity by spawning the registered MCP command with --selftest. Sources: src/cli/commands/doctor.ts:89-120

State Diagram

stateDiagram-v2
    [*] --> Candidate: storeReasoningPattern
    Candidate --> Active: Promotion
    Active --> Archived: Deprecated
    Candidate --> Discarded: Low confidence
    Active --> Active: Re-calibration

Configuration Options

Best Practices

1. Store Meaningful Patterns: Extract blocks from successful resolutions, not failed attempts
2. Keep Situations Short: The trigger should be a concise problem description
3. Include Dead Ends: Document what doesn't work to help agents avoid wasted effort
4. Verify Before Storing: Ensure the verification step actually confirms the fix
5. Calibration Matters: Lower confidence blocks remain candidates until enough positive outcomes raise their score

Related Components

Related Pages

Related topics: Block Store, Recall Mechanism

Retrieval System

Overview

The Retrieval System is TraceBase's core mechanism for matching incoming agent tasks against a knowledge base of prior problem-solving traces. It serves as the foundation for the compound-intelligence effect—enabling agents to benefit from institutional knowledge accumulated across team workflows.

The system's primary purpose is to identify relevant prior solutions when an agent encounters a problem "shape" it has seen before, thereby reducing agent steps, token consumption, and overall task completion cost.

Sources: www/src/app/whitepaper/page.tsx()

Architecture

The retrieval pipeline implements a two-stage ranking architecture:

1. Candidate Narrowing (Stage 1): Fingerprint and BM25 rapidly filter the candidate set using O(1) lookups and FTS5 full-text search respectively
2. Re-ranking (Stage 2): Four additional signals re-rank the narrowed candidates to surface the most contextually relevant traces

graph TD
    A[Query: Task Description] --> B[Stage 1: Candidate Narrowing]
    
    B --> C[Fingerprint<br/>Exact Match]
    B --> D[BM25<br/>FTS5 Lexical Search]
    
    C --> E[Candidate Set]
    D --> E
    
    E --> F[Stage 2: Re-ranking]
    
    F --> G[Jaccard<br/>Token Overlap]
    F --> H[Structural<br/>Feature Matching]
    F --> I[Cosine<br/>Semantic Similarity]
    F --> J[Freshness<br/>Temporal Decay]
    
    G --> K[Weighted Score]
    H --> K
    I --> K
    J --> K
    
    K --> L[Ranked Results]
    L --> M[Top-K Candidates]

Sources: www/src/app/whitepaper/page.tsx()

Retrieval Signals

TraceBase employs six distinct retrieval signals, each targeting a specific dimension of similarity.

Signal Overview

Sources: www/src/app/whitepaper/page.tsx()

Fingerprint (Exact Match)

The fingerprint signal provides O(1) lookup for identical problem detection. This is the fastest signal and serves as the first-line filter for known issues.

// Pattern from guard.ts - demonstrates exact matching approach
{
  name: "system-spoof",
  re: /(?<!`)<\s*\/?\s*(system|user|assistant)\s*>(?!`)/i,
}

The fingerprint implementation uses hash-based deduplication to identify exact matches without scanning the entire knowledge base.

BM25 (Lexical Search)

BM25 performs FTS5 full-text search to find candidates sharing keywords but with different phrasing. This handles paraphrased queries that fingerprint would miss.

Jaccard Similarity

The Jaccard signal measures token overlap between the query and stored traces. It identifies structural keyword matching beyond simple lexical search.

Structural Matching

Structural signals detect feature-level similarity: same error type, same programming language, same framework. This captures semantic patterns that token-based methods might miss.

Cosine Similarity (Optional)

Cosine similarity computes embedding-based semantic similarity. This is the slowest signal and is optional, enabled when semantic understanding provides value over lexical methods.

Sources: src/embeddings/openai.ts()

Freshness (Temporal Decay)

The Freshness signal applies exponential decay to recency, implementing a recency bias that prioritizes newer solutions while still surfacing older relevant ones.

Signal Weighting

Signal weights are learned per project from outcome events using Thompson sampling, as described in Agrawal & Goyal (2012).

graph LR
    A[Retrieval Event] --> B[Outcome: Helpful?]
    B --> C[Thompson Sampling]
    C --> D[Updated Weights]
    D --> A
    
    E[Block Quality] --> F[Wilson Interval Lower Bound]
    F --> G[Auto-Demotion]

Block quality is measured using the Wilson-interval lower bound on the helpfulness rate. Blocks that stop earning their keep are automatically demoted.

Sources: www/src/app/whitepaper/page.tsx()

Trace Storage

Traces are stored as three short fields optimized for compression and agent guidance:

This structure is designed to steer the agent past dead ends it would otherwise re-explore, referencing the C3oT (arxiv 2412.11664) and TALE (arxiv 2412.18547) approaches.

Sources: www/src/app/whitepaper/page.tsx()

Injection Payload

Retrieved traces are formatted into injection payloads that the agent can optionally use. The payload construction emphasizes:

• Positive constraints over negative framing ("the bug is X, fix: Y", not "do not try A, B, C")
• Skip-to-fix when prior knowledge is available (plan-and-act, not explore-first)

The injection system includes a guard layer that prevents spoofed delimiters and malicious content injection:

// From guard.ts - delimiter spoof prevention
{
  name: "delimiter-spoof",
  re: /(```\s*(system|tool_result|prior_fix|file_memory|context_fold)\b|<\s*(prior_fix|file_memory|context_fold)\b)/i,
}

Sources: src/core/guard.ts:line-range()

Data Storage

The knowledge base uses local SQLite with WAL mode for storage:

graph TD
    A[New Trace] --> B[SQLite WAL]
    B --> C[Local Index]
    C --> D[Retrieval Query]
    D --> C
    
    E[Cloud Sync] --> F[Opt-in]
    F --> B

Cloud sync is opt-in, allowing teams to control data residency and privacy settings.

Sources: www/src/app/whitepaper/page.tsx()

Performance Characteristics

Configuration

The retrieval system supports per-project configuration:

• Signal weights: Learned via Thompson sampling, not manually configured
• Cosine enablement: Optional, enables semantic matching when beneficial
• Freshness decay rate: Configurable temporal weighting
• High-confidence threshold: Determines when to auto-inject vs. surface for agent decision

Security Considerations

The guard system implements multiple layers of protection:

1. Backtick-neighbour skip: Documentation wrapped in inline backticks is not treated as spoofed markers
2. Delimiter spoof prevention: Blocks attempts to ride on injection prefixes
3. Bounded gap detection: Environment variable exfiltration patterns use verbose forms to reduce false positives

Sources: src/core/guard.ts()

Related Components

Benchmark Results

The retrieval system has been validated on:

• SWE-bench Verified: +5-25% accuracy gains depending on model
• TypeScript fixtures: 10 verified tests, 100% accuracy maintained across models
• High-confidence match rate: 55% observed in benchmarks

Sources: www/src/app/whitepaper/page.tsx()

Related Pages

Related topics: Context Fold, Retrieval System

Recall Mechanism

The Recall Mechanism is TraceBase's memory retrieval system that enables AI agents to access previously captured problem-solving knowledge during new task execution. It serves as the core pillar of TraceBase's "Learn & Apply" functionality, bridging the gap between isolated problem solutions and ongoing development work.

Overview

The recall system operates through a two-stage retrieval pipeline. First, fingerprint and BM25 searches narrow the candidate set in O(1) and FTS5 lookups respectively. Then, four additional re-ranking signals refine the results before calibrated probabilities are assigned to each candidate.

Sources: www/src/app/whitepaper/page.tsx

The mechanism integrates with the agent runtime through middleware wrappers that intercept LLM API calls. When an agent prompt is submitted, the system retrieves relevant reasoning blocks and facts, then injects them into the context window before the model processes the request.

Recall Entry Points

The recall system exposes multiple entry points depending on the type of memory being retrieved:

Sources: src/types.ts:types-interface

Chunk-Based Context Compression (rc.6)

The recallChunks method enables same-session context compression for long-running tasks:

interface RecallChunksInput {
  sessionId: string;
  /** Top-K cap (default 3). Recall is scoped to the same session only. */
  k?: number;
}

interface RecallChunksResult {
  /** Oldest-first within the K-window. May be empty. */
  hits: Array<{
    chunkStartTurn: number;
    chunkEndTurn: number;
    summary: string;
    tokensBefore: number;
    tokensAfter: number;
  }>;
}

The folding mechanism compresses {role, content}[] turn lists into rolling chunks (8 turns or ≥4k tokens, whichever first). A watermark stored in session_chunks.MAX(chunk_end_turn) ensures idempotent behavior.

Sources: src/types.ts:RecallChunksInput-RecallChunksResult

Configuration Options

The RecallForPromptOptions interface defines all configurable parameters for recall operations:

Sources: src/runtime/recall.ts:RecallForPromptOptions

Token Budget Management

The token budget controls how much memory content is injected into the context:

• Default soft budget: 1200 tokens
• Hard cap: 2200 tokens (enforced by buildInjectionPayload)
• Adjustment: Values above the cap are silently reduced

The clampToBudget function trims sections to fit within the allocated budget, returning both the clipped sections and a truncated flag.

Sources: src/lib/control-plane/issue-brief.ts

Middleware Integration

Anthropic Client Wrapping

The recall system integrates with the Anthropic API through a Proxy-based wrapper in src/middleware/anthropic.ts:

export function wrapAnthropic<T extends object>(
  client: T,
  layer: ReasoningLayer,
  recallConfig?: RecallInjectConfig,
): T

The wrapper intercepts two methods:

Sources: src/middleware/anthropic.ts:wrapAnthropic

The wrapping uses a Proxy with an apply handler that:

1. Checks if the client is already wrapped (symbol-based guard)
2. Resolves the runtime configuration
3. Applies the recall injection handler to intercepted calls

Injection Flow

graph TD
    A[Agent Prompt] --> B[Middleware Intercepts]
    B --> C{recallConfig.enabled?}
    C -->|false| D[Pass Through]
    C -->|true| E[Resolve Runtime]
    E --> F[Retrieve Memories]
    F --> G[Build Injection Payload]
    G --> H[clampToBudget]
    H --> I[Inject into Context]
    I --> J[LLM Processing]

Block Retrieval and Rendering

Block Hit Structure

When a reasoning block is matched, it is rendered as a structured block hit:

interface BlockHit {
  block: ReasoningBlock;
  calibratedProb: number;
  refs: Array<{
    traceId: string;
    role: string;
  }>;
}

Sources: src/core/block-serving.ts:renderBlockHitXml

Compact Rendering Format

Block hits are rendered in a compact bullet format:

• <Situation, capitalized>. Mechanism: <mechanism>. Fix: <unlock>. Verify: <verification>.

If dead ends exist, an additional line is appended:

Avoid: a; b; c

The rendering deliberately avoids mentioning block IDs or calibrated probabilities to maintain clean context that appears as natural guidance rather than tooling output.

Sources: src/core/build-injection-payload.ts:renderBlockSilent

XML Audit Format

For audit purposes, blocks can be rendered as XML:

<hypothesis id="block-id" calibrated="0.720">
  <situation>...</situation>
  <mechanism>...</mechanism>
  <dead_ends>
    <item>...</item>
  </dead_ends>
  <unlock>...</unlock>
  <verification>...</verification>
  <evidence trace="trace-id" role="role"/>
</hypothesis>

Loop Detection

The recall mechanism includes a loop detector that monitors for repetitive tool usage patterns:

Sources: src/sdk/runtime.ts:loop-signals

Loop detection uses the toolWindowSize parameter (default 6) to consider recent observations. When a loop is detected, the system emits a signal with:

• kind: "loop" or kind: "tool"
• label: Human-readable description (e.g., "▣ TB LOOP straight × 3 (ToolName)")
• count: Number of repetitions
• toolName: The tool involved (when applicable)

Guardrails and Security

The recall system includes guardrails to prevent prompt injection attacks:

The system-spoof rule includes backtick-neighbor skip logic: a documented tag wrapped in inline backticks (e.g., "the <system> block marker") is documentation, not a spoofed turn marker.

Sources: src/core/guard.ts:system-spoof-delimiter-spoof

Analytics and Metrics

Cohort Analysis

The recall system tracks usage through two cohorts:

interface UsageCausal {
  assisted: UsageCohort;
  holdout: UsageCohort;
  resolvedLift: number | null;
  tokensLift: UsageEstimate;
}

The resolvedLift is calculated as assisted.resolvedRate − holdout.resolvedRate and is null when either arm has fewer than minCohortSize outcomes.

Sources: src/analytics/usage-metrics.ts:UsageCausal

Token Savings Calculation

Token savings are computed as:

tokensLift = (mean(holdout.tokens) − mean(assisted.tokens)) × assisted.n

This represents the total tokens saved across the measurement window, attributable to the assist feature.

Session Scoping

The recall system implements different scoping rules for different memory types:

Sources: src/runtime/recall.ts:sessionId-docs

Calibration

The calibrator assigns probability scores to block hits based on historical accuracy data. The calibration is used to:

1. Rank multiple matching blocks
2. Determine whether to include borderline matches
3. Provide confidence signals for the injection decision

Calibrated probabilities appear in audit-mode XML output but are deliberately excluded from silent rendering to maintain clean agent-facing context.

Summary

The Recall Mechanism is a sophisticated retrieval system that:

1. Retrieves relevant memory through multi-stage ranking (fingerprint → BM25 → re-ranking)
2. Configures retrieval via token budgets, session scoping, and K limits
3. Injects matched memory into agent context through middleware interception
4. Detects loops using tool observation windows
5. Secures the pipeline against prompt injection attacks
6. Measures impact through cohort-based analytics
7. Compresses long contexts through same-session chunk folding

Related Pages

Related topics: Recall Mechanism

Loop Detection

Overview

Loop Detection is a core runtime mechanism in TraceBase that identifies and signals when a coding agent enters repetitive behavior patterns during tool execution. It serves as a feedback system that allows the system to recognize when an agent is stuck in unproductive cycles, enabling corrective actions such as injecting prior solutions or presenting user-facing notifications.

The loop detection system operates by analyzing tool call sequences and categorizing repetitive behavior into distinct pattern types. This information flows through the SDK runtime where it is surfaced as structured signals that can trigger downstream handling.

Loop Signal Types

The system recognizes three primary loop pattern kinds, each representing a different form of repetition:

Sources: src/sdk/runtime.ts:1-50

Straight Loops

A straight loop occurs when an agent repeatedly calls the same tool multiple times in sequence. This pattern is detected when consecutive tool invocations target identical tool names.

if (signal.kind === "straight" && enableLoop) {
  return {
    kind: "loop",
    label: signal.toolName
      ? `▣ TB LOOP  straight × ${signal.count} (${signal.toolName})`
      : `▣ TB LOOP  straight × ${signal.count}`,
    count: signal.count,
    ...(signal.toolName ? { toolName: signal.toolName } : {}),
  };
}

Sources: src/sdk/runtime.ts:1-30

Ping-Pong Loops

A pingpong pattern emerges when an agent alternates between two different tools back and forth. This indicates the agent is caught in a retry cycle between two approaches.

if (signal.kind === "pingpong" && enableLoop) {
  return {
    kind: "loop",
    label: signal.toolName
      ? `▣ TB LOOP  ping-pong (${signal.toolName})`
      : "▣ TB LOOP  ping-pong",
    count: signal.count,
    ...(signal.toolName ? { toolName: signal.toolName } : {}),
  };
}

Sources: src/sdk/runtime.ts:1-30

Duplicate Detection

Duplicate tool calls represent the simplest form of repetition where identical calls are made consecutively. This pattern triggers the duplicate kind.

if (signal.kind === "duplicate" && enableTool) {
  return {
    kind: "tool",
    label: signal.toolName
      ? `▣ TB LOOP  duplicate (${signal.toolName})`
      : "▣ TB LOOP  duplicate",
    count: signal.count,
    ...(signal.toolName ? { toolName: signal.toolName } : {}),
  };
}

Sources: src/sdk/runtime.ts:1-30

Tool Input Observation

The loop detection system relies on extracting file paths from tool inputs to correlate repeated operations on the same resources. The observation layer supports multiple naming conventions across different agent frameworks.

for (const key of ["file_path", "path", "filename", "filePath"] as const) {
  const v = obj[key];
  if (typeof v === "string" && v.trim().length > 0) return v;
}

Sources: src/runtime/observe-tools.ts:1-30

Loop Signal Structure

Each loop signal includes contextual metadata that enables downstream handlers to make informed decisions:

Sources: src/sdk/runtime.ts:1-30

Integration with Block Serving

Loop signals are incorporated into the block serving pipeline where they can trigger injection of previously successful solutions from the knowledge base. The XML rendering format includes calibrated probability scores for hypothesis blocks.

<hypothesis id="${hit.block.id}" calibrated="${hit.calibratedProb.toFixed(3)}">
  <situation>${escapeXml(hit.block.trigger.situation)}</situation>
  <mechanism>${escapeXml(hit.block.body.mechanism)}</mechanism>
  <unlock>${escapeXml(hit.block.body.unlock)}</unlock>
  <verification>${escapeXml(hit.block.body.verification)}</verification>
</hypothesis>

Sources: src/core/block-serving.ts:1-30

User-Facing Loop Indicators

When loop detection is enabled, the SDK renders loop badges in the agent interface to provide visual feedback to users. These badges are constructed dynamically based on the detected pattern type.

The label generation follows a consistent format:

• Straight loops: ▣ TB LOOP straight × {count} ({toolName})
• Ping-pong loops: ▣ TB LOOP ping-pong ({toolName})
• Duplicate calls: ▣ TB LOOP duplicate ({toolName})

Sources: src/sdk/runtime.ts:1-30

Architecture Summary

graph TD
    A[Tool Call] --> B[observe-tools.ts]
    B --> C[Extract file_path]
    C --> D[Loop Pattern Analysis]
    D --> E{Loop Detected?}
    E -->|No| F[Normal Flow]
    E -->|Yes| G[Classify Pattern]
    G --> H{straight | pingpong | duplicate}
    H --> I[SDK Runtime Signal]
    I --> J[Enable Loop Check]
    J -->|Enabled| K[Generate Loop Badge]
    J -->|Disabled| L[Suppress Signal]
    K --> M[Block Serving]
    L --> N[Continue Execution]
    M --> O[Inject Prior Solutions]

Configuration

Loop detection behavior is controlled through runtime flags:

Sources: src/sdk/runtime.ts:1-30

Relationship to Dead-End Detection

The loop detection system complements the dead-end detection mechanism in the distillation pipeline. While loop detection identifies repetitive tool sequences, dead-end detection analyzes analysis steps to surface abandoned hypotheses.

// Dead-end detection logic
for (let i = 0; i < analyses.length - 1; i++) {
  const cur = analyses[i];
  const next = analyses[i + 1];
  if (!hasNegativeSignal(next.description)) continue;
  const summary = summarizeDeadEnd(cur.description);
  if (summary && !seen.has(key)) {
    seen.add(key);
    deadEnds.push(summary);
  }
}

Sources: src/distillation/heuristics.ts:1-30

Summary

Loop Detection in TraceBase provides a structured mechanism for identifying repetitive agent behavior through three distinct pattern classifications: straight, pingpong, and duplicate. The system integrates with tool observation to extract file context, generates user-visible indicators via the SDK runtime, and connects to the block serving infrastructure for solution injection. This enables agents to recognize when they are stuck and either self-correct or receive assistance from the knowledge base.

Related Pages

Related topics: Recall Mechanism, Loop Detection

Context Fold

Overview

Context Fold is a structured memory injection mechanism in TraceBase that allows AI agents to preserve hierarchical, expanded file context within agent prompts. Rather than flattening file context into a simple bullet list, Context Fold maintains structural relationships and metadata about files that an agent has previously explored or analyzed during a session.

The feature is referenced throughout the codebase as both a UI presentation concept and a structured injection format. It enables agents to "remember" file exploration context across turns, allowing them to efficiently return to previously analyzed files without redundant exploration. Sources: src/core/guard.ts:17-22

Purpose and Scope

Context Fold serves two primary functions within the TraceBase system:

1. Structured Memory Preservation - It wraps file context in a structured format that maintains hierarchy (file → sections → lines) rather than converting everything to prose
2. Injection Security - The <context_fold> delimiter is explicitly protected by the guard system to prevent spoofed injections

The feature is particularly relevant when an agent has performed multi-file exploration (referred to in the whitepaper as "explore codebase" phases) and needs to preserve that context for future turns or similar tasks. Sources: www/src/components/landing/_demo-fixtures/capability-mockups.tsx:60-80

Architecture

    A[Agent Exploration] --> B[File Indexer]
    B --> C[File Summarizer]
    C --> D[Context Fold Generation]
    D --> E[Build Injection Payload]
    E --> F[Guard Validation]
    F --> G[Agent Context]
    
    H[Guard: context_fold] --> F

Component Flow

Injection Format

Context Fold is injected into agent prompts using a dedicated XML-like delimiter format:

Utility functions for string manipulation

<context_fold>
  <file path="src/utils/helper.ts">
    <section start="1" end="25">
      <content>...</content>
    </section>
  </file>
</context_fold>

This format is distinct from other injection types:

The distinction between file_memory and context_fold is intentional: file_memory is a flat reference, while context_fold preserves structural depth. Sources: src/core/guard.ts:17-22

Security: Guard Protection

The Context Fold delimiter is protected by the TraceBase guard system against injection attacks. The guard pattern delimiter-spoof specifically validates that legitimate context_fold tags are not being faked:

{
  name: "delimiter-spoof",
  re: /(```\s*(system|tool_result|prior_fix|file_memory|context_fold)\b|<\s*(prior_fix|file_memory|context_fold)\b)/i,
}

This pattern matches:

• Backtick-wrapped faked delimiters: ```prior_fix\n…\n```
• Angle-bracket faked delimiters: <context_fold>

The guard ensures that only legitimate TraceBase-generated folds are accepted, preventing malicious actors from attempting to inject content by mimicking the delimiter format. Sources: src/core/guard.ts:14-22

UI Presentation

In the TraceBase web dashboard and landing page mockups, Context Fold is visualized as part of the "live window" display, showing the agent's current exploration state:

<MockLine
  n="23–27"
  body="live window"
  trailing={<Chip tone="ember" size="sm">active</Chip>}
  highlight
/>

The FoldMockup component displays how Context Fold appears in the dashboard, showing:

• Exploration phases (e.g., "explore codebase")
• Turn ranges (e.g., "01–08", "17–22")
• Token reduction metrics (e.g., "4.2k → 340")

This visualization helps users understand how the agent's exploration context is being compressed and preserved across sessions. Sources: www/src/components/landing/_demo-fixtures/capability-mockups.tsx:60-80

Integration with Block Serving

When serving memory blocks to agents, the Context Fold data can be rendered in XML format for structured display. The renderBlockHitXml function handles the rendering of block hits, which may include Context Fold data:

function renderBlockHitXml(hit: BlockHit, audit: boolean): string {
  const parts: string[] = [];
  parts.push(`  <hypothesis id="${hit.block.id}" calibrated="${hit.calibratedProb.toFixed(3)}">`);
  parts.push(`    <situation>${escapeXml(hit.block.trigger.situation)}</situation>`);
  // ... additional fields including potential context fold data
}

This XML format provides a machine-readable structure that agents can parse while maintaining semantic clarity for human review during debugging. Sources: src/core/block-serving.ts:55-75

Use Cases

Context Fold enables several key workflows:

1. Multi-file Exploration Memory - When an agent explores a codebase to understand a bug, Context Fold preserves which files were examined and the insights gained from each

1. Efficient Context Reuse - Agents can quickly reference previously explored files without re-reading entire file contents

1. Hierarchical Compression - The fold format maintains structure (file → section → content) rather than flattening to prose, preserving searchability

1. Cross-Session Learning - When used with the distillation system, Context Fold data from successful agent runs can be stored and replayed for similar future tasks

Configuration

Context Fold behavior is typically configured through the injection payload builder. The build-injection-payload.ts module determines when and how folds are included in agent prompts based on:

• Relevance scoring from the retrieval system
• Calibration probabilities from the block serving layer
• Provenance tracking (whether content is "imported" or "extracted")

The wrapIfImported function applies special formatting for externally imported context folds to distinguish them from internally generated content:

function wrapIfImported(line: string, imported: boolean): string {
  return imported ? `${IMPORTED_TAG_OPEN}${line}${IMPORTED_TAG_CLOSE}` : line;
}

Related Components

Related Pages

Related topics: Agent Adapters

SDK Reference

Overview

TraceBase provides a Software Development Kit (SDK) for integrating its memory layer into coding agent workflows. The SDK enables agents to retain and retrieve institutional knowledge across sessions, including past solutions, file meanings, and patterns that prevent doom-loops.

Sources: www/src/components/landing/HeroInk.tsx:1-10

Architecture Overview

The SDK operates as a middleware layer that intercepts agent interactions, manages memory retrieval, and injects relevant context into the agent's runtime. The architecture follows a two-stage retrieval pipeline:

graph TD
    A[Agent Session] --> B[Middleware Layer]
    B --> C[Fingerprint Lookup]
    B --> D[BM25 FTS5 Lookup]
    C --> E[Candidate Set]
    D --> E
    E --> F[4-Signal Re-ranker]
    F --> G[Top-K Hits]
    G --> H[Injection Payload Builder]
    H --> I[Agent Context]
    I --> J[Agent Action]

Sources: www/src/app/whitepaper/page.tsx:1-50

Core Components

Middleware Layer

The middleware serves as the primary integration point with agent runtimes. It handles:

Sources: src/core/guard.ts:1-30

Block Serving

The block-serving module renders retrieved memory blocks into formatted output suitable for agent consumption:

// Compact bullet format rendering
• <Situation, capitalized>. Mechanism: <…>. Fix: <unlock>. Verify: <verification>.

Sources: src/core/block-serving.ts:1-45

Injection Payload Builder

Builds the memory context that gets injected into agent sessions:

const IMPORTED_TAG_OPEN = `<prior_fix source="imported">`;
const IMPORTED_TAG_CLOSE = `</prior_fix>`;

The builder creates compact, natural-language formatted blocks that:

• Capitalize the situation description
• Trim sentences to essential information
• Wrap imported fixes with provenance tags
• Include dead-end warnings when applicable

Sources: src/core/build-injection-payload.ts:1-35

Integration Patterns

Supported Integrations

TraceBase supports integration through:

Sources: www/src/components/landing/HeroInk.tsx:1-10

Installation Tracking

The SDK tracks installations across projects:

Sources: www/src/components/dashboard/InstallationsView.tsx:1-40

Retrieval Pipeline

The SDK uses a two-stage ranker for memory retrieval:

graph LR
    A[Fingerprint] --> C[Candidates]
    B[BM25 FTS5] --> C
    C --> D[Signal 1]
    C --> E[Signal 2]
    C --> F[Signal 3]
    C --> G[Signal 4]
    D --> H[Re-ranked Results]
    E --> H
    F --> H
    G --> H

Stage 1: Candidate Narrowing

• Fingerprint Lookup: O(1) operation for quick candidate filtering
• BM25 FTS5: Full-text search for semantic matching

Stage 2: Re-ranking Signals

Four additional signals re-rank candidates:

1. Semantic relevance
2. Temporal recency
3. Usage frequency
4. Project context match

Sources: www/src/app/whitepaper/page.tsx:1-50

Security Model

Guardrails

The SDK implements security checks to prevent injection attacks:

The guardrail uses negative lookbehind to avoid false positives from inline backticks:

/(?<!`)<\s*\/?\s*(system|user|assistant)\s*>(?!`)/i

Sources: src/core/guard.ts:1-45

Imported Fix Tagging

Imported fixes are wrapped with provenance tags:

<prior_fix source="imported">
  <!-- fix content -->
</prior_fix>

This allows agents to understand the origin of solutions and apply appropriate confidence levels.

Sources: src/core/build-injection-payload.ts:1-15

Block Data Model

BlockHit Structure

FactHit Structure

Sources: src/core/block-serving.ts:1-60

Output Formats

XML Format (Audit Mode)

<hypothesis id="block-123" calibrated="0.87">
  <situation>Memory allocation fails under load</situation>
  <mechanism>Fragmented heap from rapid allocation/deallocation</mechanism>
  <unlock>Enable memory pool with 64KB chunks</unlock>
  <verification>Run stress test with 10K concurrent requests</verification>
  <evidence trace="trace-abc" role="root_cause"/>
</hypothesis>

Markdown Format (Agent Context)

• Memory allocation fails under load. Mechanism: Fragmented heap from rapid
  allocation/deallocation. Fix: Enable memory pool with 64KB chunks. Verify:
  Run stress test with 10K concurrent requests. Avoid: Increasing heap size
  without addressing fragmentation.

Sources: src/core/block-serving.ts:1-70

Configuration

Retrieval Configuration

Output Configuration

Quick Start

# Initialize TraceBase in a project
npx tracebase-ai init

# This creates a link between the project and the workspace
# allowing the SDK to track and retrieve project-specific memory

After initialization, the SDK automatically:

1. Detects agent sessions in the project
2. Retrieves relevant memory on each prompt
3. Injects formatted context into the agent's context window

Sources: www/src/components/dashboard/OverviewView.tsx:1-30

Memory Funnel

The SDK tracks the effectiveness of memory retrieval through a funnel:

This funnel helps measure the realized cost savings from memory injection.

Sources: www/src/components/dashboard/ImpactView.tsx:1-60

Related Pages

Related topics: SDK Reference

Agent Adapters

Agent Adapters are the integration layer that enables TraceBase to communicate with and capture context from various AI coding agents (Claude Code, Cursor, Codex). They handle agent detection, configuration management, memory injection, and hook registration across different agent ecosystems.

Overview

TraceBase supports multiple AI coding agents through a unified adapter system. Each adapter translates between TraceBase's internal abstractions and the agent's specific configuration formats, hooks, and communication protocols.

The adapter system handles:

• Agent Detection: Automatically identifying which agent is running
• Configuration Management: Writing agent-specific configuration files
• MCP Server Integration: Registering TraceBase as an MCP server for each agent
• Hook Management: Installing and maintaining agent hooks for memory capture and injection
• Instruction Injection: Providing agent-specific instruction files (CLAUDE.md, cursor rules, etc.)

Supported Agents

TraceBase currently supports three major AI coding agents:

Agent Detection Logic

Agent detection occurs through two complementary strategies: environment-based detection and project-based detection.

Environment-Based Detection

The detectAgentFromEnvironment() function examines process environment variables to identify the active agent:

function detectAgentFromEnvironment(): InstallAgent | null {
  // Codex detection
  if (
    (process.env.CODEX_SHELL === "1" || 
     process.env.CODEX_CI === "1" || 
     process.env.CODEX_THREAD_ID) &&
    isCommandAvailable("codex", ["--version"])
  ) {
    return "codex";
  }
  
  // Cursor detection
  if (
    process.env.CURSOR_TRACE_ID ||
    process.env.CURSOR_AGENT ||
    process.env.TERM_PROGRAM?.toLowerCase() === "cursor"
  ) {
    return "cursor";
  }
  
  // Claude Code detection
  if (
    process.env.CLAUDECODE ||
    process.env.CLAUDE_CODE ||
    process.env.CLAUDE_PROJECT_DIR ||
    process.env.CLAUDE_DESKTOP
  ) {
    return "claude-code";
  }
  
  return null;
}

Sources: src/cli/install-targets.ts:38-58

Project-Based Detection

The detectAgentFromProject() function inspects project files to determine which agent owns the project:

function detectAgentFromProject(basePath: string): InstallAgent | null {
  if (
    existsSync(join(basePath, ".claude", "settings.json")) || 
    existsSync(join(basePath, "CLAUDE.md"))
  ) {
    return "claude-code";
  }
  return null;
}

Sources: src/cli/install-targets.ts:60-67

Detection Priority

graph TD
    A[Start Detection] --> B{Environment Variables Present?}
    B -->|Yes| C{Check CODEX_*}
    C -->|Found| D[Return: codex]
    C -->|Not Found| E{Check CURSOR_*}
    E -->|Found| F[Return: cursor]
    E -->|Not Found| G{Check CLAUDE_*}
    G -->|Found| H[Return: claude-code]
    G -->|Not Found| I[Return: null]
    B -->|No| J{Check Project Files}
    J -->|Found .claude/settings.json| H
    J -->|Found CLAUDE.md| H
    J -->|Neither Found| I

Configuration Files

Each agent uses different configuration mechanisms that the adapter system must manage.

Claude Code Configuration

Claude Code uses:

// Hook configuration structure in Claude Code
interface ClaudeHookConfig {
  hooks: {
    [eventName: HookEventName]: string | string[];
  };
}

Cursor Configuration

Cursor uses:

The adapter writes Cursor instructions via:

export function writeAgentsMarkdown(basePath: string): StepResult {
  return writeAgentInstructionFile(basePath, "cursor");
}

Sources: src/cli/install-targets.ts:27-29

Codex Configuration

Codex uses MCP (Model Context Protocol) exclusively:

// MCP configuration structure for Codex
interface McpConfig {
  mcpServers: {
    tracebase: {
      command: string;
      args: string[];
    };
  };
}

MCP Server Integration

The MCP (Model Context Protocol) server provides a standardized communication channel between TraceBase and supported agents.

Server Architecture

graph LR
    A[Agent] -->|MCP Protocol| B[tracebase-ai serve --mcp]
    B --> C[MCP Handler]
    C --> D[Block Serving]
    C --> E[Fact Retrieval]
    C --> F[Memory Injection]
    D --> G[Knowledge Base]
    E --> G
    F --> H[Agent Context]

Server Startup

The MCP server is started via:

npx -y tracebase-ai@latest serve --mcp

For Claude Code specifically, the adapter invokes:

claude mcp add tracebase --scope local -- npx -y tracebase-ai@latest serve --mcp

Sources: src/cli/commands/doctor.ts:120-122

MCP Health Checks

The doctor command validates MCP configuration for each agent:

const mcpInspection = inspectMcpConfig(projectRoot, agent);
if (!mcpInspection.present) {
  checks.push({
    name: mcpCheckName,
    level: "warn",
    message: `${getAgentTargetMeta(agent).mcpLocationLabel} is missing`,
    fix: `Run \`${initCommand}\` to register the MCP server.`,
  });
} else if (!mcp.canonical) {
  checks.push({
    name: mcpCheckName,
    level: "warn",
    message: "tracebase MCP entry has a non-canonical shape",
    fix: `Re-run \`${initCommand} --force\` to reset.`,
  });
}

Sources: src/cli/commands/doctor.ts:124-144

Hook System

The hook system enables agents to capture memory context at specific lifecycle events.

Supported Hook Events

Hook Self-Healing

The adapter includes a self-healing mechanism to maintain hook integrity:

const HEALTH_FILE_REL = ".tracebase/hook-health.json";
const DEFAULT_THROTTLE_MS = 24 * 60 * 60 * 1000; // 24 hours

The self-heal system:

1. Checks hook health on each init run
2. Compares current config against expected state
3. Repairs missing or corrupted hooks
4. Preserves user customizations (entries not managed by TraceBase)
5. Never modifies foreign hooks (entries not from TraceBase)

Sources: src/cli/hook-self-heal.ts:1-37

Hook Validation

The doctor command validates hook health:

const hookInspection = inspectAgentHookConfig(projectRoot, agent);
if (hookInspection.supported) {
  const managedEvents = hookEventsForAgent(agent);
  const states = managedEvents.map(
    (e) => [e, hookInspection.events[e] ?? "missing"] as const,
  );
  const missing = states.filter(([, s]) => s === "missing");
  
  // Report missing hooks as warnings
  if (missing.length > 0) {
    checks.push({
      name: `${agent}-hooks`,
      level: "warn",
      message: `Missing hooks: ${missing.map(([e]) => e).join(", ")}`,
    });
  }
}

Sources: src/cli/commands/doctor.ts:70-95

Instruction File Management

Agent adapters write instruction files that guide agent behavior with TraceBase integration.

Claude Code Instructions (CLAUDE.md)

The adapter appends a managed block to CLAUDE.md:

<!-- tracebase: managed -->
<!-- Everything below is managed by TraceBase. Edits preserved above. -->

... managed instructions ...

<!-- /tracebase: managed -->

Cursor Instructions (cursor.md)

Similar management for Cursor's instruction file.

Validation

The doctor command validates instruction files:

if (!instruction.present) {
  checks.push({
    name: instructionCheckName,
    level: "warn",
    message: `${instructionFile} is missing`,
    fix: `Run \`${initCommand}\` to create the instruction block.`,
  });
} else if (!instruction.managed) {
  checks.push({
    name: instructionCheckName,
    level: "warn",
    message: `${instructionFile} exists but the managed section is missing`,
    fix: `Re-run \`${initCommand}\` to append the managed block.`,
  });
}

Sources: src/cli/commands/doctor.ts:44-63

Installation Flow

sequenceDiagram
    participant User
    participant CLI
    participant Adapter
    participant Agent
    
    User->>CLI: tracebase init
    CLI->>Adapter: detectAgent()
    Adapter->>Adapter: detectAgentFromEnvironment()
    Adapter->>Adapter: detectAgentFromProject()
    Adapter->>Agent: getAgentTargetMeta(agent)
    
    alt Agent is Claude Code
        Adapter->>Agent: Write .claude/settings.json hooks
        Adapter->>Agent: Append to CLAUDE.md
        Adapter->>Agent: Register MCP via claude mcp add
    else Agent is Cursor
        Adapter->>Agent: Write cursor.md rules
        Adapter->>Agent: Register MCP
    else Agent is Codex
        Adapter->>Agent: Register MCP
    end
    
    Adapter->>Adapter: Self-heal hooks
    CLI->>User: Installation complete

Configuration Options

Init Command Options

interface InitOptions {
  path: string;              // Project root (default: cwd)
  agent?: InstallAgent;      // Force specific agent
  apiUrl?: string;           // TraceBase API URL
  apiKey?: string;           // Workspace API key
  force?: boolean;           // Overwrite existing config
  dryRun?: boolean;          // Preview changes (deprecated)
}

Agent Meta Configuration

Each agent has metadata defining its configuration locations:

const AGENT_TARGETS: Record<InstallAgent, AgentTarget> = {
  "claude-code": {
    mcpLocation: ".claude/settings.json",
    mcpLocationLabel: ".claude/settings.json",
    instructionFile: "CLAUDE.md",
    hooksSupported: true,
    hookSpec: "inline",
  },
  "cursor": {
    mcpLocation: "cursor.md",
    mcpLocationLabel: "cursor rules",
    instructionFile: "cursor.md",
    hooksSupported: false,
    hookSpec: "none",
  },
  "codex": {
    mcpLocation: ".codex.json",
    mcpLocationLabel: ".codex.json",
    instructionFile: null,
    hooksSupported: false,
    hookSpec: "none",
  },
};

Error Handling

Missing Configuration

When MCP configuration is missing:

checks.push({
  name: mcpCheckName,
  level: missingConfigSurface ? "warn" : "fail",
  message: `${getAgentTargetMeta(agent).mcpLocationLabel} is missing`,
  fix: `Run \`${initCommand}\` (${displayName} will not see TraceBase until then).`,
});

Non-Canonical Configuration

When an MCP entry exists but has unexpected structure:

checks.push({
  name: mcpCheckName,
  level: "warn",
  message: "tracebase MCP entry has a non-canonical shape",
  fix: `Re-run \`${initCommand} --force\` to reset to the canonical entry.`,
});

CLI Commands

`tracebase init`

Primary installation command. Auto-detects agent and configures all necessary integrations.

tracebase init [options]

`tracebase doctor`

Diagnostic command that validates adapter configuration:

tracebase doctor [options]

Validates:

• Instruction file presence and managed section
• Hook configuration (for supported agents)
• MCP server registration

`tracebase setup`

Legacy alias for tracebase init for backward compatibility.

tracebase setup [options]

Sources: src/cli/commands/setup.ts:1-20

Summary

Agent Adapters form the abstraction layer that enables TraceBase to integrate with multiple AI coding agents through a consistent interface. The system:

1. Detects agents through environment variables and project files
2. Configures agents with agent-specific file formats and locations
3. Registers MCP servers for standardized communication
4. Manages hooks with self-healing capabilities
5. Validates installations via the doctor command

The adapter architecture allows TraceBase to remain agent-agnostic at its core while supporting the unique configuration mechanisms of Claude Code, Cursor, and Codex.

Doramagic Pitfall Log

Doramagic extracted 7 source-linked risk signals. Review them before installing or handing real data to the project.

1. Configuration risk: Configuration risk needs validation

• Severity: medium
• Finding: Configuration risk is backed by a source signal: Configuration risk needs validation. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: capability.host_targets | github_repo:1203006515 | https://github.com/64envy64/tracebase | host_targets=mcp_host, claude, chatgpt

2. Capability assumption: README/documentation is current enough for a first validation pass.

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: The project should not be treated as fully validated until this signal is reviewed.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: capability.assumptions | github_repo:1203006515 | https://github.com/64envy64/tracebase | README/documentation is current enough for a first validation pass.

3. Maintenance risk: Maintainer activity is unknown

• Severity: medium
• Finding: Maintenance risk is backed by a source signal: Maintainer activity is unknown. Treat it as a review item until the current version is checked.
• User impact: Users cannot judge support quality until recent activity, releases, and issue response are checked.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: evidence.maintainer_signals | github_repo:1203006515 | https://github.com/64envy64/tracebase | last_activity_observed missing

4. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: downstream_validation.risk_items | github_repo:1203006515 | https://github.com/64envy64/tracebase | no_demo; severity=medium

5. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: risks.scoring_risks | github_repo:1203006515 | https://github.com/64envy64/tracebase | no_demo; severity=medium

6. Maintenance risk: issue_or_pr_quality=unknown

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: Users cannot judge support quality until recent activity, releases, and issue response are checked.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: evidence.maintainer_signals | github_repo:1203006515 | https://github.com/64envy64/tracebase | issue_or_pr_quality=unknown

7. Maintenance risk: release_recency=unknown

• Severity: low
• Finding: release_recency=unknown。
• User impact: Users cannot judge support quality until recent activity, releases, and issue response are checked.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: evidence.maintainer_signals | github_repo:1203006515 | https://github.com/64envy64/tracebase | release_recency=unknown

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using tracebase with real data or production workflows.

• Configuration risk needs validation - GitHub / issue