Doramagic Project Pack · Human Manual

nlm-memory

Local-first non-linear memory OS for AI operators. One index across Claude Code, Codex, Cursor, Windsurf, Hermes, OpenCode, Aider, pi, and more — with an editable timeline.

Overview & System Architecture

Related topics: Recall, Classification & Signals, Data Storage, Adapters & Supersedence, Deployment, Configuration & Operations

Section Related Pages

Continue reading this section for the full explanation and source context.

Related topics: Recall, Classification & Signals, Data Storage, Adapters & Supersedence, Deployment, Configuration & Operations

Overview & System Architecture

nlm-memory is a local-first, non-linear memory operating system for AI operators. It captures, indexes, and recalls agent session transcripts, structured facts, and committed code exemplars across multiple AI runtimes (Claude Code, Codex, Hermes, Pi, generic JSONL, and webhook-ingested sessions). The project ships as a single Node.js CLI named nlm, a long-running HTTP daemon, a Model Context Protocol (MCP) stdio server, and a Vite/React single-page UI. Source: package.json:1-16

Purpose & Design Philosophy

The codebase is organized around a composition-root pattern: src/cli/nlm.ts is "the one file that knows about every concrete implementation" (storage, LLM clients, HTTP, MCP), while every other module depends on ports rather than concrete classes. Swapping a backend is a localized edit to the CLI rather than a sweep through core/. Source: src/cli/nlm.ts:1-37

The system is local-first (SQLite is the canonical store), runtime-agnostic (source-kind enum covers six adapter shapes), and degradation-tolerant (recall-quality enhancements fail-open to raw queries rather than blocking recall). Examples of the latter appear throughout: query rewriting parses strict JSON and "throws so the caller fails open back to the raw query rather than embedding garbage," and pickRelatedFacts "returns an empty array so the pointer block degrades to the session-only format." Sources: src/core/recall/rewrite-prompt.ts:1-22, src/core/recall/related-facts.ts:1-30

System Architecture

The runtime stack decomposes into four layers. The CLI composition root wires concrete implementations into ports consumed by HTTP and MCP transports. Source: src/cli/nlm.ts:18-37

flowchart TB
    subgraph Clients["Agent Hosts"]
        CC[Claude Code]
        CX[Codex]
        HM[Hermes]
        PI[Pi]
        WH[Webhook]
    end

    subgraph Edge["Edge Surface"]
        MCP[MCP stdio server]
        HK[Recall hook]
        HTTPD[Hono HTTP daemon<br/>:3940]
        UI[Vite/React SPA]
    end

    subgraph Core["Core Services (port-driven)"]
        ING[Ingest / Classifier]
        REC[RecallService]
        FRS[FactRecallService]
        EXC[Code Exemplar lane]
        SCHED[Scheduler<br/>auto-chunk oversized]
        DOC[Doctor / Verify]
    end

    subgraph Storage["Storage Adapters"]
        SQ[(SQLite + sqlite-vec)]
        PG[(Postgres + pgvector)]
        BAK[Backup rotation<br/>stageRestore]
    end

    CC --> HK
    CC --> MCP
    CX --> MCP
    HM --> MCP
    PI --> MCP
    WH --> HTTPD
    HK --> HTTPD
    UI --> HTTPD
    MCP --> HTTPD
    HTTPD --> ING
    HTTPD --> REC
    HTTPD --> FRS
    HTTPD --> EXC
    ING --> SCHED
    ING --> SQ
    ING --> PG
    REC --> SQ
    REC --> PG
    FRS --> SQ
    FRS --> PG
    EXC --> SQ
    EXC --> PG
    DOC --> SQ
    DOC --> PG
    SQ <-.-> BAK
    PG <-.-> BAK

Key architectural facts verified against source:

  • Two interchangeable storage backendsSqliteStorage (better-sqlite3 + sqlite-vec) and PgStorage (pg + pgvector). Both implement the same store ports (e.g. CodeExemplarStore), and the active backend is chosen in the CLI composition root. Source: src/cli/nlm.ts:23-26, src/core/storage/sqlite-code-exemplar-store.ts, src/core/storage/pg-code-exemplar-store.ts
  • Dual recall surfaces — Session recall ("what the orchestrator pulls answering questions about past work") and fact recall ("structured facts agents pull mid-task"). Both share port-driven store contracts but expose separate MCP tools. Source: src/ui/pages/Recall.tsx:1-15
  • Hook as distribution surface — The recall hook's pointer block is described as "the only cross-runtime distribution surface for teaching the tool inventory" because fresh-install users never edit prompts; it therefore must name all four NLM MCP tools inline. Source: src/core/hook/pointer-block.ts:1-17
  • Citation detection with two channels — Strong signal: the model invoked an MCP tool whose input references a surfaced ID. Weak signal: the surfaced ID appears as a substring in response prose. This forms "the training-data substrate for a future learned reranker." Source: src/core/hook/citation-detect.ts:1-26

Subsystems & Components

SubsystemSource filesResponsibility
CLI / composition rootsrc/cli/nlm.tsWires storage, providers, sources, HTTP, MCP; subcommands: start, migrate, recall, mcp, setup, install, connect/disconnect, doctor, digest
Ingest + classifiercore/ingest, core/classifierPer-session classification; recent releases added JSON-resilience (v0.19.0), hierarchical classification + oversized-session recovery (v0.18.0), and auto-chunking on ingest (v0.20.0)
Schedulercore/schedulerAuto-chunks oversized sessions on ingest (v0.20.0); also drives digest jobs and backup rotation
Recallcore/recallQuery rewriting (Spec C), related-facts selection (Spec G.2), citation detection; degradation-tolerant by design
Fact recallcore/recall-factsStructured (subject, predicate, value) facts with corroboration gating (≥ MIN_CORROBORATION sessions)
Code exemplarscore/storage/{sqlite,pg}-code-exemplar-store.tsDeterministic IDs via `sha256(install_scoperepocode_hashoutcome)[:16]`; passive recall added in v0.16.0
Storage adapterscore/storage/sqlite-storage.ts, core/storage/pg-storage.ts, core/storage/db-restore.ts, core/storage/backup-rotation.tsBoth SQLite and Postgres paths; daily rolling backups via runRollingBackup; restore is staged through stageRestore / applyPendingRestore
Provider registrycore/providers/provider-registry.ts, core/providers/provider-models.tsSix provider kinds; model discovery uses /api/tags for Ollama, /models for OpenAI-compatible, hardcoded lists for Deepseek/Anthropic
Source registrycore/sources/source-registry.tsSix source kinds (claude-code, codex, hermes, pi, jsonl-generic, webhook); token redacted in HTTP responses except on one-time reveal
Hookscore/hook/pointer-block.ts, core/hook/citation-detect.tsPointer-only injection + citation logging for training data
UI (Vite/React SPA)src/ui/pages/*, src/ui/components/*Pages: Pulse, Recall, Thread, Stub; shared <Drawer> and <Pagination> components per the components README

Release History & Recent Changes

Recent release notes — directly relevant to architecture:

  • v0.20.0feat(scheduler): auto-chunk oversized sessions on ingest (#341). Affects the ingest → scheduler → storage path in the diagram.
  • v0.19.0feat(classifier): JSON-resilience — retry + per-chunk tolerance. Strengthens the ingest path when the LLM emits malformed classifier JSON.
  • v0.18.0feat(classifier): hierarchical classification + oversized-session recovery (#340). Adds the recovery loop that v0.20.0's scheduler now automates.
  • v0.16.0feat(recall): passive code-exemplar recall (Phase 2). Extends the recall surface to include exemplar hits alongside session hits.
  • v0.15.0feat(exemplars): auto-capture code exemplars from committed sessions (Phase 1). Introduces the CodeExemplar store tables and deterministic IDs.
  • v0.14.1fix(pg): add facts.retired_at to the PG schema. Brings the Postgres path to parity with SQLite for fact retirement.
  • v0.14.0 — Postgres backend for exemplars, runtime-tested webhook ingestSession PG branch, and the recall_code /mcp guarantee.
  • v0.13.0 — Fact-recall coverage + hybrid fix; install hardening (doctor/verify/codex-repair); daily backups; cleared app.ts of rawDb/cast escape hatches in favor of registries + actions + data-mgmt (#215a).
  • v0.12.0 — Preceding baseline; full changelog covers v0.11.1 → v0.12.0.

Source: package.json:2-3 (current version 0.20.0), src/cli/nlm.ts:91-117 (doctor command and --fix repair modes I1 self-loop edges, I2 orphaned superseded/replaced sessions).

See Also

Source: https://github.com/pbmagnet4/nlm-memory / Human Manual

Recall, Classification & Signals

Related topics: Overview & System Architecture, Data Storage, Adapters & Supersedence

Section Related Pages

Continue reading this section for the full explanation and source context.

Related topics: Overview & System Architecture, Data Storage, Adapters & Supersedence

Recall, Classification & Signals

1. Purpose & Scope

nlm-memory is a local-first non-linear memory operating system for AI operators (package.json:1-9). The Recall, Classification & Signals subsystem is the brain of that memory: it decides *what* an agent should be reminded of, *how* a fresh session is categorized, and *when* background work should re-classify or chunk oversized material.

Three concerns share this surface:

  • Recall — turn a natural-language prompt into a ranked list of past sessions, related facts, and code exemplars.
  • Classification — turn a session transcript into a stable taxonomy label (entity, kind, coherence) plus a per-class confidence.
  • Signals / Hooks — gate when recall fires, render the pointer block, detect citations, and schedule re-classification when ingest fails or chunks blow up.

Community-driven evolution is visible in the release history: v0.18.0 introduced hierarchical classification + oversized-session recovery, v0.19.0 added JSON-resilience to the classifier, and v0.20.0 added scheduler-driven auto-chunking on ingest (releases).

2. The Recall Pipeline

RecallService is the central use case. It composes filters, keyword scoring, semantic search, and optional related-fact / related-exemplar enrichment behind a single search() call. It depends only on ports (SessionStore, LLMClient, FactStore, CodeEmbedder, CodeExemplarStore), with no framework or HTTP imports (recall-service.ts:1-39).

The pipeline runs in this order:

flowchart TD
    A[Prompt arrives at hook] --> B[gate.ts: classifyPrompt<br/>generative vs evaluate]
    B -->|evaluate| C[RecallService.search]
    C --> D[Optional query rewrite<br/>rewrite-prompt.ts]
    D --> E[Keyword leg +<br/>Semantic leg]
    E --> F[query-shape.ts:<br/>force-include keyword rank-1<br/>on temporal+entity shape]
    F --> G[reranker.ts: citation boosts]
    G --> H{withRelatedFacts?}
    H -->|yes| I[related-facts.ts:<br/>top-N entities, ≥ MIN_CORROBORATION]
    H -->|no| J
    I --> J{withRelatedExemplars?}
    J -->|yes| K[related-exemplars.ts:<br/>CodeRankEmbed NN search]
    J -->|no| L[pointer-block.ts:<br/>render block]
    K --> L
    L --> M[Injected into agent context]
    B -->|generative| N[Skip recall<br/>no pointer block]

Key building blocks:

  • Query rewrite — vague natural-language prompts ("that pgvector thing") are rewritten into keywordQuery and semanticQuery phrasings. The prompt is strict-JSON; any parse error throws LLMUnreachableError so the caller falls back to the raw query (rewrite-prompt.ts:1-80).
  • Query shape — when a query contains a temporal marker *and* a named-entity token, the keyword rank-1 session is force-included in merged top-k. Probe data shows 17.3 % of temporal-reasoning queries match this shape, with 0 % false-positive rate on single-session preference/assistant classes (query-shape.ts:1-35).
  • Related facts — walks the top hits, dedupes entities, fetches non-superseded facts, computes corroboration counts, and filters to MIN_CORROBORATION ≥ 2. Fails closed: any error returns [] and the pointer block degrades to session-only (related-facts.ts:1-30). Tunable via NLM_HOOK_FACT_LIMIT, NLM_HOOK_FACT_MIN_CORROBORATION, NLM_HOOK_FACT_MIN_CONFIDENCE.
  • Related exemplars — embeds the user's task in CodeRankEmbed space, returns at most k=2 exemplars within maxDistance (default 1.0, env NLM_EXEMPLAR_RECALL_MAX_DISTANCE) (related-exemplars.ts:1-30).
  • Pointer block — pure renderer; pointer-only by design. The footer names all four NLM MCP tools because the pointer block is the only cross-runtime surface for teaching the tool inventory (pointer-block.ts:1-25).

3. Classification

The classifier labels each ingested session with an entity, kind, and coherence score. v0.18.0 added *hierarchical* classification so oversized sessions are split, chunked, and re-labeled (hierarchical-classify.ts). v0.19.0 layered JSON-resilience: retry-on-parse-failure plus per-chunk tolerance so one malformed chunk does not poison the whole classification batch.

prompt.ts carries the system prompt that drives the LLM. reclassify-oversized.ts is the recovery path invoked when an ingest finishes but the classified payload exceeds model context — the session is staged for the scheduler rather than blocked at ingest time.

4. Signals, Hooks & Scheduling

The hook layer is the policy boundary between "the agent is about to type something" and "we should interfere."

  • GateclassifyPrompt() is a conservative *excluder*: only high-precision generative openers (write, draft, create, compose, generate, brainstorm, design, outline, sketch, invent, rename, come up with) short-circuit to generative; everything else evaluates recall. A false "generative" wrongly skips recall — the exact failure this gate fixes (gate.ts:1-25).
  • Citation detection — for each surfaced recall ID, the system scans two channels in priority order: tool_use (the model invoked an NLM MCP tool referencing the ID — the strong signal) and prose (the ID substring appears in the response text). Both channels carry a kind tag for downstream log analysis (citation-detect.ts:1-40). This is the training substrate for a future learned reranker.
  • Scheduler / auto-chunkv0.20.0 moved oversized-session handling out of ingest-time into a scheduled scan (scan-once.ts) that picks up sessions flagged by reclassify-oversized.ts and chunk-classifies them asynchronously. This unblocks ingest and bounds the blast radius of any single oversized session (releases/v0.20.0).

5. Telemetry Surfaces

The UI exposes adoption and coverage so operators can distinguish a corpus-coverage problem from an agent-adoption problem. Recall.tsx reports session-recall and fact-recall hit rate, by-source usage, top queries, and top subjects/predicates; cached refetches reuse stale data so users do not see reflow (Recall.tsx:1-50). Pulse.tsx shows per-runtime activity (this-week, last-week, total sessions, last session age) plus top topics (Pulse.tsx).

See Also

Source: https://github.com/pbmagnet4/nlm-memory / Human Manual

Data Storage, Adapters & Supersedence

Related topics: Overview & System Architecture, Recall, Classification & Signals, Deployment, Configuration & Operations

Section Related Pages

Continue reading this section for the full explanation and source context.

Related topics: Overview & System Architecture, Recall, Classification & Signals, Deployment, Configuration & Operations

Data Storage, Adapters & Supersedence

nlm-memory persists AI operator session data through a port-and-adapter storage layer that lets the daemon run on either SQLite (default, local-first) or PostgreSQL (with the pgvector extension). The supersedence subsystem is woven directly into the session model so that re-ingests, mechanical re-runs, and corrections leave a navigable graph rather than orphaned duplicates.

Architecture: Composition Root and Ports

The CLI is the single composition root that knows about every concrete backend. src/cli/nlm.ts is described in its own header as "the one file that knows about every concrete implementation: SqliteSessionStore (storage), OllamaClient (LLM), Hono (HTTP), McpServer (MCP)" — swapping a backend means editing this file, not anything inside core/. Source: src/cli/nlm.ts:11-14

The daemon wires up multiple storage adapters side by side, not a single one. In the createApp(...) call the composition root passes store, factStore: facts, exemplarStore: storage.exemplars, and a code embedder, so each port (sessions, facts, code exemplars) has its own concrete backend instance. Source: src/cli/nlm.ts:88-101

Two backing databases are first-class:

DependencyRoleSource
better-sqlite3 ^12.10.0Default local-first enginepackage.json:36
pg ^8.21.0 + pgvector ^0.3.0Optional Postgres backend with vector typepackage.json:38-39
sqlite-vec ^0.1.6SQLite-side vector searchpackage.json:40

Postgres support shipped in stages: v0.14.0 introduced the Postgres code-exemplar backend alongside the runtime test for the webhook ingestSession branch; v0.14.1 added facts.retired_at to the PG schema after a v0.14.0 gap was discovered. Source: v0.14.1 release notes, v0.14.0 release notes

Adapter Parity: Code Exemplars

The exemplar lane is a useful case study in adapter parity. SqliteCodeExemplarStore and PgCodeExemplarStore share the same row shape and the same deterministic primary-key scheme — sha256(install_scope|repo|code_hash|outcome)[:16] — guaranteeing one row per unique (scope, repo, exact-code-content, outcome) triple. The row interface is identical across both backends:

id, install_scope, signal_id, session_id, repo, model, lang,
task_context, code, code_hash, outcome, git_sha, survived, ts,
created_at, retired_at, label_source

Source: src/core/storage/sqlite-code-exemplar-store.ts:11-27, src/core/storage/pg-code-exemplar-store.ts:12-29

Insert-side equivalence is enforced by an insertMany batch path on the SQLite side and a parameter-builder on the PG side that pre-computes the same id from the same inputs. Source: src/core/storage/sqlite-code-exemplar-store.ts:51-58, src/core/storage/pg-code-exemplar-store.ts:31-58. The exemplars property is always attached to the storage object the composition root hands to the app, so the same code path reaches the exemplars regardless of the active backend. Source: src/cli/nlm.ts:97-100

Supersedence: Session Graph and Invariants

Sessions are not a flat log. The dataset contract surfaces five status values: active | idle | closed | superseded | replaced, plus explicit supersedes, superseded_by, replaces, and replaced_by edges. The Thread UI (#299) collapses replaced predecessors behind an "earlier versions" affordance, and the dataset carries the replaces / replaced_by fields alongside supersedes / superseded_by because they encode mechanically different relations. Source: src/ui/lib/dataset.ts:23-44

A supersede edge points from the new session to the prior one, so traversal is "follow the outgoing edge to find the predecessor." The two relations are deliberately distinct:

  • supersedes — semantic: this session took over from that one.
  • replaces — mechanical re-ingest relation; same content, re-processed.

A closed status can be coerced back to non-closed by supersede or replace events; the doctor command's --fix mode repairs "orphaned superseded/replaced sessions" (invariant I2) by restoring them to closed. Source: src/cli/nlm.ts:42-49

The nlm doctor subcommand enforces the schema's referential integrity: --fix repairs the two mechanically safe violations — I1: self-loop edges are deleted, and I2: orphaned superseded/replaced sessions are restored to closed. On Postgres, repairs route through the PG-aware storage path. Source: src/cli/nlm.ts:46-49

How Recall Consumes the Graph

RecallService is the use case that filters, scores, and ranks. It depends only on ports — SessionStore, LLMClient, CodeEmbedder, CodeExemplarStore, FactStore — and no framework, SQLite, or HTTP import appears inside core/recall/. Tests substitute fake adapters. Source: src/core/recall/recall-service.ts:7-19

Two enrichments ride on top of recall and touch storage:

  1. Related factspickRelatedFacts walks the top-N hits, dedupes their entities, fetches current (non-superseded) facts from FactStore.list, and filters to those corroborated across ≥ NLM_HOOK_FACT_MIN_CORROBORATION (default 2) sessions. This is what populates the "Known facts" section of the pointer block that the recall hook injects into a live Claude Code / Codex turn. Source: src/core/recall/related-facts.ts:14-23
  2. Related exemplars — symmetric logic on CodeExemplarStore, gated by the NLM_CODE_EXEMPLARS_ENABLED flag at request time. v0.16.0 made code-exemplar recall passive (Phase 2), so it surfaces automatically when a query is rewritten and the flag is on. Source: v0.16.0 release notes

The pointer block itself is pointer-only by design — ids and labels, no session content — and the footer names all four NLM MCP tools because the pointer block is the only cross-runtime distribution surface for teaching the tool inventory. Source: src/core/hook/pointer-block.ts:1-14

Failure Modes and Repair

Two storage-layer failure modes recur in the community and have shipped fixes:

  • PG schema drift. v0.14.0 introduced the Postgres exemplar backend but missed facts.retired_at; v0.14.1 added the column. v0.13.0 also cleared app.ts of rawDb/cast escape hatches by routing through registries, actions, and data-management. Source: v0.14.1 release notes, v0.13.0 release notes
  • Oversized sessions on ingest. v0.20.0 added a scheduler that auto-chunks sessions that exceed classifier capacity (#341). Combined with the v0.19.0 classifier JSON-resilience (retry + per-chunk tolerance) and v0.18.0 hierarchical classification + oversized-session recovery, the ingestion path now degrades gracefully instead of failing the whole session. Source: v0.20.0 release notes, v0.19.0 release notes, v0.18.0 release notes
  • MCP transport gaps. v0.13.1 wired the code-exemplar lane into the HTTP /mcp transport after the stdio path exposed it; without that fix, container agents (e.g. Hermes WebUI) couldn't reach recall_code. Source: v0.13.1 release notes

Operator-facing observability lives in the Recall page: it reports session hit rate, fact hit rate, by-source breakdown, top queries, top subjects, and top predicates, so a coverage problem (zero hits because the corpus lacks the answer) can be distinguished from an adoption problem (the agent never asks). Source: src/ui/pages/Recall.tsx:12-34

See Also

Source: https://github.com/pbmagnet4/nlm-memory / Human Manual

Deployment, Configuration & Operations

Related topics: Overview & System Architecture, Data Storage, Adapters & Supersedence

Section Related Pages

Continue reading this section for the full explanation and source context.

Section Headline subcommands

Continue reading this section for the full explanation and source context.

Section Install hardening and recovery (v0.13.0+)

Continue reading this section for the full explanation and source context.

Section Environment variables

Continue reading this section for the full explanation and source context.

Related topics: Overview & System Architecture, Data Storage, Adapters & Supersedence

Deployment, Configuration & Operations

nlm-memory (v0.20.0) is a local-first non-linear memory operating system for AI operators (package.json:2). It is a Node 22 ESM application whose composition root lives in a single CLI entry point (src/cli/nlm.ts:1-30). This page documents how the daemon is deployed, configured, and operated day-to-day: the CLI subcommand surface, environment variables and .env file, storage backends and migrations, integrity tooling (doctor/verify/codex-repair), backups, and the runtimes it integrates with.

CLI Surface and Lifecycle

The CLI is the one file that knows about every concrete implementation (storage, LLM provider, HTTP framework, MCP transport). Every other module depends on ports (src/cli/nlm.ts:14-19). The published bin is dist/cli/nlm.js (package.json:39).

Headline subcommands

CommandPurpose
nlm startBoot the HTTP server on $NLM_PORT (default 3940). Dev script: tsx watch src/cli/nlm.ts start (package.json:42-44).
nlm mcpRun as an MCP stdio server for ~/.mcp.json wiring.
nlm setupInteractive first-run wizard (the recommended entry point).
nlm migrateRun pending migrations against the canonical SQLite.
nlm recallOne-shot recall query from the shell (debugging).
nlm install / nlm uninstallInstall or remove the macOS LaunchAgent for auto-start on login.
nlm hook install / nlm hook uninstallAdd or remove the recall hook in Claude Code.
nlm connect <runtime>Wire MCP into Claude Code, Codex, Hermes, Hermes Agent, Cursor, Windsurf, Pi, OpenCode.
nlm disconnect <runtime>Remove the corresponding MCP/plugin block.
nlm doctorCheck DB integrity invariants and install/runtime health; --fix repairs safe violations (self-loop edges I1, orphaned superseded/replaced sessions I2).
nlm digestPrint a daily-activity digest, optionally --telegram to post.
nlm backfill-factsOne-shot classification of historical sessions into the FactStore (Phase B.5), resumable via --state.

Source: header doc-comment at src/cli/nlm.ts:16-37; nlm doctor definition at src/cli/nlm.ts:120-140; backfill-facts at src/cli/nlm.ts:200-220.

Install hardening and recovery (v0.13.0+)

The release notes for v0.13.0 introduced doctor, verify, and codex-repair subcommands plus daily backups. nlm doctor --fix is safe-by-default and only repairs mechanically safe violations: it deletes self-loop edges (invariant I1) and restores orphaned superseded/replaced sessions to closed (invariant I2) (src/cli/nlm.ts:124-130). The daemon also reports a staleNlmMemoryTs flag for Codex configs that still reference the old @nlm-memory-ts trust entry, so it can be cleaned up without breaking Codex's own registry (src/cli/nlm.ts:90-110).

Configuration

Configuration is layered: process environment variables first, then a single .env file at ~/.nlm/.env that is autoloaded when a non-Ollama provider is selected. The CLI probes this path on every doctor run (src/cli/nlm.ts:95-115).

Environment variables

VariableDefaultPurpose
NLM_PORT3940HTTP port the daemon binds to.
NLM_CLASSIFIERollamaProvider for classification: ollama keeps the daemon local-first and key-free; deepseek is faster (src/cli/nlm.ts:148-160).
NLM_CLASSIFIER_MODELqwen3.5:4b (ollama) / deepseek-v4-flash (deepseek)Classifier model. qwen3.5:4b scored 74.4% decision-precision in the 2026-06-11 eval and requires think:false (src/cli/nlm.ts:150-160).
OLLAMA_URLhttp://localhost:11434Ollama endpoint.
NLM_MCP_TOKEN_(unset)_Shared-secret token for the MCP HTTP transport; absence is reported by doctor as env.hasMcpToken: false (src/cli/nlm.ts:100-115).
NLM_ADAPTERS_(unset)_Comma-separated name filter to force a subset of source adapters during development.
~/.nlm/.envThe env file path is hard-coded; doctor reports its existence (src/cli/nlm.ts:110-115).

engines.node requires >=22.0.0 (package.json:6-8).

Storage, Backups, and Data Operations

The composition root picks one of two stores at startup: SqliteStorage or PgStorage (src/cli/nlm.ts:20-30). Auxiliary stores (e.g. code exemplars) are available in both backends.

Storage backends

BackendModuleNotes
SQLite (canonical)src/core/storage/sqlite-storage.tsDefault. Migrations live under migrations/ and are shipped in package.json files (package.json:30-40).
Postgressrc/core/storage/pg-storage.tsUsed when a Postgres DSN is configured; v0.14.1 added facts.retired_at to the PG schema.

The exemplar stores share a deterministic primary-key convention: sha256(install_scope|repo|code_hash|outcome)[:16] guarantees one row per unique (scope, repo, exact-code-content, outcome) triple (src/core/storage/sqlite-code-exemplar-store.ts:18-30; same contract mirrored in src/core/storage/pg-code-exemplar-store.ts:35-50). Retired rows are flagged with retired_at rather than deleted (src/core/storage/sqlite-code-exemplar-store.ts:80-100).

Backups and restore

Daily backups were added in v0.13.0. The CLI imports runRollingBackup, listBackupDates, resolveBackup, stageRestore, and applyPendingRestore (src/cli/nlm.ts:25-30), which together implement a rolling backup window, on-demand restore staging, and atomic apply. Recovery flow: list candidates → resolve a date → stage → apply.

Migrations

nlm migrate runs the pending set against the canonical SQLite store (src/cli/nlm.ts:24-26). Migration SQL is shipped under migrations/ and is part of the published files list (package.json:30-35). The schema must stay aligned with code: v0.14.1 was a one-line fix(pg) adding facts.retired_at, illustrating that PG-side schemas are still tuned release-by-release.

flowchart LR
  A[Sources: Claude Code, Codex, Hermes, Pi, jsonl-generic, webhook] --> B[nlm-memory daemon]
  B --> C[Storage: SQLite or Postgres]
  C --> D[Migrations: nlm migrate]
  C --> E[Daily backups: runRollingBackup]
  E --> F[Restore: stageRestore / applyPendingRestore]
  B --> G[doctor / verify / codex-repair]
  G -->|--fix| C
  B --> H[HTTP :3940 + MCP stdio]
  H --> I[Agent runtimes via nlm connect]

Runtime Integrations and Recall Wiring

nlm-memory ingests from six source kinds: claude-code, codex, hermes, pi, jsonl-generic, and webhook (src/ui/lib/registries.ts:30-40). Each enabled row in the sources table maps to one adapter; detection still gates registration, so a row pointing at a missing directory will not poll (src/cli/nlm.ts:165-175). For each runtime, the CLI exposes paired connect/disconnect commands that write or remove the relevant config (e.g. MCP block in ~/.mcp.json, Codex marketplace plugin, Hermes plugin dir).

Provider kinds

Providers are pluggable in the same registry shape: deepseek, ollama, openai, anthropic, openrouter, openai-compatible (src/ui/lib/registries.ts:55-75). HTTP responses always carry redacted rows — no api_key, no source token — except the one-time reveal on insert or regenerate.

Live recall via the hook

When the recall hook runs in live mode, the daemon injects a pointer block (## Possibly-relevant prior sessions) followed by an NLM tools: footer that names all four MCP tools — the pointer block is the only cross-runtime distribution surface for teaching the tool inventory (src/core/hook/pointer-block.ts:1-15). When fact recall is wired in, a ## Known facts about top entities section is inserted with <subject> <predicate>: <value> [N sessions] rows. On the model side, stripInjectedContext recognises two specific block headers plus the NLM tools: footer and removes the entire block, collapsing the blank-line runs that removal leaves behind (src/core/hook/strip-injected-context.ts:1-30). The rewrite step that feeds recall (query rewriting, Spec C) parses strict JSON from the classifier; parse failures throw so callers fail open to the raw query rather than embedding garbage (src/core/recall/rewrite-prompt.ts:1-20).

Recent Operational Changes (Release Notes)

  • v0.20.0feat(scheduler): auto-chunk oversized sessions on ingest (#341); classifier JSON resilience in v0.19.0; hierarchical classification + oversized-session recovery in v0.18.0.
  • v0.16.0 — Passive code-exemplar recall (Phase 2); auto-capture from committed sessions shipped in v0.15.0 (Phase 1).
  • v0.14.0 / v0.14.1 — Postgres backend for exemplars plus recall_code MCP guarantee; facts.retired_at PG-schema fix.
  • v0.13.0 / v0.13.1doctor/verify/codex-repair install hardening, daily backups, and the code-exemplar lane wired into HTTP /mcp transport.

See Also

Source: https://github.com/pbmagnet4/nlm-memory / Human Manual

Doramagic Pitfall Log

Source-linked risks stay visible on the manual page so the preview does not read like a recommendation.

medium Configuration risk requires verification

May increase setup, validation, or first-run risk for the user.

medium Capability evidence risk requires verification

May increase setup, validation, or first-run risk for the user.

medium Maintenance risk requires verification

May increase setup, validation, or first-run risk for the user.

medium Security or permission risk requires verification

May increase setup, validation, or first-run risk for the user.

Doramagic Pitfall Log

Found 7 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Configuration risk - Configuration risk requires verification.

1. Configuration risk: Configuration risk requires verification

  • Severity: medium
  • Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: capability.host_targets | https://github.com/pbmagnet4/nlm-memory

2. Capability evidence risk: Capability evidence risk requires verification

  • Severity: medium
  • Finding: README/documentation is current enough for a first validation pass.
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: capability.assumptions | https://github.com/pbmagnet4/nlm-memory

3. Maintenance risk: Maintenance risk requires verification

  • Severity: medium
  • Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: evidence.maintainer_signals | https://github.com/pbmagnet4/nlm-memory

4. Security or permission risk: Security or permission risk requires verification

  • Severity: medium
  • Finding: no_demo
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: downstream_validation.risk_items | https://github.com/pbmagnet4/nlm-memory

5. Security or permission risk: Security or permission risk requires verification

  • Severity: medium
  • Finding: no_demo
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: risks.scoring_risks | https://github.com/pbmagnet4/nlm-memory

6. Maintenance risk: Maintenance risk requires verification

  • Severity: low
  • Finding: issue_or_pr_quality=unknown。
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: evidence.maintainer_signals | https://github.com/pbmagnet4/nlm-memory

7. Maintenance risk: Maintenance risk requires verification

  • Severity: low
  • Finding: release_recency=unknown。
  • User impact: May increase setup, validation, or first-run risk for the user.
  • Recommended check: Reproduce the official install and quickstart path in an isolated environment.
  • Evidence: evidence.maintainer_signals | https://github.com/pbmagnet4/nlm-memory

Source: Doramagic discovery, validation, and Project Pack records

Community Discussion Evidence

These external discussion links are review inputs, not standalone proof that the project is production-ready.

Sources 11

Count of project-level external discussion links exposed on this manual page.

Use Review before install

Open the linked issues or discussions before treating the pack as ready for your environment.

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using nlm-memory with real data or production workflows.

Source: Project Pack community evidence and pitfall evidence