# https://github.com/nitekeeper/memex Project Manual Generated at: 2026-06-27 01:50:33 UTC ## Table of Contents - [Memex Overview and Three-Layer Architecture](#page-1) - [Memex Dashboard, 3D Knowledge Graph, and Federated Search](#page-2) - [Core Data Operations: Index, Brain, Internal Agents, and Code Graph](#page-3) - [Installation, Bootstrap, Deployment, and Extensibility](#page-4) ## Memex Overview and Three-Layer Architecture ### Related Pages Related topics: [Core Data Operations: Index, Brain, Internal Agents, and Code Graph](#page-3), [Installation, Bootstrap, Deployment, and Extensibility](#page-4)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/nitekeeper/memex/blob/main/README.md) - [CLAUDE.md](https://github.com/nitekeeper/memex/blob/main/CLAUDE.md) - [USER_GUIDE.md](https://github.com/nitekeeper/memex/blob/main/USER_GUIDE.md) - [skills/run/SKILL.md](https://github.com/nitekeeper/memex/blob/main/skills/run/SKILL.md) - [skills/README.md](https://github.com/nitekeeper/memex/blob/main/skills/README.md) - [internal/brain/ingest/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/ingest/SKILL.md) - [internal/brain/capture/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/capture/SKILL.md) - [internal/brain/synthesize/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/synthesize/SKILL.md) - [internal/index/write/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/index/write/SKILL.md) - [internal/index/archive/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/index/archive/SKILL.md) - [internal/core/insert/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/insert/SKILL.md) - [internal/core/get-agent/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/get-agent/SKILL.md) - [internal/steward/dashboard/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/steward/dashboard/SKILL.md) - [scripts/dashboard.py](https://github.com/nitekeeper/memex/blob/main/scripts/dashboard.py) - [scripts/__init__.py](https://github.com/nitekeeper/memex/blob/main/scripts/__init__.py) - [prompts/librarian.md](https://github.com/nitekeeper/memex/blob/main/prompts/librarian.md) - [prompts/reference_librarian.md](https://github.com/nitekeeper/memex/blob/main/prompts/reference_librarian.md) - [prompts/synthesizer.md](https://github.com/nitekeeper/memex/blob/main/prompts/synthesizer.md)
# Memex Overview and Three-Layer Architecture ## Purpose and Scope Memex is a **personal knowledge runtime and shared memory plane for the agent fleet**. It hosts articles, notes, and syntheses that the user captures, and serves as a durable memory layer for AI agents (Claude Code and consumer plugins) that need to recall prior decisions, indexed sources, and cross-store relationships. Source: [README.md:1-3]() The product is **plugin-agnostic**: any Claude Code plugin can register its own SQLite store with Memex via the Core layer and let its documents flow through the same indexing pipeline. Consumer plugins such as project-management or workspace tools use Memex for persistent memory; Memex itself does not know or care which plugin wired it in. Source: [README.md:3-5]() Development context: most code in this repository is developed collaboratively with Claude Code (commits are typically `Co-Authored-By: Claude Opus …`), and the v2.0 design + implementation plans were pair-written. The originating design body is recoverable via git history pre-2026-05-26 or `memex:run ask` (the canonical store is Memex itself going forward). Source: [README.md:5-9](), [CLAUDE.md:1-15]() The Memex repo is **Layer 2** of Skill Atelier — it is the product, not the framework. Framework-level changes belong to the upstream `skill-atelier` repository; changes to Memex files commit here. Source: [CLAUDE.md:17-22]() ## The Three Layers Memex is organized as **three layers** plus two derived subsystems, all behind a single Claude-Code-visible skill called `memex:run`. Source: [README.md:11-21](), [CLAUDE.md:5-7]() ```mermaid graph TD User([User / Claude Code session]) -->|intent| Skill[memex:run skill] Skill -->|routes to| Internal[31 internal procedures
internal/<category>/<name>/SKILL.md] Internal --> Brain Internal --> Index Internal --> Core Brain[Brain
opinionated second-brain skills
ingest, ask, capture, synthesize, lint] Index[Index + 5 internal agents
Librarian, Reference Librarian,
Archivist, DBA, Data Steward
FTS5 + embeddings + relations] Core[Core
CRUD substrate
Provisions arbitrary SQLite stores
from consumer-supplied SQL] Brain -->|writes via| Index Index -->|persists via| Core Core -->|hosted in| Stores[(~/.memex/*.db
article.db, index.db,
code_graph.db, ...)] Stores --> Dashboard[Memex Dashboard
local read-only web UI] Stores --> CodeGraph[Code graph subsystem] Stores --> GraphRAG[GraphRAG community reports] ``` **Layer 1 — Memex Brain.** The opinionated second-brain skill layer exposed as user-facing intents: `ingest`, `ask`, `capture`, `lint`, `synthesize`. It stores articles, free-form notes, and syntheses in `~/.memex/article.db`. Each operation is a small orchestration script in `scripts/brain.py` plus a SKILL.md recipe. Source: [README.md:11-13](), [internal/brain/ingest/SKILL.md:1-12](), [internal/brain/capture/SKILL.md:1-9](), [internal/brain/synthesize/SKILL.md:1-18]() **Layer 2 — Memex Index + 5 internal agents.** The mandatory write-path gateway. The five internal agents are **Librarian, Reference Librarian, Archivist, Database Administrator, and Data Steward**. The Librarian classifies incoming documents and assigns an `index_id` plus relations; the Reference Librarian resolves ask queries into FTS5 + vector query plans; the Archivist canonicalizes and stores raw payloads; the DBA manages stores and migrations; the Data Steward exposes read-side reporting such as the dashboard. Source: [README.md:13-16](), [internal/steward/dashboard/SKILL.md:1-12](), [prompts/librarian.md:1-30](), [prompts/reference_librarian.md:1-20]() **Layer 3 — Memex Core.** A schema-agnostic CRUD substrate that provisions and hosts arbitrary SQLite stores from consumer-supplied SQL migration files. It manages the `~/.memex/` directory, the `registry.json` of registered stores, and the per-store `migrations` ledger (which self-heals on ledger/schema desync as of v2.12.2). Source: [README.md:16-19](), [scripts/__init__.py:1-2](), [internal/core/insert/SKILL.md:1-5]() ## Skill Routing and the `memex:run` Entry Point Memex registers a **single** Claude Code skill — `memex:run`. There is no top-level `memex:brain:ingest` or `memex:brain:ask` skill; instead the user invokes `memex:run` and expresses intent in natural language. The router reads the matching internal procedure under `internal///SKILL.md` (categories: `core`, `index`, `brain`, `steward`, `dba`, `embed`, `codegraph` — 31 procedures total) and follows it inline. Source: [CLAUDE.md:5-9](), [USER_GUIDE.md:7-13](), [skills/run/SKILL.md:1-7]() Every top-level invocation runs a **Step 0 preflight** before routing: it verifies a usable Python interpreter and that `~/.memex/` is bootstrapped. If either is missing, Step 0 ends the turn (no routing, no fallback). On first run, when `~/.memex/` is absent, Step 0 prints a consent block listing what will be created and prompts `(y/n)`; on `y` it auto-bootstraps, seeding the 5 internal agents and writing `article.db` and `registry.json`. Source: [USER_GUIDE.md:1-6](), [skills/run/SKILL.md:7-9]() The visibility model keeps the plugin under Claude Code's **1% skill-description budget**: only `memex:run` is exposed in `plugin.json`; internal SKILL.md files are read on demand by the agent. Source: [CLAUDE.md:5-9]() ## Derived Subsystems and Observability Two subsystems build on the three core layers: - **Code graph** — a separate code-navigation store (`code_graph.db`) for consumer plugins. An external extractor (`graphify`) produces the graph; Memex ingests it via `internal/codegraph/ingest` + `scripts/code_graph.py` + `db/code_graph.sql` and serves bounded, deterministic queries (`where_is`, `callers`, `dependencies`, `neighbors`, `module_map`) through `internal/codegraph/query`. Memex is **EXTRACTOR-EXTERNAL** — it does no source/AST parsing itself (memex#34). Source: [README.md:21-25]() - **GraphRAG community summarization** — a derived knowledge layer over indexed documents: a key-free lexical similarity graph, hierarchical community detection (`scripts/communities.py`), and bottom-up per-community reports persisted to `index.db` (`internal/brain/community-report`, `scripts/agents/community_reporter.py`) that power `global`-mode ask. Source: [README.md:25-29]() **Observability** is provided by `memex:steward:dashboard`, a loopback-only stdlib HTTP server that opens every registered store **read-only**, aggregates a JSON summary, and serves a self-contained HTML page plus a `/api/summary` JSON endpoint. The dashboard ships with keyword document search (v2.15.0), a Markdown-rendered content overlay (v2.16.0), an interactive 3D knowledge graph at `/graph` (v2.14.0), and grouped search across documents, agents, and code (v2.17.0). Because it writes to nothing, it is **not** a Librarian write-path bypass. Source: [internal/steward/dashboard/SKILL.md:1-12](), [scripts/dashboard.py:1-22]() ## See Also - [Memex Installation and First-Run Bootstrap](USER_GUIDE.md) - [Brain Skills: ingest / capture / ask / synthesize](internal/brain/ingest/SKILL.md) - [Index Write Path and the Librarian Subagent](internal/index/write/SKILL.md) - [Data Steward Dashboard](internal/steward/dashboard/SKILL.md) - [Core CRUD Substrate](internal/core/insert/SKILL.md) - [Archive (Archivist) Primitive](internal/index/archive/SKILL.md) --- ## Memex Dashboard, 3D Knowledge Graph, and Federated Search ### Related Pages Related topics: [Memex Overview and Three-Layer Architecture](#page-1), [Core Data Operations: Index, Brain, Internal Agents, and Code Graph](#page-3)
Related Source Files The following source files were used to generate this page: - [scripts/dashboard.py](https://github.com/nitekeeper/memex/blob/main/scripts/dashboard.py) - [internal/steward/dashboard/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/steward/dashboard/SKILL.md) - [README.md](https://github.com/nitekeeper/memex/blob/main/README.md) - [internal/index/write/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/index/write/SKILL.md) - [internal/brain/ingest/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/ingest/SKILL.md) - [internal/brain/graph-rebuild/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/graph-rebuild/SKILL.md) - [internal/core/get-agent/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/get-agent/SKILL.md) - [internal/core/insert/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/insert/SKILL.md) - [CLAUDE.md](https://github.com/nitekeeper/memex/blob/main/CLAUDE.md) - [prompts/librarian.md](https://github.com/nitekeeper/memex/blob/main/prompts/librarian.md)
# Memex Dashboard, 3D Knowledge Graph, and Federated Search ## Overview and Purpose The Memex Dashboard is a **local, read-only observability surface** that summarizes everything stored across Memex's federated stores. It was introduced in **v2.13.0** as a `memex:run` intent routed to the Data Steward (`memex:steward:dashboard`), backed deterministically by `scripts/dashboard.py` Source: [internal/steward/dashboard/SKILL.md](). It exposes per-store row counts, the federated index, knowledge communities, Brain captures, the code-navigation graph, and the agent registry — all without writing to any store. Subsequent releases layered three interactive features on top of the static dashboard: | Release | Feature | |---|---| | v2.13.0 | Initial read-only summary dashboard | | v2.14.0 | Interactive 3D knowledge-graph view at `/graph` | | v2.15.0 | Keyword document search + click-to-read content overlay | | v2.16.0 | Markdown rendering inside the content overlay | | v2.17.0 | Search now spans Documents / Agents / Code (grouped) | All three are bounded by the same design constraints: **stdlib only** (no Flask/FastAPI/jinja), **read-only** (`mode=ro` on every store), **loopback binding** (`127.0.0.1`), and **no server-side HTML interpolation** Source: [scripts/dashboard.py](). ## Dashboard Architecture The dashboard is a single self-contained HTML page served over `http.server`, with a JSON `/api/summary` endpoint that aggregates data from every registered store plus the fixed-path `code_graph.db` Source: [scripts/dashboard.py](). It is **not** a Librarian write-path bypass — spec §6 / M3 govern document writes; this is pure read-side reporting Source: [internal/steward/dashboard/SKILL.md](). ```mermaid flowchart LR User[User intent:
"show me a dashboard"] -->|memex:run| Skill[memex:steward:dashboard] Skill -->|dispatch| Dash[scripts/dashboard.py] Dash -->|mode=ro| Stores[(Registered SQLite stores)] Dash -->|mode=ro| CodeGraph[(code_graph.db)] Dash -->|/api/summary| Browser[Self-contained HTML page] Browser -->|render| Docs[Documents view] Browser -->|render| Search[Keyword search box] Browser -->|render| Graph3D[3D graph at /graph] Browser -->|render| Overlay[Markdown content overlay] ``` The server binds `127.0.0.1` and refuses a non-local bind unless `--allow-non-local` is passed explicitly Source: [internal/steward/dashboard/SKILL.md](). The default port is `8765`, auto-incrementing if busy up to +20 Source: [internal/steward/dashboard/SKILL.md]()-20. ### When to use vs. when not to - Use the dashboard for: quick health glances, visual previews before/after large ingests, and sharing what's in Memex with collaborators. - Use `memex:steward:audit` instead for deep integrity checks (orphans, broken relations, schema drift) Source: [internal/steward/dashboard/SKILL.md](). ## Federated Search Prior to **v2.17.0**, dashboard keyword search covered only the federated document index. Search now spans **three surfaces**, grouped in the results list: - **Documents** — the federated index (FTS5 + LIKE fallback), with highlighted snippets - **Agents** — the agents registry, previously unsearchable from the dashboard - **Code** — the code-navigation graph (modules, symbols, relations) The Documents surface inherits from `memex:index:search` and the write path that produced it: documents are persisted via `memex:index:write` or its Brain wrappers (`memex:brain:ingest`, `memex:brain:capture`), each of which routes through the Librarian subagent for classification Source: [internal/index/write/SKILL.md]() and Source: [internal/brain/ingest/SKILL.md](). The Librarian assigns a stable `index_id`, a `key`, a `domain`, a `searchable` text payload, and a `relations` list — all the fields the dashboard ranks against Source: [prompts/librarian.md](). The Agents surface is backed by the agent registry. Agent profiles are fetched via `memex:core:get-agent`, which calls `scripts/agents/__init__.py:get_agent(db_path, agent_id)` and returns the agent's full profile including its markdown body Source: [internal/core/get-agent/SKILL.md](). New agents are registered through `memex:core:insert`, which writes a row to the `agents` table of a registered store via `scripts/stores.py:insert(name, table, row)` Source: [internal/core/insert/SKILL.md](). The Code surface draws from `code_graph.db`, which Memex ingests from an external extractor (`graphify`); Memex itself does **no source/AST parsing** Source: [README.md](). ## 3D Knowledge Graph View The dashboard gains an **Obsidian-style knowledge graph, orbitable in 3D**, reached from a "◉ 3D graph" link on the dashboard or directly at `/graph` Source: [internal/steward/dashboard/SKILL.md](). Documents are nodes, relations are edges, and nodes are **colored by GraphRAG community** — the hierarchical community detection output of `scripts/communities.py` and the bottom-up reports generated by `internal/brain/community-report` Source: [internal/brain/graph-rebuild/SKILL.md](). Interactions: - **Drag** to orbit the camera - **Scroll/pinch** to zoom - **Hover** for tooltips - **Click** a node to open the document's full content in the overlay Click-to-read is the bridge between the 3D graph and the content overlay. Since v2.16.0, that overlay **renders Markdown** — headings, bold/italic, inline and fenced code, lists, blockquotes, horizontal rules, links, and GFM primitives — rather than displaying raw text. The overlay also **dismisses on outside-click**, so it never traps the user mid-exploration. ## Community Context and Known Limitations The dashboard features were iterated rapidly over five minor releases (v2.13.0 → v2.17.0) in mid-2026. Users following the dashboard's evolution care most about: 1. **Markdown fidelity** — early overlays showed raw text, which made reading rendered tables and code blocks impossible; v2.16.0's Markdown renderer fixed this. 2. **Cross-surface search** — finding an agent or a code module previously required leaving the dashboard; v2.17.0 collapsed all three surfaces into a single grouped result list. 3. **Read-only safety** — the dashboard is intentionally unable to mutate stores. If you need to add or change data, use the appropriate write skill (`memex:index:write`, `memex:core:insert`, or a Brain wrapper). 4. **GraphRAG freshness** — community colors reflect the **derived** graph/community/report artifacts rebuilt by `memex:brain:graph-rebuild`. If a community's color looks stale after a batch of ingests, run the rebuild; it never runs on the write path Source: [internal/brain/graph-rebuild/SKILL.md](). ## See Also - [Brain Capture, Ingest, and Synthesis](./brain-flows.md) - [Code Graph and External Extractors](./code-graph.md) - [GraphRAG Community Layer](./graphrag.md) - [Data Steward and Audit](./steward.md) --- ## Core Data Operations: Index, Brain, Internal Agents, and Code Graph ### Related Pages Related topics: [Memex Overview and Three-Layer Architecture](#page-1), [Memex Dashboard, 3D Knowledge Graph, and Federated Search](#page-2), [Installation, Bootstrap, Deployment, and Extensibility](#page-4)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/nitekeeper/memex/blob/main/README.md) - [CLAUDE.md](https://github.com/nitekeeper/memex/blob/main/CLAUDE.md) - [skills/run/SKILL.md](https://github.com/nitekeeper/memex/blob/main/skills/run/SKILL.md) - [skills/README.md](https://github.com/nitekeeper/memex/blob/main/skills/README.md) - [internal/brain/ingest/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/ingest/SKILL.md) - [internal/brain/ask/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/ask/SKILL.md) - [internal/brain/capture/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/capture/SKILL.md) - [internal/brain/synthesize/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/synthesize/SKILL.md) - [internal/index/write/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/index/write/SKILL.md) - [internal/core/insert/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/insert/SKILL.md) - [internal/core/get-agent/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/get-agent/SKILL.md) - [prompts/librarian.md](https://github.com/nitekeeper/memex/blob/main/prompts/librarian.md) - [prompts/reference_librarian.md](https://github.com/nitekeeper/memex/blob/main/prompts/reference_librarian.md) - [prompts/synthesizer.md](https://github.com/nitekeeper/memex/blob/main/prompts/synthesizer.md) - [prompts/community_reporter.md](https://github.com/nitekeeper/memex/blob/main/prompts/community_reporter.md) - [scripts/brain.py](https://github.com/nitekeeper/memex/blob/main/scripts/brain.py) - [scripts/upgrade_from_v1.py](https://github.com/nitekeeper/memex/blob/main/scripts/upgrade_from_v1.py)
# Core Data Operations: Index, Brain, Internal Agents, and Code Graph Memex is a personal knowledge runtime and shared memory plane structured as **three stacked layers** plus two derived subsystems. This page explains how documents enter Memex, how they are classified and linked, how queries are resolved, and how the optional code-navigation graph fits alongside the Brain. ## Layered Architecture Memex exposes a single Claude-Code-visible skill, `memex:run`, which routes 31 internal procedures on demand — keeping the plugin's skill-description footprint well under Claude Code's 1% budget (Source: [README.md:1-1]()). The three core layers are: 1. **Memex Brain** — opinionated second-brain skill layer (`ingest`, `ask`, `capture`, `lint`, `synthesize`). Stores articles, free-form notes, and syntheses in `~/.memex/article.db`. 2. **Memex Index + 5 internal agents** — Librarian, Reference Librarian, Archivist, Database Administrator, Data Steward. This is the mandatory write-path gateway: every document flows through the Librarian subagent before it lands in any store (Source: [README.md:2-2]()). 3. **Memex Core** — CRUD substrate. Provisions and hosts arbitrary SQLite stores from consumer-supplied SQL migrations. Schema-agnostic; any consumer plugin can declare its own tables (Source: [README.md:3-3]()). ```mermaid flowchart LR User[User / Consumer Plugin] -->|intent| Run[memex:run router] Run --> Brain[Brain layer
ingest/ask/capture/synthesize] Run --> Core[Core CRUD
internal/core/*] Brain -->|dispatch| LibAgent[Librarian subagent] Brain -->|query| RefLib[Reference Librarian subagent] LibAgent --> Index[Federated Index
index.db] RefLib --> Index LibAgent --> Store[Consumer Store
article.db / code_graph.db / ...] Core --> Store Index -->|FTS5 + vectors| Ask[Ask / Search] CodeExt[External extractor: graphify] -->|ingest| CodeGraph[code_graph.db] CodeGraph --> Index ``` The architecture enforces a **structural data/instruction boundary**: content arriving via `internal/brain/ingest`, `internal/brain/capture`, and `internal/index/write` is treated as data, never as instructions. The Librarian and Archivist must delimit ingested payloads and never concatenate them into a system-role section (Source: [CLAUDE.md:1-1]()). ## The Brain Layer The Brain layer provides five user-facing procedures, each implemented as a `SKILL.md` orchestrator that prepares a Task-tool prompt and dispatches a bounded subagent. ### Ingest and Capture `memex:brain:ingest` ingests an external article or source. It hashes the canonical content for rerun safety — re-ingesting the same body yields `status="skipped"` and stops before any subagent dispatch (Source: [internal/brain/ingest/SKILL.md:1-1]()). `memex:brain:capture` is the lighter cousin: no source URL, no raw archive, no dedup — every capture is a fresh thought that still flows through the Librarian for classification (Source: [internal/brain/capture/SKILL.md:1-1]()). ### Ask (Query) `memex:brain:ask` supports three modes (Source: [internal/brain/ask/SKILL.md:1-1]()): | Mode | Behavior | |---|---| | `corpus` (default) | Standard FTS5 + vector search over the federated index | | `global` | Map-reduce over `community_reports` — MAP each report via a Haiku subagent, REDUCE sort-desc and budget-fill | | `local` | Seed top docs by cosine similarity, expand the `relations` neighborhood, attach community reports | Both `global` and `local` depend on the derived community-reports layer; if absent, the skill reports `no_reports` / `no_signal` rather than failing — degraded but not broken. ### Synthesize `memex:brain:synthesize` is the only Brain flow with **two LLM steps**: a Synthesizer subagent produces a citing prose synthesis from a list of source index_ids, then a Librarian subagent classifies the synthesis as a new document. Sources are concatenated under a 50,000-char budget to keep the Task prompt bounded, mirroring the budget-fill loops in community_reporter and global_ask_reduce_prepare (Source: [scripts/brain.py:1-1]()). ## Memex Index and the Five Internal Agents The Index is the mandatory write-path gateway. Every document must be classified by the Librarian subagent before persistence. ### The Librarian Subagent The Librarian receives a JSON payload and produces a classification with `index_id`, `key`, `domain`, `searchable`, `metadata`, and `relations` (Source: [prompts/librarian.md:1-1]()). Relations are constrained — only index_ids appearing in the existing-index snippet may be referenced; invented index_ids are forbidden (Source: [prompts/librarian.md:2-2]()). The Librarian is dispatched with `model: claude-sonnet-4-6` — a deliberate downshift from the Opus default because classification is reasoning-heavy but operates on a bounded payload (Source: [internal/brain/capture/SKILL.md:2-2]()). ### Reference Librarian and Query Planning The Reference Librarian receives a natural-language query and returns a JSON query plan (`fts_query`, `vector_query`, `filters`, `limit`). If the query is ambiguous, it returns a `"clarify"` question instead of guessing (Source: [prompts/reference_librarian.md:1-1]()). ### Write Path and the Five Agents `memex:index:write` is the lower-level primitive that `memex:brain:ingest` and `memex:brain:capture` build on. Consumers (e.g., Atelier adding a decision) may supply a pre-computed `librarian_output` to skip the LLM classification when writing structured rows (Source: [internal/index/write/SKILL.md:1-1]()). The five internal agents — Librarian, Reference Librarian, Archivist, Database Administrator, Data Steward — collectively own the write path. `memex:core:get-agent` fetches an agent's full profile by `agent_id` for context-aware decision-making (Source: [internal/core/get-agent/SKILL.md:1-1]()). Non-document inserts go through `memex:core:insert` → `scripts/stores.py:insert(name, table, row)` (Source: [internal/core/insert/SKILL.md:1-1]()). ### Dispatched-Subagent Cost Floor The per-dispatch cost floor is **enforced, not advisory**: every dispatching `internal/.../SKILL.md` Task block carries an explicit `model:` line, and `tests/test_model_tier_dispatch.py` fails CI if any line is stripped, downgraded to Opus, or set to a stale model-id (Source: [CLAUDE.md:2-2]()). This is how memex keeps LLM subagent costs predictable — no dispatch silently inherits the expensive Opus default. ## The Code Graph Subsystem The code graph is a separate code-navigation store (`code_graph.db`) for consumer recon (e.g., kaizen, atelier). It is built by an **external extractor** (`graphify`) and ingested by Memex — Memex itself does **no source/AST parsing** to derive docstrings or symbols (Source: [README.md:4-4]()). The graph is stored via `db/code_graph.sql` and served by bounded, deterministic queries — `where_is`, `callers`, `dependencies`, `neighbors`, `module_map` — implemented in `internal/codegraph/query`. The query layer is pure SQL; no LLM and no extraction runs inside Memex (Source: [README.md:5-5]()). ### v2.10.0 — Docstring-Presence Signal v2.10.0 added forward-compatible storage for a docstring-presence signal on code-graph nodes while explicitly documenting the limitation: Memex does not (and must not) compute docstring presence itself. The signal is supplied by the external extractor (Source: [community_context:v2.10.0]()). This keeps Memex's contract with consumers clean: schema and query surface are stable, extraction lives elsewhere. ## Community Layer (Derived) On top of the indexed documents, Memex builds a **GraphRAG community summarization** layer: a key-free lexical similarity graph, hierarchical community detection (`scripts/communities.py`), and bottom-up per-community reports persisted to `index.db` (Source: [README.md:6-6]()). The Community Reporter subagent produces a structured JSON report (`title`, `summary`, `rating`, `findings`) from member documents and child-community summaries (Source: [prompts/community_reporter.md:1-1]()). The community layer is rebuilt on demand via `internal/brain/graph-rebuild` and is **never on the write path** — write performance is not coupled to graph rebuild cost (Source: [README.md:7-7]()). ## See Also - [README.md](https://github.com/nitekeeper/memex/blob/main/README.md) — full project overview and installation - [CLAUDE.md](https://github.com/nitekeeper/memex/blob/main/CLAUDE.md) — operating rules and the M-rules that govern this repo - [skills/run/SKILL.md](https://github.com/nitekeeper/memex/blob/main/skills/run/SKILL.md) — the routing table from user intents to internal procedures - [Memex release notes (v2.17.0)](https://github.com/nitekeeper/memex/releases/tag/v2.17.0) — dashboard search now spans documents, agents, and the code graph --- ## Installation, Bootstrap, Deployment, and Extensibility ### Related Pages Related topics: [Memex Overview and Three-Layer Architecture](#page-1), [Core Data Operations: Index, Brain, Internal Agents, and Code Graph](#page-3)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/nitekeeper/memex/blob/main/README.md) - [CLAUDE.md](https://github.com/nitekeeper/memex/blob/main/CLAUDE.md) - [scripts/upgrade_from_v1.py](https://github.com/nitekeeper/memex/blob/main/scripts/upgrade_from_v1.py) - [scripts/release.py](https://github.com/nitekeeper/memex/blob/main/scripts/release.py) - [scripts/bump.py](https://github.com/nitekeeper/memex/blob/main/scripts/bump.py) - [skills/README.md](https://github.com/nitekeeper/memex/blob/main/skills/README.md) - [internal/brain/capture/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/capture/SKILL.md) - [internal/brain/ask/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/ask/SKILL.md) - [internal/brain/synthesize/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/brain/synthesize/SKILL.md) - [internal/core/get-agent/SKILL.md](https://github.com/nitekeeper/memex/blob/main/internal/core/get-agent/SKILL.md)
# Installation, Bootstrap, Deployment, and Extensibility Memex ships as a Claude Code plugin: a self-contained bundle of skills, SQLite stores, Python helpers, and prompts that turn natural-language intents into structured operations on a personal knowledge plane. This page covers the four operational lifecycles of that bundle — how it gets onto a machine, how it initializes on first use, how new versions are assembled and shipped, and how downstream agents and skill authors extend it. ## Installation Paths Memex has two distinct audiences with different install surfaces, and the README is explicit about which path each audience takes. Source: [README.md](). ### For Consumers A consumer installs Memex through the Claude Code marketplace. The marketplace unpacks the bundle into `~/.claude/plugins/cache//memex//`, and Claude Code manages that path automatically — users must not place files there manually. Each release ships its own `INSTALL.md` inside the bundle, which is the authoritative, version-specific install checklist. Source: [README.md](). After unpack, the user restarts Claude Code (or invokes `/plugin reload memex` on supporting versions) and then calls `memex:run` with a natural-language intent. ### For Developers / Upgraders from v1 The v2 line treats v1 (`~/.ai/wiki/`) as legacy rather than a migration source. `scripts/upgrade_from_v1.py` detects a prior v1 install via `$MEMEX_V1_PATH`, validates that the path and its `.ai/` directory are not symlinks, and copies `.ai/` into `~/.memex/legacy/v1-wiki/` — symlinks under `.ai/` are preserved as links, not dereferenced, so the archive remains a faithful snapshot. Source: [scripts/upgrade_from_v1.py](). The script rejects symlink violations with `ValueError` (security-sensitive), and treats missing-path / outside-`$HOME` cases as "no v1 found" (idempotent no-ops). Per the design decision referenced at the top of the file, v1 content is NOT migrated into `brain.db`; users manually re-ingest what they want preserved. ## First-Run Bootstrap Bootstrap is driven by the `memex:run` skill rather than by an installer. On the very first invocation, Step 0 detects the missing store state and seeds the minimum required schema. Source: [README.md](). The first-run flow has three observable behaviors: 1. **Store creation.** The Core, Brain, Index, and Captures stores under `~/.memex/` are materialized from the SQL files in `db/` and the `migrations` ledger. v2.12.2 introduced a self-healing migration runner in `scripts/stores.py` that no longer crashes with `duplicate column name` / `table already exists` when the ledger falls behind the actual schema. 2. **Skills registry.** `skills/README.md` enumerates the expected skill set (`capture/`, `sync/`, `search/`, `capture-lesson/`, `review-lessons/`, `propose-wiki-entry/`, `review-wiki/`), which are populated during the build phase and then exposed via `memex:run`'s intent router. Source: [skills/README.md](). 3. **Agent registry.** Memex's internal agents (Librarian, Reference Librarian, Community Reporter, Synthesizer) are seeded on bootstrap; later procedures such as `memex:core:get-agent` resolve them by id. Source: [internal/core/get-agent/SKILL.md](). ## Deployment and Release Assembly Memex release engineering is a thin, declarative pipeline. Source: [scripts/release.py](). `scripts/release.py` builds a `dist/v/` bundle by copying a fixed include set: | Layer | Path | Purpose | |---|---|---| | Manifest | `.claude-plugin/` | Claude Code reads `plugin.json`; required for `claude --plugin-dir` | | Code | `scripts/`, `skills/`, `internal/`, `db/`, `prompts/` | Runtime payload | | Top-level | `pyproject.toml`, `README.md`, `USER_GUIDE.md`, `CHANGELOG.md` | Project metadata | The build emits a `manifest.json` inventory alongside the bundle, hashing each file for reproducibility. `dist/` body is gitignored; only the manifest is tracked. Version bumps are handled by `scripts/bump.py`, which enforces the `X.Y.Z` shape (no leading `v`), rewrites the version inside `.claude-plugin/plugin.json`, and updates the `description` field's embedded `Memex v` token. The script intentionally does NOT write CHANGELOG entries, commit, tag, or push — those are human decisions, and the GitHub `release.yml` workflow triggers on tag push. Source: [scripts/bump.py](). The Layer-1 / Layer-2 boundary is enforced by `CLAUDE.md`: Memex is Layer 2 (a Skill Atelier product); framework changes commit to `skill-atelier/`, plugin changes commit here, and they must never mix. Source: [CLAUDE.md](). ## Extensibility: Skills, Agents, and Subagent Tiers Memex exposes exactly one Claude-Code-visible skill — `memex:run` — to stay well under the 1% skill-description budget. Everything else is an `internal/.../SKILL.md` procedure dispatched on demand from natural-language intents. Source: [README.md](). ### Adding a Brain procedure A new Brain flow is a single markdown file plus a Python helper. The pattern is visible across the existing skills: - `internal/brain/capture/SKILL.md` documents a `capture_prepare(...)` Python call, then dispatches a Task-tool subagent (`subagent_type=general-purpose`, `model: claude-sonnet-4-6`) for classification. Source: [internal/brain/capture/SKILL.md](). - `internal/brain/synthesize/SKILL.md` shows the two-dispatch variant: Synthesizer produces prose, Librarian classifies the result and emits `synthesizes` relations back to the source `index_id`s. Source: [internal/brain/synthesize/SKILL.md](). - `internal/brain/ask/SKILL.md` shows the `global`/`local` mode split, both of which depend on the derived layer (`community_reports`); if empty, the flow reports a degraded result rather than failing. Source: [internal/brain/ask/SKILL.md](). ### Subagent cost floors Every `internal/.../SKILL.md` Task block carries an explicit `model:` line — and `tests/test_model_tier_dispatch.py` fails CI if any line is stripped, downgraded to Opus, or set to a stale model id. The rationale is that dispatched subagents (e.g., `claude-haiku-4-5` for Community Reporter, `claude-sonnet-4-6` for Librarian) handle bounded mechanical work and must not silently inherit the orchestrator's Opus default. Source: [CLAUDE.md](). ### Prompt-injection boundary Ingested content — from `internal/brain/ingest`, `internal/brain/capture`, `internal/index/write`, URL fetchers, and `memex:run ask` results — is structurally treated as data, never as instructions. The Librarian and Archivist must delimit payloads at write time, and any payload that appears to request a tool call is logged and rejected. Source: [CLAUDE.md](). ## See Also - [User Guide: Using Memex in Your Claude Code](README.md) - [Reference: Brain Skills](internal/brain/ask/SKILL.md) - [Reference: Capture Flow](internal/brain/capture/SKILL.md) - [Reference: Synthesize Flow](internal/brain/synthesize/SKILL.md) - [Project: Layer-1 Framework (skill-atelier)](CLAUDE.md) --- --- ## Pitfall Log Project: nitekeeper/memex Summary: Found 17 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification. ## 1. Installation risk - Installation risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this installation risk before relying on the project: v2.10.0 - User impact: Upgrade or migration may change expected behavior: v2.10.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.10.0 ## 2. Installation risk - Installation risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this installation risk before relying on the project: v2.11.0 - User impact: Upgrade or migration may change expected behavior: v2.11.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.11.0 ## 3. Installation risk - Installation risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this installation risk before relying on the project: v2.12.0 - User impact: Upgrade or migration may change expected behavior: v2.12.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.12.0 ## 4. Installation risk - Installation risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this installation risk before relying on the project: v2.16.0 - User impact: Upgrade or migration may change expected behavior: v2.16.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.16.0 ## 5. Configuration risk - Configuration risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: capability.host_targets | https://github.com/nitekeeper/memex ## 6. Configuration risk - Configuration risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this configuration risk before relying on the project: v2.13.0 - User impact: Upgrade or migration may change expected behavior: v2.13.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.13.0 ## 7. Capability evidence risk - Capability evidence risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: README/documentation is current enough for a first validation pass. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: capability.assumptions | https://github.com/nitekeeper/memex ## 8. Maintenance risk - Maintenance risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Developers should check this migration risk before relying on the project: v2.12.2 - User impact: Upgrade or migration may change expected behavior: v2.12.2 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.12.2 ## 9. Maintenance risk - Maintenance risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/nitekeeper/memex ## 10. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: downstream_validation.risk_items | https://github.com/nitekeeper/memex ## 11. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: risks.scoring_risks | https://github.com/nitekeeper/memex ## 12. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: issue_or_pr_quality=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/nitekeeper/memex ## 13. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: release_recency=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/nitekeeper/memex ## 14. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: Developers should check this maintenance risk before relying on the project: v2.12.1 - User impact: Upgrade or migration may change expected behavior: v2.12.1 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.12.1 ## 15. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: Developers should check this maintenance risk before relying on the project: v2.14.0 - User impact: Upgrade or migration may change expected behavior: v2.14.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.14.0 ## 16. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: Developers should check this maintenance risk before relying on the project: v2.15.0 - User impact: Upgrade or migration may change expected behavior: v2.15.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.15.0 ## 17. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: Developers should check this maintenance risk before relying on the project: v2.17.0 - User impact: Upgrade or migration may change expected behavior: v2.17.0 - Evidence: failure_mode_cluster:github_release | https://github.com/nitekeeper/memex/releases/tag/v2.17.0