# https://github.com/nikhilkagita04/continuum Project Manual
Generated at: 2026-06-24 06:40:19 UTC
## Table of Contents
- [Introduction and Quickstart](#page-intro)
- [The Four-Stage Pipeline](#page-pipeline)
- [Capture Quality, Noise Filtering, and Self-Exclusion](#page-capture-quality)
- [MCP Server, Graph Sidecar, and Custom Tools](#page-extensibility)
## Introduction and Quickstart
### Related Pages
Related topics: [The Four-Stage Pipeline](#page-pipeline), [MCP Server, Graph Sidecar, and Custom Tools](#page-extensibility)
Related Source Files
The following source files were used to generate this page:
- [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md)
- [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json)
- [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md)
- [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift)
- [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift)
- [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md)
- [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py)
- [examples/claude-desktop.json](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json)
- [packaging/menubar.swift](https://github.com/nikhilkagita04/continuum/blob/main/packaging/menubar.swift)
# Introduction and Quickstart
## What Continuum Is and Why It Exists
Continuum is an open, local-first **capture and memory layer** for your computer. It addresses a specific failure mode: when you switch between apps (Claude, Slack, X, a design tool, your email), each one starts blank, and *you* become the courier re-pasting context between them. Continuum quietly remembers what you do on-device and exposes that memory to the next tool, so an agent can answer "what was I just doing in X?" without any re-explanation. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
The project is positioned explicitly as a **primitive, not an app** — a shovel others build use cases on, not a closed "brain" you hand everything to. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md). Concretely, it is:
- **Event-driven, not continuous** — it captures only when the screen or focused window meaningfully changes, so it is faithful without being a firehose. Source: [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift) (perceptual-hash gating before OCR).
- **Local-first** — all data lives under `~/.continuum`; credential managers are excluded by default and PII is redacted. Source: [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift) (`EXCLUDED` set + `CONTINUUM_EXCLUDE` env var).
- **Composable** — queryable from the CLI, an importable Node SDK, or via MCP, so any agent can use the memory. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
The package itself is named `continuum-core` at version `0.1.5`, ships as an ESM module with Node 18+ as the minimum engine, and has **zero runtime dependencies** by design. Source: [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json).
## Installation
The README's install path takes roughly 30 seconds and requires only Node 18+. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
```bash
npm install -g continuum-core
continuum verify # proves the whole loop works — no keys, no permissions
```
`continuum` is wired in as the `bin` entry `continuum-core` exposes; the CLI is implemented at `bin/continuum.mjs`. Source: [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json). The `verify` subcommand is also registered as a `npm run verify` script for local development. Source: [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json).
For contributors and source-build users, the repo provides equivalent `npm` scripts for every stage of the toolchain (`build:capture`, `build:app`, `start`, `test`). Source: [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json).
## Running Continuum Day-to-Day
Three commands cover almost all everyday use:
```bash
continuum start # begin on-device capture (macOS prompts for Screen Recording)
continuum dashboard # open the searchable timeline at http://localhost:3939
continuum mcp-install # register Continuum with Claude Desktop (preserves existing config)
```
Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md). After `mcp-install`, the user must **fully quit and reopen** Claude Desktop (Cmd+Q) for the new MCP server to load. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
For non-Claude MCP clients (Cursor, custom agents), `continuum mcp-config` prints a JSON snippet the user merges manually. The shipped example for Claude Desktop wraps the Node server in an `mcpServers` block under the key `continuum`. Source: [examples/claude-desktop.json](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json).
A lightweight macOS menu-bar app (`◌` icon) is also provided for users who prefer a GUI over the CLI: it exposes Start/Stop, an "Open data folder" action, and a Quit item, and runs as an `LSUIElement` (no Dock icon). Source: [packaging/menubar.swift](https://github.com/nikhilkagita04/continuum/blob/main/packaging/menubar.swift).
## How It Works — The Four-Stage Pipeline
Continuum turns a torrent of raw screen events into a small set of high-signal episodes, with the LLM kept off the hot path. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md). The pipeline has four named stages, all importable as separate modules:
```mermaid
flowchart LR
A[Stage 1: Capture
screen.swift / capture.swift
NDJSON CaptureEvents] --> B[Stage 2: Segment
daemon/stage2/segmenter.mjs]
B --> C[Stage 3: Index
daemon/stage3/index.mjs
local hybrid index]
C --> D[Stage 4: Distill
daemon/stage4/distill.mjs
~30 LLM calls/day]
D --> E[Retrieval
daemon/retrieval.mjs]
E --> F[MCP Server
daemon/mcp-server.mjs]
```
- **Stage 1 — Capture.** Two Swift helpers share one contract: `screen.swift` uses ScreenCaptureKit + Apple Vision OCR, gated by a perceptual hash so a static window costs almost nothing. Source: [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift). `capture.swift` is the accessibility-based alternative that watches the focused element on a 1-second pasteboard poll plus per-pid `AXObserver`, emitting clean `AXFocusedUIElement` text keyed by `focus+role`. Source: [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift).
- **Stages 2–4** (segment, index, distill) are pure Node modules that take *injected* adapters (embedder, LLM, graph store) and must run offline — this is a hard project rule, not a suggestion. Source: [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md).
- **Capture-quality guardrails.** The screen helper already excludes credential managers, the Continuum dashboard, and the terminal running the daemon, and it crops the top band of browser windows before OCR so episodes start with page content rather than the tab strip. Source: [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift). These correspond directly to the open community issues #3 and #4 tracked for the project, which explicitly link the design back to `docs/architecture/capture-quality.md`.
The headline performance figure: ~29k raw daily events are funneled into roughly **30 LLM calls per day**, which is what makes the system tractable. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
## Tiers and What's Free
Continuum ships in three tiers today; only the first is fully open and free. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
| | Free | Pro *(later)* | Enterprise *(later)* |
|---|---|---|---|
| Capture · recall · MCP | ✅ | ✅ | ✅ |
| Embeddings / LLM | local, $0 | OpenAI / Anthropic | hosted |
| Temporal knowledge graph | — | ✅ | ✅ team graph |
The graph tier is intentionally the **paid line** because reliable entity/relation extraction needs a frontier model (Claude/GPT-class) — open models cannot satisfy the strict `EdgeDuplicate` / `NodeResolutions` schemas in `graphiti-core` and raise pydantic `ValidationError`. Source: [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py). The optional sidecar runs as a FastAPI service on Neo4j, with Anthropic as the default extraction LLM and `gpt-4o-mini` selectable as a roughly 15–20× cheaper alternative. Source: [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md).
## Development Quickstart
To hack on Continuum itself, clone the repo, run the offline test suite, and (on macOS) build the Swift helpers. Source: [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md).
```bash
git clone https://github.com/nikhilkagita04/continuum && cd continuum
npm test # 36 unit + integration tests, no network
node bin/continuum.mjs verify # end-to-end smoke
swiftc daemon/stage1/screen.swift -o daemon/stage1/screen
```
Source: [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md). All contributions must be signed off with `git commit -s` (DCO) and the project is licensed under Apache-2.0. Source: [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md).
## See Also
- Ingestion pipeline deep dive: `docs/architecture/ingestion-pipeline.md` — referenced from [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
- Capture-quality design notes: `docs/architecture/capture-quality.md` — referenced from the open issues #1–#4 for AX-focused input, LLM structuring, browser-chrome cropping, and self-exclusion.
---
## The Four-Stage Pipeline
### Related Pages
Related topics: [Capture Quality, Noise Filtering, and Self-Exclusion](#page-capture-quality), [MCP Server, Graph Sidecar, and Custom Tools](#page-extensibility), [Introduction and Quickstart](#page-intro)
Related Source Files
The following source files were used to generate this page:
- [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md)
- [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json)
- [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md)
- [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift)
- [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift)
- [daemon/stage2/segmenter.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage2/segmenter.mjs)
- [daemon/stage3/index.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage3/index.mjs)
- [daemon/stage4/distill.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage4/distill.mjs)
- [daemon/pipeline.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/pipeline.mjs)
- [daemon/retrieval.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/retrieval.mjs)
- [daemon/adapters.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/adapters.mjs)
- [daemon/config.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/config.mjs)
- [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py)
- [examples/claude-desktop.json](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json)
# The Four-Stage Pipeline
## Overview
Continuum is described as an *open capture + memory layer*: a small, dependency-free primitive that turns on-device activity into a queryable timeline. The entire design is organized as a four-stage pipeline — **capture → segment → index → distill** — where each stage is a small, tested module under `daemon/`, and the LLM is kept entirely off the hot capture path. The README states the goal quantitatively: *"Four stages turn ~29k raw daily events into ~30 LLM calls — and the LLM never touches the capture path"* [README.md](README.md).
Stages 2–4 are designed as pure modules that take injected adapters (`embedder`, `LLM`, `graph`) and *"must run offline"* — there is no hard-coded cloud dependency in the core [CONTRIBUTING.md](CONTRIBUTING.md). This separation is what keeps the capture path light and the overall system local-first.
```mermaid
flowchart LR
A[Stage 1
Capture] -->|CaptureEvent NDJSON| B[Stage 2
Segment]
B -->|Episodes| C[Stage 3
Index]
C -->|Hybrid index| D[Stage 4
Distill]
D -->|Entities / facts| E[(~/.continuum store)]
E -->|query| R[Retrieval / MCP]
```
The pipeline is the package's main export: `"."` resolves to `daemon/pipeline.mjs`, and each stage is independently importable via subpath exports such as `continuum-core/segmenter`, `continuum-core/index`, and `continuum-core/distill` [package.json](package.json).
## Stage 1 — Capture (event-driven, on-device)
Stage 1 emits a normalized NDJSON stream of `CaptureEvent`s on stdout: `{t, source, app, window_id, url_host?, title?, text, secure?}` [daemon/stage1/capture.swift:7-9](daemon/stage1/capture.swift). There are two interchangeable helpers:
| Helper | Backend | Permission | Trigger |
|---|---|---|---|
| `daemon/stage1/capture` (Swift) | macOS Accessibility API (`AXObserver`) + 1s pasteboard poll | Accessibility | App focus, AX notifications, clipboard change |
| `daemon/stage1/screen` (Swift) | ScreenCaptureKit + Apple Vision OCR | Screen Recording | Perceptual-hash change in focused window |
The AX helper *retargets* on `NSWorkspace.didActivateApplicationNotification`, tears down and rebuilds per-pid observers, and coalesces emits at ≤1 per 400ms per window to avoid floods [daemon/stage1/capture.swift](daemon/stage1/capture.swift). It also descends the focused subtree, **skipping chrome roles** (buttons, menus, toolbars, tabs) and only collecting prose ≥3 words / ≥24 chars — explicitly separating *signal from window furniture* [daemon/stage1/capture.swift](daemon/stage1/capture.swift).
The screen helper uses ScreenCaptureKit with a perceptual-hash dedup, so a static window costs ~nothing; when a change is detected, Apple Vision runs OCR in reading order and emits a `source: "ocr"` event with the same contract as the AX path [daemon/stage1/screen.swift](daemon/stage1/screen.swift). Both helpers pipe into `node daemon/pipeline.mjs` — the downstream stages don't care which one produced the stream.
## Stage 2 — Segment
Stage 2 turns the raw event stream into discrete, meaningful *episodes*. It is exported as `continuum-core/segmenter` and is exercised by the test suite (`npm test` runs `daemon/stage2/segmenter.test.mjs` first) [package.json](package.json). The segmenter's contract is to coalesce noisy per-window events into coherent activities (e.g., "writing an email in Mail.app"), bounded by app switches, idle gaps, or text deltas. As an injected module, it depends only on the configured `embedder` adapter and writes structured segments to the local store.
## Stage 3 — Index
Stage 3 builds the **hybrid index** — typically a combination of lexical (BM25/FTS) and vector recall over episodes. It is exported as `continuum-core/index` [package.json](package.json). The README describes this as the substrate of the free tier: *"The free tier runs entirely on the local hybrid index (capture → segment → vector recall)"* [backend/graphiti/README.md](backend/graphiti/README.md). Because this stage takes an injected `embedder`, the core can run with local embeddings ($0) or a hosted provider [README.md](README.md).
## Stage 4 — Distill
Stage 4 is the only stage that talks to the LLM. It is exported as `continuum-core/distill` and runs on the already-segmented, already-indexed data — selectively, not per-event — which is why the headline number is "~30 LLM calls / day" for ~29k raw events [README.md](README.md). Distillation is also the natural place for higher-level structuring, and the community has identified it as the right insertion point for an LLM-based attribution pass that turns raw OCR text into structured fields (`author`, `content`, `app`) and drops chrome/noise — see [issue #2](https://github.com/nikhilkagita04/continuum/issues/2).
## Adapters, Config, and Retrieval
Stages 2–4 are decoupled from concrete implementations via two modules:
- `daemon/adapters.mjs` — the injection points (`embedder`, `llm`, `graph`) so the core remains cloud-free by default [CONTRIBUTING.md](CONTRIBUTING.md).
- `daemon/config.mjs` — runtime configuration loaded by the pipeline.
Retrieval (`daemon/retrieval.mjs`, exported as `continuum-core/retrieval`) is what the MCP server and dashboard call. It reads from the on-disk store at `~/.continuum` and answers natural-language queries, optionally calling out to a graph sidecar (`backend/graphiti/sidecar.py`) when the temporal knowledge graph tier is enabled [package.json](package.json), [backend/graphiti/sidecar.py](backend/graphiti/sidecar.py).
## Known Capture-Quality Failure Modes
The community has surfaced four capture-quality concerns, all of which feed *into* Stage 1 and propagate through the rest of the pipeline. The design notes for resolving them live in `docs/architecture/capture-quality.md`:
- **Self-capture loop** ([issue #4](https://github.com/nikhilkagita04/continuum/issues/4)) — the dashboard (`localhost:3939`) and the running terminal are currently captured. The intended fix is a title/window exclusion (already partially in place for `screen.swift` via `SELF_MARKERS` heuristic strings [daemon/stage1/screen.swift](daemon/stage1/screen.swift)).
- **Browser chrome noise** ([issue #3](https://github.com/nikhilkagita04/continuum/issues/3)) — every browser episode is prefixed by the tab/bookmarks bar. A conservative top-band crop is the proposed mitigation; the screen helper already recognizes a `BROWSERS` set, though the crop itself is the open work [daemon/stage1/screen.swift](daemon/stage1/screen.swift).
- **OCR vs. authored input** ([issue #1](https://github.com/nikhilkagita04/continuum/issues/1)) — OCR fuses the user's typed text with the surrounding page; `AXFocusedUIElement` reliably yields *just the user's text* and should be emitted as a distinct `source: "input"` event alongside the OCR stream.
- **Episode attribution** ([issue #2](https://github.com/nikhilkagita04/continuum/issues/2)) — a Stage-4 structuring pass would convert raw OCR text into structured fields, avoiding brittle per-site parsers.
## See Also
- [README.md](README.md) — project introduction and tier table
- [CONTRIBUTING.md](CONTRIBUTING.md) — ground rules and test invocation
- [package.json](package.json) — package exports and `npm` scripts
- [`docs/architecture/ingestion-pipeline.md`](docs/architecture/ingestion-pipeline.md) — referenced deep dive on the four-stage design
- [`docs/architecture/capture-quality.md`](docs/architecture/capture-quality.md) — design notes referenced by issues #1–#4
- [backend/graphiti/README.md](backend/graphiti/README.md) — optional temporal knowledge graph tier (Pro)
---
## Capture Quality, Noise Filtering, and Self-Exclusion
### Related Pages
Related topics: [The Four-Stage Pipeline](#page-pipeline), [MCP Server, Graph Sidecar, and Custom Tools](#page-extensibility)
Related Source Files
The following source files were used to generate this page:
- [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift)
- [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift)
- [daemon/stage4/distill.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage4/distill.mjs)
- [daemon/pipeline.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/pipeline.mjs)
- [daemon/config.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/config.mjs)
- [docs/architecture/capture-quality.md](https://github.com/nikhilkagita04/continuum/blob/main/docs/architecture/capture-quality.md)
- [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md)
- [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md)
# Capture Quality, Noise Filtering, and Self-Exclusion
## Purpose and Scope
Continuum is an open, local-first capture + memory layer that records what the user sees on screen and turns it into a searchable timeline. Because every app the user interacts with is a potential capture source, **capture quality** is the primary lever that determines whether the downstream segmenter, indexer, and distiller produce useful episodes or noisy ones. The project addresses quality at three layers:
1. **Stage 1 (capture)** — pre-OCR noise removal: skip UI chrome, crop browser toolbars, suppress self-capture and credential apps.
2. **Stage 1 (AX helper)** — accessibility-tree filtering: ignore button/menu/toolbar roles, deduplicate repeats, enforce text budgets.
3. **Stage 4 (distill)** — semantic structuring: an injected LLM pass that attributes who-said-what and drops residual noise (designed, in flight).
This page documents the first two layers, which are fully implemented in the current source tree, and summarizes the Stage-4 design referenced in [issue #2](https://github.com/nikhilkagita04/continuum/issues/2) and [`docs/architecture/capture-quality.md`](https://github.com/nikhilkagita04/continuum/blob/main/docs/architecture/capture-quality.md).
---
## Stage 1 — Universal OCR Capture: `screen.swift`
The screen-capture helper in [`daemon/stage1/screen.swift`](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift) is the "what you see" path. It uses ScreenCaptureKit to screenshot the focused window, runs Apple Vision OCR in reading order, and emits NDJSON `CaptureEvent`s on stdout. Several quality mechanisms live here.
### Self-exclusion via `SELF_MARKERS`
The helper hard-codes markers that identify Continuum's own surfaces so the OCR loop never records itself:
```swift
let SELF_MARKERS = ["What your machine captured", "continuum: tier=", "capture=screen embed="]
```
The text is matched against the OCR output, and any match is silently dropped before emission (`Source: [daemon/stage1/screen.swift:46]()`). This is the minimal implementation of the requirement tracked in [issue #4](https://github.com/nikhilkagita04/continuum/issues/4) ("exclude Continuum's own windows from capture") — title/window-based exclusion is the planned follow-up. Contributors should follow the guardrail in [`docs/architecture/capture-quality.md`](https://github.com/nikhilkagita04/continuum/blob/main/docs/architecture/capture-quality.md): **do not scrape site DOMs** as a substitute.
### Browser toolbar cropping
For Chromium-family browsers, Safari, Arc, Firefox, Edge, Brave, and Opera, the helper crops a top band from the screenshot before OCR so episodes begin with page content rather than the tab strip and bookmarks bar:
```swift
let BROWSERS: Set = ["Google Chrome", "Safari", "Arc", "Firefox",
"Microsoft Edge", "Brave Browser", "Opera"]
func cropTopBand(_ image: CGImage, fraction: Double) -> CGImage { ... }
```
The fraction is intentionally heuristic — toolbar height varies — and is called only for known browser apps. This is the design called out in [issue #3](https://github.com/nikhilkagita04/continuum/issues/3).
### Excluded apps and user extension
A second denylist excludes credential managers so that secrets never enter the capture stream:
```swift
let EXCLUDED: Set = ["1Password", "Keychain Access", "Bitwarden",
"Dashlane", "LastPass"]
```
Users can extend it at runtime via the `CONTINUUM_EXCLUDE` environment variable (comma-separated app names). The set is consulted both by the OCR loop and by `focusedInput()` when reading the AX focused element.
### Focused-input capture (AX)
`focusedInput()` reads the frontmost app's `AXFocusedUIElement` and emits its `AXValue` as a *separate* event when it differs from the last seen value. This is the user-authored text path — the reply you typed, the email you composed — which OCR fuses with surrounding page text. The helper bails out on `AXSecureTextField` so passwords never leak. This implements the design in [issue #1](https://github.com/nikhilkagita04/continuum/issues/1), which proposes emitting this as a distinct `CaptureEvent` (e.g. `source: "input"`).
---
## Stage 1 — Accessibility Capture: `capture.swift`
The accessibility helper in [`daemon/stage1/capture.swift`](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift) walks the focused window's AX tree and extracts *meaningful* text only. Two filters do most of the work:
### Chrome-role allowlist
A static set of UI-furniture roles is excluded from text collection. The walker still **descends** through them (a container may wrap real content) but does not collect their labels:
```swift
let CHROME_ROLES: Set = [
"AXButton", "AXMenuButton", "AXPopUpButton", "AXMenuBar", "AXMenuBarItem",
"AXMenu", "AXMenuItem", "AXToolbar", "AXTabGroup", "AXRadioButton",
"AXCheckBox", "AXImage", "AXScrollBar", "AXSlider", "AXDisclosureTriangle",
"AXIncrementor", "AXColorWell", "AXBusyIndicator", "AXProgressIndicator",
"AXValueIndicator",
]
```
### Substring, dedup, and budget
`extract()` keeps a string only if it is at least 3 words **or** 24 characters, no longer than 5000 characters, and not a previously seen exact repeat. A per-call `budget` (byte/char counter) caps the total volume of text any one window contributes, protecting downstream stages from accidental blow-ups.
```swift
if (words >= 3 || t.count >= 24), t.count <= 5000, !seen.contains(t) {
seen.insert(t); out.append(t); budget -= t.count
}
```
The result is a single whitespace-joined string emitted under `source: "ax"`. Source: [daemon/stage1/capture.swift:60-80]().
---
## Pipeline Wiring and Configuration
Both helpers feed `daemon/pipeline.mjs` over stdout NDJSON. The pipeline is the only "core" entry point exported from the package (`Source: [package.json:11]()`), and the design rule from [`CONTRIBUTING.md`](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md) is that **stages 2–4 take injected adapters and must run offline** — keeping the core dependency-free and local.
| Environment variable | Effect | Source |
|---|---|---|
| `CONTINUUM_EXCLUDE` | Comma-separated app names appended to the OCR denylist. | `daemon/stage1/screen.swift:32-38` |
| `CONTINUUM_OCR_INTERVAL` | Seconds between OCR ticks (default `2.5`). | `daemon/stage1/screen.swift:120` |
| `CONTINUUM_DATA` | Override data directory used by the test runner. | `package.json:25` |
The Stage-4 LLM structuring pass proposed in [issue #2](https://github.com/nikhilkagita04/continuum/issues/2) is intentionally **distill-time, not capture-time**, because running an LLM in the capture path would defeat the project's "light, on-device" design.
---
## See Also
- [Ingestion Pipeline Architecture](https://github.com/nikhilkagita04/continuum/blob/main/docs/architecture/ingestion-pipeline.md) — the four-stage design.
- [Stage 4 Distill](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage4/distill.mjs) — where the LLM structuring pass will land.
- Issue [#1](https://github.com/nikhilkagita04/continuum/issues/1), [#2](https://github.com/nikhilkagita04/continuum/issues/2), [#3](https://github.com/nikhilkagita04/continuum/issues/3), [#4](https://github.com/nikhilkagita04/continuum/issues/4) — open capture-quality work.
---
## MCP Server, Graph Sidecar, and Custom Tools
### Related Pages
Related topics: [The Four-Stage Pipeline](#page-pipeline), [Capture Quality, Noise Filtering, and Self-Exclusion](#page-capture-quality), [Introduction and Quickstart](#page-intro)
Related Source Files
The following source files were used to generate this page:
- [daemon/mcp-server.mjs](https://github.com/nikhilkagita04/continuum/blob/main/daemon/mcp-server.mjs)
- [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py)
- [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md)
- [examples/claude-desktop.json](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json)
- [examples/standup.mjs](https://github.com/nikhilkagita04/continuum/blob/main/examples/standup.mjs)
- [packaging/menubar.swift](https://github.com/nikhilkagita04/continuum/blob/main/packaging/menubar.swift)
- [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json)
- [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md)
- [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md)
- [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift)
- [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift)
# MCP Server, Graph Sidecar, and Custom Tools
## Overview — the extensibility layer
Continuum positions itself as a primitive, not an app. The capture/segment/index/distall stages are importable Node modules, and three explicit extension points turn that core into agent-ready infrastructure: the **MCP server** (`daemon/mcp-server.mjs`) for tool-calling agents, the **graph sidecar** (`backend/graphiti/sidecar.py`) for the optional temporal knowledge graph tier, and the **public exports** declared in `package.json` for users building custom tools against the SDK. Together they make Continuum "a shovel: build your own use cases on top." Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
The packaging contract is small and intentional: every Stage 2–4 module takes **injected adapters** (embedder / LLM / graph) and must run offline, so no cloud provider becomes a hard dependency in core. Source: [CONTRIBUTING.md](https://github.com/nikhilkagita04/continuum/blob/main/CONTRIBUTING.md). That contract is what makes the sidecar and the custom-tool SDK possible without forking the daemon.
```mermaid
flowchart LR
A[Stage 1 capture
Swift NDJSON] --> B[Stage 2 segment]
B --> C[Stage 3 index
embedder adapter]
C --> D[Stage 4 distill
LLM adapter]
D --> E[Local hybrid index
~/.continuum]
D -. optional .-> F[Graph sidecar
sidecar.py /add /search]
E --> G[MCP server
mcp-server.mjs]
F --> G
E --> H[Custom tools
examples/standup.mjs]
G --> I[Claude Desktop / Cursor / agent]
```
## MCP Server — exposing memory to agents
The Model Context Protocol server is the payoff surface: agents (Claude Desktop, Cursor, custom clients) call it to recall what the user did. The CLI ships two helper commands: `continuum mcp-install` patches Claude Desktop's config non-destructively (existing config preserved and backed up), while `continuum mcp-config` prints the config so users on other clients can wire it up by hand. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md). The MCP entry point is `daemon/mcp-server.mjs`, referenced by the published package example.
The published example shows the exact shape of a working registration:
```json
{
"mcpServers": {
"continuum": {
"command": "",
"args": ["/daemon/mcp-server.mjs"]
}
}
}
```
Source: [examples/claude-desktop.json](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json). The README recommends fully quitting and reopening Claude Desktop (Cmd+Q) so the new server is loaded before the first query. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
Operationally, the MCP server depends on `continuum start` being kept running — there must be a daemon process for it to recall from. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md). Because the core pipeline modules are exported as ESM subpaths (see `exports` in `package.json`), MCP tool implementations can `import { ... } from "continuum-core/retrieval"` rather than shelling out, which keeps tool responses low-latency.
## Graph Sidecar — the optional Pro tier
The graph tier is "optional and advanced": it layers entity/relation extraction with bi-temporal edges on top of [Graphiti](https://github.com/getzep/graphiti), backed by Neo4j. Source: [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md). The `sidecar.py` service is a FastAPI app exposing three endpoints that the Continuum pipeline calls from its distill stage.
| Endpoint | Method | Purpose | Source |
| --- | --- | --- | --- |
| `/add` | POST | Append a text episode into Graphiti with a `group_id` bucket | `backend/graphiti/sidecar.py` |
| `/search` | POST | Query edges (facts) by text, scoped to `group_id` | `backend/graphiti/sidecar.py` |
| `/health` | GET | Liveness probe | `backend/graphiti/sidecar.py` |
The sidecar is intentionally separated from the daemon. When enabled in `~/.continuum/config.json` (`"graph": { "enabled": true, "url": "http://localhost:8000" }`), the distill step writes daily rollups into the graph and retrieval fuses graph facts with vector recall. Source: [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md).
LLM selection is runtime-configurable via `GRAPHITI_LLM_PROVIDER`. The default `openai` path uses `gpt-4o-mini`; `anthropic` swaps in Claude Sonnet for extraction with Haiku as the small model. Embeddings and the cross-encoder reranker are pinned to OpenAI (`text-embedding-3-small` at 1536 dims, `gpt-4o-mini` reranker) because Anthropic has no native embeddings API. Source: [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py). This is also why the README notes that graph extraction is "the natural paid line" — open/local models can't satisfy Graphiti's strict extraction schemas. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
Privacy isolation is preserved end-to-end: each user/workspace bucket maps to a Graphiti `group_id`, so memories stay separated even when self-hosted. Source: [backend/graphiti/README.md](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md).
## Custom Tools — the SDK and examples
The package's `exports` field is the SDK surface. Each pipeline stage is importable as a subpath, so a custom tool is roughly twenty lines instead of a fork:
| Subpath | Module | Purpose |
| --- | --- | --- |
| `.` | `daemon/pipeline.mjs` | Full pipeline runner |
| `./segmenter` | `daemon/stage2/segmenter.mjs` | Turn raw events into episodes |
| `./index` | `daemon/stage3/index.mjs` | Hybrid index writer |
| `./distill` | `daemon/stage4/distill.mjs` | LLM-driven structuring (Stage 4) |
| `./retrieval` | `daemon/retrieval.mjs` | Query interface |
| `./adapters` | `daemon/adapters.mjs` | Inject embedder / LLM / graph |
| `./config` | `daemon/config.mjs` | `~/.continuum/config.json` reader |
Source: [package.json](https://github.com/nikhilkagita04/continuum/blob/main/package.json).
Two reference examples ship in `examples/`: a **standup generator** (`examples/standup.mjs`) that calls the retrieval module, and the **Claude Desktop config** (`examples/claude-desktop.json`) that wires the MCP server into the agent. The README frames these as templates: "the stages are importable modules — a useful tool is ~20 lines." Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
The community roadmap reinforces where this surface is heading. Issue #2 proposes adding a Stage-4 LLM structuring pass that turns a raw OCR episode into structured fields (`author`, `content`, `app`) — a custom tool that drops chrome/noise at distill time rather than via per-site parsers. Source: [issue #2](https://github.com/nikhilkagita04/continuum/issues/2). The macOS Stage-1 capture helpers (`daemon/stage1/capture.swift`, `daemon/stage1/screen.swift`) emit normalized `CaptureEvent` NDJSON with a `source` field (e.g. `"input"` for AX-focused-element text vs. `"ocr"` for Vision OCR), which gives the structuring pass a clean signal to attribute authorship. Sources: [daemon/stage1/capture.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/capture.swift) and [daemon/stage1/screen.swift](https://github.com/nikhilkagita04/continuum/blob/main/daemon/stage1/screen.swift).
## Failure modes and guardrails
- **Self-capture.** Issue #4 reports that Continuum currently captures its own dashboard (`localhost:3939`) and the running terminal; the fix is a title/window exclusion in Stage 1 rather than scraping site DOMs. Source: [issue #4](https://github.com/nikhilkagita04/continuum/issues/4).
- **Browser chrome noise.** Issue #3 notes that browser windows prefix every page with the tab/bookmarks bar; the in-progress fix crops the top band before OCR. Source: [issue #3](https://github.com/nikhilkagita04/continuum/issues/3).
- **Local-LLM extraction is not viable.** The sidecar relies on Graphiti's strict structured outputs; open-weight models fail validation, which is why the graph tier is paid-only. Source: [backend/graphiti/sidecar.py](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/sidecar.py).
- **MCP server requires the daemon.** `continuum start` must be running for the MCP server to recall anything. Source: [README.md](https://github.com/nikhilkagita04/continuum/blob/main/README.md).
## See Also
- [Ingestion Pipeline Architecture](https://github.com/nikhilkagita04/continuum/blob/main/docs/architecture/ingestion-pipeline.md) — the four-stage design (capture → segment → index → distill).
- [Graph Tier README](https://github.com/nikhilkagita04/continuum/blob/main/backend/graphiti/README.md) — running the sidecar with Neo4j + Graphiti.
- [Claude Desktop MCP Config Example](https://github.com/nikhilkagita04/continuum/blob/main/examples/claude-desktop.json) — the registration template referenced above.
---
---
## Pitfall Log
Project: nikhilkagita04/continuum
Summary: Found 13 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Configuration risk - Configuration risk requires verification.
## 1. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.host_targets | https://github.com/nikhilkagita04/continuum
## 2. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: Capture: LLM structuring pass to attribute episodes (who said what)
- User impact: Developers may misconfigure credentials, environment, or host setup: Capture: LLM structuring pass to attribute episodes (who said what)
- Evidence: failure_mode_cluster:github_issue | https://github.com/nikhilkagita04/continuum/issues/2
## 3. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: Capture: emit AX focused-element as clean user-authored text
- User impact: Developers may misconfigure credentials, environment, or host setup: Capture: emit AX focused-element as clean user-authored text
- Evidence: failure_mode_cluster:github_issue | https://github.com/nikhilkagita04/continuum/issues/1
## 4. Capability evidence risk - Capability evidence risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://github.com/nikhilkagita04/continuum
## 5. Maintenance risk - Maintenance risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/nikhilkagita04/continuum
## 6. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://github.com/nikhilkagita04/continuum
## 7. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://github.com/nikhilkagita04/continuum
## 8. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/nikhilkagita04/continuum/issues/2
## 9. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/nikhilkagita04/continuum/issues/1
## 10. Capability evidence risk - Capability evidence risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this conceptual risk before relying on the project: Capture: exclude Continuum's own windows (dashboard/terminal) from capture
- User impact: Developers may hit a documented source-backed failure mode: Capture: exclude Continuum's own windows (dashboard/terminal) from capture
- Evidence: failure_mode_cluster:github_issue | https://github.com/nikhilkagita04/continuum/issues/4
## 11. Capability evidence risk - Capability evidence risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this conceptual risk before relying on the project: Capture: reduce browser-chrome noise (crop toolbar/tab band before OCR)
- User impact: Developers may hit a documented source-backed failure mode: Capture: reduce browser-chrome noise (crop toolbar/tab band before OCR)
- Evidence: failure_mode_cluster:github_issue | https://github.com/nikhilkagita04/continuum/issues/3
## 12. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/nikhilkagita04/continuum
## 13. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/nikhilkagita04/continuum