# https://github.com/Patdolitse/piia-engram Project Manual
Generated at: 2026-05-30 18:15:14 UTC
## Table of Contents
- [Overview](#overview)
- [Installation](#installation)
- [Quick Start Guide](#quick-start)
- [Setup Wizard](#setup-wizard)
- [System Architecture](#architecture)
- [Core Components](#components)
- [MCP Tools Reference](#mcp-tools)
- [Knowledge Management](#knowledge-management)
- [Search and Retrieval](#search-retrieval)
- [Security and Encryption](#security-encryption)
## Overview
### Related Pages
Related topics: [Installation](#installation), [Quick Start Guide](#quick-start), [System Architecture](#architecture)
Related Source Files
The following source files were used to generate this page:
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
- [PRIVACY.md](https://github.com/Patdolitse/piia-engram/blob/main/PRIVACY.md)
- [SECURITY.md](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md)
- [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
- [.github/workflows/ci.yml](https://github.com/Patdolitse/piia-engram/blob/main/.github/workflows/ci.yml)
- [engram/__init__.py](https://github.com/Patdolitse/piia-engram/blob/main/engram/__init__.py)
- [src/engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/engram/cli.py)
- [src/engram/mcp.py](https://github.com/Patdolitse/piia-engram/blob/main/src/engram/mcp.py)
- [src/engram/corpus.py](https://github.com/Patdolitse/piia-engram/blob/main/src/engram/corpus.py)
# Overview
**piia-engram** (also referred to as **Engram**) is a local-first, privacy-preserving AI knowledge and memory management system designed as a [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server. It enables AI agents and developers to maintain persistent, searchable, and encrypted knowledge corpora that remain entirely under local control.
Source: [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
---
## Purpose and Scope
Engram addresses a fundamental challenge in AI-assisted development: maintaining contextual continuity across sessions without relying on cloud services or third-party infrastructure.
### Core Objectives
| Objective | Description |
|-----------|-------------|
| **Local-First Privacy** | All knowledge data remains stored locally, encrypted at rest, with no external network dependencies |
| **AI Agent Memory** | Provides persistent memory capabilities for AI agents via the MCP protocol |
| **Knowledge Management** | Supports storing, searching, and organizing knowledge with hybrid keyword + semantic search |
| **Governance & Permissions** | Implements trust-tier based access control and permission boundaries per AI tool |
| **Decision Tracking** | Maintains decision threads with history and supersession tracking |
Source: [PRIVACY.md](https://github.com/Patdolitse/piia-engram/blob/main/PRIVACY.md)
### What Engram Is Not
Engram explicitly does **not** depend on any third-party cloud services. As stated in the project:
> piia-engram 是一个本地优先的开源项目，我们的核心原则是不依赖任何第三方云服务
This local-first philosophy means:
- No cloud API dependencies for core functionality
- No external data storage or synchronization
- Knowledge corpus remains entirely under user control
Source: [GitHub Issue #8](https://github.com/Patdolitse/piia-engram/issues/8)
---
## Architecture
### System Components
```mermaid
graph TD
subgraph "Client Layer"
Claude["Claude Desktop"]
CodeX["CodeX CLI"]
Trae["Trae IDE"]
Custom["Custom MCP Client"]
end
subgraph "MCP Interface"
MCP["Model Context Protocol
piia-engram-mcp"]
end
subgraph "Core Engine"
CLI["engram CLI"]
API["Python API"]
end
subgraph "Data Layer"
KG["Knowledge Graph
SQLite + Edges"]
Corpus["Corpus Storage
Encrypted at Rest"]
Index["Search Index
FTS + Vector"]
end
subgraph "Governance Layer"
Trust["Trust Tier Checks"]
Perms["Permission Profiles"]
end
ClientLayer --> MCP
Custom --> MCP
MCP --> CLI
MCP --> API
CLI --> KG
CLI --> Corpus
API --> Index
Trust -.-> KG
Perms -.-> Corpus
```
### Project Structure
| Directory/File | Purpose |
|----------------|---------|
| `src/engram/` | Core Python package containing CLI, MCP, corpus, and search modules |
| `src/engram/cli.py` | Command-line interface implementation (`engram` command) |
| `src/engram/mcp.py` | MCP server entry point (`piia-engram-mcp`) |
| `src/engram/corpus.py` | Knowledge corpus management with encryption |
| `tests/` | Comprehensive test suite (1000+ tests) |
| `.github/workflows/` | CI/CD pipelines including Codex independent review gate |
Source: [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
---
## Key Features
### 1. MCP Server Integration
Engram operates as a Model Context Protocol server, allowing AI tools to access knowledge through the standardized MCP interface.
```mermaid
graph LR
A["AI Tool
(Claude, CodeX, etc.)"] -->|"MCP JSON-RPC"| B["piia-engram-mcp"]
B -->|"Knowledge Queries"| C["Local Engram Root"]
C -->|"Encrypted Results"| B
B -->|"MCP JSON-RPC"| A
```
**Zero-Install MCP Configuration:**
```bash
uvx --from piia-engram piia-engram-mcp
```
Source: [src/engram/mcp.py](https://github.com/Patdolitse/piia-engram/blob/main/src/engram/mcp.py)
### 2. Hybrid Search Engine
Engram combines keyword search with semantic vector search using Reciprocal Rank Fusion (RRF).
| Search Mode | Description | Use Case |
|-------------|-------------|----------|
| `keyword` | Full-text search (FTS5) | Exact phrase matching, code snippets |
| `semantic` | Embedding-based KNN search | Concept similarity, natural language queries |
| `hybrid` | RRF fusion of both | Best of both worlds, higher recall |
**Important:** When using hybrid search, keyword results are always preserved before applying RRF fusion to ensure keyword recall is not degraded.
Source: [v3.33.2 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.2)
### 3. Identity-Layer Security
```mermaid
graph TD
subgraph "Encryption at Rest"
K["Knowledge Content"] -->|"Encrypt"| E["Encrypted Corpus"]
E -->|"Decrypt on Read"| K
end
subgraph "Permission Boundary"
T1["Tool A - Trust Tier 1"]
T2["Tool B - Trust Tier 2"]
T1 -->|"Full Access"| KG["Knowledge Graph"]
T2 -->|"Limited Access"| KG
end
subgraph "Governance Seal"
G["Governance Layer"] -.->|"Blocks"| WB["Write Bypass"]
G -.->|"Blocks"| RS["Read Side Effects"]
end
```
Every AI tool sees its own permission boundary inline, and the governance layer is sealed against both write bypass and read-path side effects.
Source: [v3.36.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0)
### 4. Decision Threads
Decision threads enable tracking how decisions evolved over time with automatic supersession handling.
- `add_decision` automatically creates a `supersedes` edge when the same question receives a new answer
- Users can trace decision history and understand context changes
Source: [v3.35.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0)
### 5. Session Continuity
```mermaid
graph LR
S1["Session 1
Claude Coding"] -->|"Save"| Sessions["Saved Sessions"]
S2["Session 2
CodeX Review"] -->|"Save"| Sessions
Sessions -->|"List"| CLI["engram sessions"]
CLI -->|"Resume"| S3["Session 3"]
```
The `engram sessions` command lists saved cross-tool agent sessions for continuity across work sessions.
Source: [v3.39.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.0)
---
## CLI Commands
The `engram` CLI provides comprehensive commands for managing the knowledge corpus.
| Command | Description |
|---------|-------------|
| `engram status` | Display system status and MCP client health |
| `engram status --html` | HTML-formatted status with MCP Clients table |
| `engram doctor` | Terminal encoding diagnostics |
| `engram sessions` | List saved cross-tool agent sessions |
| `engram review` | Review staged knowledge |
| `engram repair-encoding` | Dry-run encoding repairs for mojibake |
| `engram repair-encoding --apply` | Apply encoding repairs with backup |
| `engram reindex` | Rebuild search indices |
| `engram setup` | Configure MCP for supported tools |
Source: [src/engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/engram/cli.py)
### Status Command
The `engram status` command provides an at-a-glance view of the system:
```bash
engram status
```
Output includes:
- MCP client configuration health with redacted metadata
- Knowledge corpus statistics
- Next recommended commands
Source: [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
### Encoding Repair
Encoding issues can be diagnosed and repaired using the built-in tools:
```bash
# Dry-run to see what would be repaired
engram repair-encoding
# Apply repairs with timestamped backup
engram repair-encoding --apply
```
The repair tool handles:
- High-confidence mojibake in JSON, JSONL, Markdown, and text files
- Reversible cases are repaired automatically
- Lossy/suspicious cases are reported for manual review
Source: [v3.38.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.38.0)
---
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `ENGRAM_ROOT` | Root directory for Engram data | `~/.engram` |
| `ENGRAM_EMBED_MODEL` | Embedding model for semantic search | Auto-detected |
| `PYTHONIOENCODING` | Python output encoding | System default |
### MCP Setup
Engram supports multiple MCP clients through `engram setup`:
| Client | Config Location |
|--------|-----------------|
| Claude Desktop | `~/Library/Application Support/Claude/claude_desktop_config.json` |
| Trae | `~/.trae/mcp.json` |
| Tencent Tools | Supported via `engram setup` |
Source: [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0)
---
## Data Model
### Knowledge Graph Structure
```mermaid
erDiagram
Knowledge ||--o{ Edge : "supersedes"
Knowledge ||--o{ Edge : "related_to"
Knowledge {
string id
string content
string embed_model
float[] embedding
timestamp created_at
timestamp updated_at
}
Edge {
string source_id
string target_id
string relationship_type
timestamp created_at
}
```
### Index Freshness Tracking
Engram tracks index freshness using a fingerprint that includes:
- Knowledge content hash
- Embedding model identifier
- Vector backend availability
When the embedding model changes, the index is automatically rebuilt to ensure vector dimensions match.
Source: [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
---
## Privacy Model
Engram's privacy architecture is designed around several key principles:
| Principle | Implementation |
|-----------|-----------------|
| **Local Storage** | All data stored in `~/.engram` or `$ENGRAM_ROOT` |
| **Encryption at Rest** | Knowledge corpus encrypted using identity-layer security |
| **No Cloud Dependencies** | Core functionality operates entirely offline |
| **Trust Tiers** | Per-tool permission boundaries enforced on every read path |
| **Governance Sealed** | Write bypass and read-side effects blocked at the governance layer |
Source: [PRIVACY.md](https://github.com/Patdolitse/piia-engram/blob/main/PRIVACY.md)
---
## Security Model
The security implementation follows a defense-in-depth approach:
1. **Identity-Layer Encryption**: Knowledge content encrypted at rest
2. **Governance Cutover**: All governed/export tools enforce trust-tier checks on every read path
3. **File-Side-Effect Harness**: Parametrized tests cover all governed operations
4. **Independent Review**: Codex-based independent security review required for releases
Source: [SECURITY.md](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md), [v3.34.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.34.0)
---
## Release Cadence and Quality Gates
Releases go through multiple quality gates:
```mermaid
graph LR
A["Self Review"] --> B["Codex Independent Review"]
B --> C["Benchmark Gate"]
C --> D["1006+ Tests Pass"]
D --> E["Release Published"]
```
| Gate | Description |
|------|-------------|
| Self Review | Developer internal review |
| Codex Review | Independent security/correctness review |
| Benchmark | Chinese recall zero-regression + cross-language tests |
| Test Suite | Minimum 1006 tests must pass |
Source: [v3.33.2 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.2)
---
## Common Failure Modes
### Encoding Issues
**Symptom:** Display shows garbled characters (mojibake) in terminal output.
**Diagnosis:**
```bash
engram doctor
```
This command helps distinguish between:
- Terminal display mojibake (cosmetic)
- Real data corruption (requires repair)
**Resolution:**
```bash
engram repair-encoding --apply
```
Note: On Windows, `cp65001` is recognized as UTF-8 and avoids false warnings when stdout/stderr are already UTF-8.
Source: [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### Index Freshness Issues
**Symptom:** Semantic search returns empty results after changing embedding model.
**Cause:** Previous index was built with different vector dimensions.
**Resolution:**
```bash
engram reindex
```
This is automatically triggered when `ENGRAM_EMBED_MODEL` changes.
Source: [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
### Hybrid Search Recall Issues
**Symptom:** Keyword matches missing from hybrid search results.
**Resolution:** Hybrid mode now preserves keyword results before RRF fusion, ensuring keyword recall is not degraded.
Source: [v3.33.2 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.2)
---
## Dependencies
### Core Dependencies
| Package | Purpose |
|---------|---------|
| `mcp[dev]` | Model Context Protocol SDK |
| `pydantic` | Data validation and settings |
| `httpx` | HTTP client for embeddings |
| `numpy` | Vector operations |
| `chromadb` | Vector database (optional, for semantic search) |
### Development Dependencies
| Package | Purpose |
|---------|---------|
| `pytest` | Test framework |
| `pytest-asyncio` | Async test support |
| `ruff` | Linting and formatting |
| `mypy` | Type checking |
Source: [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
---
## See Also
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md) — Primary project documentation
- [PRIVACY.md](https://github.com/Patdolitse/piia-engram/blob/main/PRIVACY.md) — Privacy architecture details
- [SECURITY.md](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md) — Security implementation
- [Releases](https://github.com/Patdolitse/piia-engram/releases) — Version history and changelog
- [Model Context Protocol](https://modelcontextprotocol.io/) — MCP protocol specification
---
## Installation
### Related Pages
Related topics: [Quick Start Guide](#quick-start), [Setup Wizard](#setup-wizard)
```markdown
Related Source Files
The following source files were used to generate this page:
- [install.sh](https://github.com/Patdolitse/piia-engram/blob/main/install.sh)
- [install.ps1](https://github.com/Patdolitse/piia-engram/blob/main/install.ps1)
- [Dockerfile](https://github.com/Patdolitse/piia-engram/blob/main/Dockerfile)
- [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
# Installation
This page covers all supported methods for installing and setting up **Engram** (piia-engram), a local-first memory system for AI agents. Engram stores knowledge, decisions, and context in a local corpus with identity-layer encryption and governance controls.
## Overview
Engram supports multiple installation methods targeting different environments and use cases. The project is a Python package distributed via PyPI, with platform-specific shell installers for Linux/macOS and PowerShell installers for Windows. Source: [pyproject.toml:1-15]()
```mermaid
graph TD
A[Choose Installation Method] --> B[Package Managers]
A --> C[Container]
A --> D[Portable Scripts]
B --> B1[pip install piia-engram]
B --> B2[uv pip install piia-engram]
B --> B3[uv tool install piia-engram]
C --> C1[Docker Build]
C --> C2[Docker Pull]
D --> D1[install.sh - Unix/macOS]
D --> D2[install.ps1 - Windows]
B1 --> E[Engram CLI: engram]
B2 --> E
B3 --> E
C1 --> F[Docker Container]
C2 --> F
D1 --> G[Interactive Setup]
D2 --> G
```
## Prerequisites
Before installing Engram, ensure your system meets the following requirements:
| Requirement | Minimum Version | Notes |
|-------------|------------------|-------|
| Python | 3.11+ | Required for core package |
| pip/uv | Latest recommended | uv provides faster installs |
| Git | 2.30+ | For version control integration |
| Disk Space | 500 MB | For package and data storage |
### Optional Dependencies
For full feature availability, the following optional dependencies may be installed:
| Feature | Dependency | Purpose |
|---------|------------|---------|
| Vector Search | sentence-transformers | Semantic embedding for knowledge |
| GUI AI Tools | piia-engram-mcp | MCP server for Claude Desktop, Trae, etc. |
| Docker | docker | Containerized deployment |
## Installation Methods
### Method 1: Package Managers (Recommended)
#### Using pip
The standard Python package installer method:
```bash
pip install piia-engram
```
Source: [pyproject.toml:1-15]()
#### Using uv
The faster Rust-based Python package manager (recommended for development):
```bash
# Install as a regular dependency
uv pip install piia-engram
# Install as a tool (provides console entry points)
uv tool install piia-engram
```
The `uv tool install` method is preferred because it automatically creates executable entry points for the `engram` CLI and `piia-engram-mcp` server. Source: [pyproject.toml:15-25]()
#### Zero-Install MCP Setup with uvx
For MCP clients that want to use Engram without a permanent installation, use `uvx`:
```bash
uvx --from piia-engram piia-engram-mcp
```
This method pulls the latest version on-demand and is ideal for GUI AI tools like Claude Desktop or Trae. Source: [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0)
### Method 2: Shell Script Installers
#### Unix/macOS: install.sh
The `install.sh` script provides an interactive installation experience for Unix-like systems:
```bash
curl -fsSL https://raw.githubusercontent.com/Patdolitse/piia-engram/main/install.sh | bash
```
**Features of install.sh:**
- Detects the current operating system and architecture
- Creates necessary directories under `~/.local/share/engram/`
- Sets up shell completion and environment variables
- Verifies installation integrity with checksum validation
Source: [install.sh](https://github.com/Patdolitse/piia-engram/blob/main/install.sh)
#### Windows: install.ps1
Windows users can use the PowerShell installer:
```powershell
irm https://raw.githubusercontent.com/Patdolitse/piia-engram/main/install.ps1 | iex
```
**Features of install.ps1:**
- Supports Windows PowerShell 5.1+ and PowerShell Core 7+
- Detects UTF-8 encoding issues and configures `PYTHONIOENCODING` if needed
- Creates shortcuts and PATH entries
- Handles Windows-specific directory structures (`%APPDATA%`, `%LOCALAPPDATA%`)
Source: [install.ps1](https://github.com/Patdolitse/piia-engram/blob/main/install.ps1)
### Method 3: Docker Container
Engram can be run in a Docker container for isolated execution:
```dockerfile
# Build from the project's Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY . /app
RUN pip install -e .
ENTRYPOINT ["engram"]
```
**Building and running:**
```bash
# Build the image
docker build -t engram:latest .
# Run with local data volume
docker run -v ~/.local/share/engram:/data engram:latest status
```
Source: [Dockerfile](https://github.com/Patdolitse/piia-engram/blob/main/Dockerfile)
**Docker volume considerations:**
| Volume Path | Purpose | Permissions |
|-------------|---------|-------------|
| `~/.local/share/engram` | Knowledge corpus | Read/Write |
| `/app/logs` | Application logs | Read/Write |
## Post-Installation Setup
After installing Engram, run the initial setup wizard:
```bash
engram setup
```
The setup process performs the following steps:
1. **Corpus initialization** — Creates the local knowledge database
2. **Encryption key generation** — Sets up identity-layer encryption (v3.36.0+)
3. **MCP client configuration** — Detects and configures compatible AI tools
4. **Trust tier assignment** — Establishes governance boundaries
Source: [v3.36.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0)
### GUI AI Tool Setup
Engram integrates with several GUI AI tools via the MCP protocol:
| Tool | Config File | Setup Command |
|------|-------------|---------------|
| Claude Desktop | `~/.claude.json` | `engram setup claude` |
| Trae | `~/.trae/mcp.json` | `engram setup trae` |
| Tencent Hunyuan | Platform-specific | `engram setup tencent` |
Source: [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0)
**Example MCP configuration for Claude Desktop:**
```json
{
"mcpServers": {
"engram": {
"command": "uvx",
"args": ["--from", "piia-engram", "piia-engram-mcp"]
}
}
}
```
## Verifying Installation
After installation, verify that Engram is correctly installed:
```bash
# Check CLI availability
engram --version
# Run diagnostics
engram doctor
# View system status
engram status
```
The `engram doctor` command checks terminal encoding settings and distinguishes between display issues (mojibake) and actual data corruption. Source: [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### Expected output from `engram status`
```
Engram Status
=============
Version: 3.40.0
Corpus: ~/.local/share/engram
Encryption: Enabled
Governance: Active
MCP Clients: 2 configured
```
## Configuration
### Environment Variables
Engram respects the following environment variables:
| Variable | Default | Purpose |
|----------|---------|---------|
| `ENGRAM_ROOT` | `~/.local/share/engram` | Corpus data directory |
| `ENGRAM_EMBED_MODEL` | (built-in default) | Embedding model for vector search |
| `PYTHONIOENCODING` | `utf-8` | Terminal encoding (set automatically) |
### Directory Structure
After installation, Engram creates the following directory structure:
```
~/.local/share/engram/
├── corpus/ # Encrypted knowledge storage
│ ├── knowledge/ # Knowledge entries
│ ├── decisions/ # Decision threads
│ └── sessions/ # Saved agent sessions
├── config/ # Configuration files
│ ├── trust.json # Trust tier definitions
│ └── policy/ # Governance policies
├── logs/ # Application logs
└── cache/ # Temporary cache
```
## Troubleshooting
### Common Installation Issues
| Issue | Cause | Resolution |
|-------|-------|------------|
| `command not found: engram` | PATH not updated | Restart shell or run `source ~/.bashrc` |
| Mojibake in output | Terminal encoding mismatch | Run `engram doctor` to diagnose; set `PYTHONIOENCODING=utf-8` |
| Permission denied | Directory permissions | Check `~/.local/share/engram` permissions |
| MCP connection failed | Config file error | Run `engram setup` to regenerate configs |
### Encoding Diagnostics
If you experience character display issues, the `engram doctor` command provides detailed diagnostics:
```bash
engram doctor --encoding
```
This command:
- Detects terminal encoding capabilities
- Recognizes Windows `cp65001` as UTF-8
- Warns only when actual encoding mismatches exist
Source: [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### Repairing Corrupted Data
For data corruption issues, use the encoding repair tool:
```bash
# Dry-run to see what would be repaired
engram repair-encoding --dry-run
# Apply repairs with timestamped backup
engram repair-encoding --apply
```
Source: [v3.38.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.38.0)
## Upgrading
### Package Manager Upgrades
```bash
# pip
pip install --upgrade piia-engram
# uv
uv pip install --upgrade piia-engram
uv tool upgrade piia-engram
```
### Reindexing After Upgrades
When upgrading to a new version with embedding model changes, Engram automatically detects the change and triggers reindexing. To manually trigger a reindex:
```bash
engram reindex
```
This ensures vector embeddings are regenerated with the new model, preventing silent degradation of semantic search quality. Source: [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
## Uninstallation
### Package Manager Removal
```bash
# pip
pip uninstall piia-engram
# uv
uv tool uninstall piia-engram
```
### Removing Data
To completely remove Engram and all local data:
```bash
# Remove the corpus and configuration
rm -rf ~/.local/share/engram
# Remove shell completions (manual)
# Edit ~/.bashrc or ~/.zshrc and remove engram-related lines
```
## See Also
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md) — Project overview and quick start
- [Security Architecture](wiki/security-architecture) — Identity-layer encryption and governance
- [MCP Integration](wiki/mcp-integration) — Connecting Engram to AI tools
- [CLI Reference](wiki/cli-reference) — Complete `engram` command documentation
---
## Quick Start Guide
### Related Pages
Related topics: [Installation](#installation), [Setup Wizard](#setup-wizard)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/setup_wizard.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/setup_wizard.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [demos/setup_engram.py](https://github.com/Patdolitse/piia-engram/blob/main/demos/setup_engram.py)
- [.mcp/server.json](https://github.com/Patdolitse/piia-engram/blob/main/.mcp/server.json)
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
- [src/piia_engram/status.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/status.py)
- [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
# Quick Start Guide
Welcome to Engram, a local-first knowledge management system designed for AI agents and developers. This guide walks you through installation, initial setup, and basic usage patterns to get you productive in minutes.
## Overview
Engram is a privacy-focused knowledge management tool that runs entirely on your local machine. It provides:
- **Knowledge Corpus**: Store and retrieve structured knowledge with semantic and keyword search
- **MCP Server Integration**: Connect AI tools like Claude Desktop, Cursor, and Trae directly to your knowledge base
- **Session Continuity**: Track cross-tool agent sessions with saved state
- **Decision Tracking**: Maintain decision threads with provenance and supersession chains
- **Encoding Repair**: Diagnose and fix text encoding issues automatically
**Core Principle**: Engram never depends on third-party cloud services. All data remains on your machine.
Source: [README.md:1-10]()
## Prerequisites
Before installing Engram, ensure your system meets these requirements:
| Requirement | Minimum | Recommended |
|-------------|---------|-------------|
| Python | 3.10+ | 3.11+ |
| Disk Space | 100 MB | 500 MB for vector index |
| Memory | 512 MB | 2 GB for embedding models |
| OS | Windows/macOS/Linux | Any with UTF-8 terminal |
### Required Environment Configuration
Set the following environment variables before first-run:
```bash
# Required for terminal encoding (Windows especially)
export PYTHONIOENCODING=utf-8
# Optional: Override default embedding model
# export ENGRAM_EMBED_MODEL=text-embedding-3-small
```
Source: [src/piia_engram/cli.py:42-58]()
## Installation
### Method 1: pip (Recommended for CLI Users)
```bash
pip install piia-engram
```
This installs the `engram` CLI and all core dependencies.
Source: [pyproject.toml:1-15]()
### Method 2: uvx (Zero-Install for MCP Clients)
For MCP server integration without full installation:
```bash
uvx --from piia-engram piia-engram-mcp
```
Source: [.mcp/server.json:1-20]()
### Method 3: Development Installation
```bash
git clone https://github.com/Patdolitse/piia-engram.git
cd piia-engram
pip install -e ".[dev]"
```
## First-Run Setup
The first time you run `engram`, the setup wizard guides you through configuration.
```bash
engram setup
```
### Setup Wizard Flow
```mermaid
graph TD
A[engram setup] --> B[Detect Environment]
B --> C{Has existing config?}
C -->|No| D[Create Engram Root]
C -->|Yes| E[Load existing config]
D --> F[Configure MCP Clients]
E --> F
F --> G[Initialize Knowledge Corpus]
G --> H[Run diagnostics]
H --> I[Setup complete]
```
### MCP Client Configuration
The setup wizard supports configuring multiple MCP clients. Currently supported:
| Client | Config Path |
|--------|-------------|
| Claude Desktop | `~/Library/Application Support/Claude/claude_desktop_config.json` |
| Cursor | `~/.cursor/mcp.json` |
| Trae | `~/.trae/mcp.json` |
| Tencent (custom) | `~/.tencent/mcp.json` |
Source: [src/piia_engram/setup_wizard.py:85-120]()
## Verifying Installation
After setup, verify everything works correctly:
```bash
engram status
```
The status command provides an MCP client configuration health summary with redacted metadata. For HTML output with detailed tables:
```bash
engram status --html
```
### Status Output Interpretation
| Field | Healthy | Issue |
|-------|---------|-------|
| Engram Root | Exists and writable | Missing or permission denied |
| MCP Clients | Connected | Misconfigured or missing |
| Knowledge Corpus | Indexed | Empty or corrupted |
Source: [src/piia_engram/status.py:45-80]()
## Running Diagnostics
If you encounter issues, use the doctor command:
```bash
engram doctor
```
This command:
1. Checks terminal encoding configuration
2. Validates Engram root permissions
3. Tests MCP server connectivity
4. Reports corpus integrity
Source: [src/piia_engram/cli.py:120-145]()
### Terminal Encoding Diagnostics
v3.39.1 introduced enhanced encoding diagnostics that distinguish between:
- Terminal display mojibake (visual artifacts)
- Actual data corruption in Engram files
The command recognizes Windows `cp65001` as UTF-8 and avoids false warnings when stdout/stderr are already UTF-8.
Source: [src/piia_engram/cli.py:130-135]()
## Basic CLI Commands
### Knowledge Management
```bash
# Add knowledge to your corpus
engram add --type decision --content "Use PostgreSQL for user data"
# Search knowledge base
engram search "database selection"
# List recent entries
engram list --limit 10
```
### Session Management
```bash
# List saved agent sessions
engram sessions
# Review staged knowledge
engram review
```
Source: [demos/setup_engram.py:30-55]()
## MCP Server Integration
### Quick MCP Setup
Add Engram as an MCP server in your AI tool's configuration:
```json
{
"mcpServers": {
"engram": {
"command": "uvx",
"args": ["--from", "piia-engram", "piia-engram-mcp"]
}
}
}
```
Source: [.mcp/server.json:1-25]()
### GUI Setup via CLI
For GUI tools that support it, use the built-in setup:
```bash
engram setup --client claude-desktop
```
This automatically configures the specified client without manual file editing.
Source: [src/piia_engram/setup_wizard.py:150-180]()
## Data Flow Architecture
```mermaid
graph LR
subgraph "User Layer"
A[CLI Commands]
B[MCP Clients]
end
subgraph "Engram Core"
C[Knowledge Corpus]
D[Search Engine]
E[Decision Tracker]
end
subgraph "Storage Layer"
F[SQLite Database]
G[Vector Index]
H[File System]
end
A --> C
B --> C
C --> F
C --> G
C --> H
D --> G
E --> F
```
## Common Workflows
### Workflow 1: Daily Knowledge Capture
```bash
# Capture a decision
engram add decision "Migrated auth to OAuth2"
# Verify it's indexed
engram search "OAuth2 auth migration"
```
### Workflow 2: AI Agent Session Review
```bash
# After an agent session
engram sessions
# Review what was captured
engram review
```
### Workflow 3: Encoding Repair
If you encounter mojibake in terminal output:
```bash
# Dry-run to see what would be fixed
engram repair-encoding
# Apply fixes with backup
engram repair-encoding --apply
```
Source: [demos/setup_engram.py:60-85]()
## Configuration Reference
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `ENGRAM_ROOT` | `~/.engram` | Root directory for all data |
| `ENGRAM_EMBED_MODEL` | `text-embedding-3-small` | Embedding model for vector search |
| `PYTHONIOENCODING` | System default | Terminal encoding (set to `utf-8`) |
### Directory Structure
```
~/.engram/
├── corpus/ # Knowledge content (encrypted at rest)
├── index/ # Vector and FTS indexes
├── sessions/ # Saved agent sessions
├── decisions/ # Decision thread data
└── config.json # Local configuration
```
Source: [src/piia_engram/setup_wizard.py:200-220]()
## Troubleshooting
### Issue: Mojibake in Output
This is likely a terminal encoding issue, not data corruption. Run:
```bash
engram doctor
```
The diagnostic will distinguish between display issues and actual corruption.
Source: [src/piia_engram/cli.py:130-135]()
### Issue: MCP Client Not Connecting
1. Verify the MCP server is running: `engram status`
2. Check client config has correct path to `piia-engram-mcp`
3. Review logs: `engram status --html`
### Issue: Empty Search Results After Adding Content
The index may need rebuilding:
```bash
engram reindex
```
This rebuilds both keyword and vector indexes.
Source: [src/piia_engram/cli.py:180-195]()
## Next Steps
After completing this guide:
1. **Explore CLI Commands**: Run `engram --help` for all available commands
2. **Configure MCP Clients**: Connect your preferred AI tools
3. **Review Session Features**: Use `engram sessions` to track agent workflows
4. **Understand Security**: Review the Identity-Layer Security features in v3.36.0
## See Also
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md) - Project overview and detailed documentation
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) - Complete command documentation
- [MCP Server Setup](https://github.com/Patdolitse/piia-engram/wiki/MCP-Server-Setup) - Detailed MCP integration guide
- [Release Notes](https://github.com/Patdolitse/piia-engram/releases) - Version history and changelog
- [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0) - First-run confidence improvements
- [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1) - Terminal encoding diagnostics
- [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0) - GUI entry adoption
---
## Setup Wizard
### Related Pages
Related topics: [Quick Start Guide](#quick-start), [MCP Tools Reference](#mcp-tools)
```details
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/setup_wizard.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/setup_wizard.py)
- [demos/setup_engram.py](https://github.com/Patdolitse/piia-engram/blob/main/demos/setup_engram.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [src/piia_engram/config.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/config.py)
- [src/piia_engram/mcp_config.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_config.py)
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
# Setup Wizard
The Setup Wizard is Engram's guided initialization system that walks users through configuring their local-first knowledge corpus. It handles the creation of the Engram root directory, establishes the initial knowledge graph structure, configures MCP (Model Context Protocol) client integrations, and applies security settings such as corpus encryption.
## Overview
When a user runs `engram setup`, the wizard performs a series of initialization tasks that establish the foundation for all subsequent Engram operations. The setup process is designed to be idempotent—running it multiple times is safe and will not corrupt existing data.
```mermaid
graph TD
A[User runs engram setup] --> B{Engram Root Exists?}
B -->|No| C[Create Engram Root Directory]
B -->|Yes| D[Verify Existing Configuration]
C --> E[Initialize Knowledge Graph DB]
D --> E
E --> F[Configure MCP Clients]
F --> G[Apply Security Settings]
G --> H[Display Setup Summary]
H --> I[Setup Complete]
```
## Command Interface
The Setup Wizard is invoked through the `engram setup` CLI command. It accepts several flags to customize the initialization behavior.
### Basic Usage
```bash
engram setup
```
### Command Flags
| Flag | Description | Default |
|------|-------------|---------|
| `--gui` | Enable GUI-based AI tool integration | `false` |
| `--cursor` | Configure for Cursor IDE | `false` |
| `--claude-desktop` | Configure for Claude Desktop | `false` |
| `--trae` | Configure for Trae Editor | `false` |
| `--windsurf` | Configure for Windsurf IDE | `false` |
| `--help` | Display help message | N/A |
Source: [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
## MCP Client Configuration
The Setup Wizard supports configuration of multiple MCP clients simultaneously. This allows users to integrate Engram with various AI-assisted development environments.
### Supported Clients
The following table lists the MCP clients that the Setup Wizard can configure:
| Client | Config File | Description |
|--------|-------------|-------------|
| Cursor | `~/.cursor/mcp.json` | AI-first code editor |
| Claude Desktop | `~/.claude_desktop_config.json` | Anthropic's desktop application |
| Trae | `~/.trae/mcp.json` | Tencent's AI code editor |
| Windsurf | `~/.codeium/workspace/mcp_config.json` | Codeium's AI IDE |
Source: [src/piia_engram/mcp_config.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_config.py)
### Configuration Structure
The MCP configuration uses the standard MCP JSON format. The Setup Wizard generates a configuration entry that points to the `piia-engram-mcp` console script, which can be invoked using `uvx` for zero-install operation:
```json
{
"mcpServers": {
"engram": {
"command": "uvx",
"args": ["--from", "piia-engram", "piia-engram-mcp"]
}
}
}
```
This zero-install approach (introduced in v3.37.0) eliminates the need for users to manually install the package before configuring MCP clients.
## Initialization Workflow
The Setup Wizard executes a structured initialization sequence that ensures all required components are properly configured.
### Step 1: Engram Root Creation
The wizard first determines the location of the Engram root directory, typically `~/.engram/`. If the directory does not exist, it creates the complete directory structure including subdirectories for the knowledge graph, configuration, and encrypted corpus storage.
### Step 2: Knowledge Graph Initialization
The knowledge graph database is initialized using SQLite with graph extensions. This establishes the schema for nodes, edges, and metadata required for knowledge representation.
### Step 3: Configuration File Setup
The wizard creates the `engram.json` configuration file in the Engram root with sensible defaults:
```json
{
"version": "3.40.0",
"corpus_encryption": "enabled",
"trust_tier": "local",
"embed_model": "default"
}
```
### Step 4: Security Configuration
If corpus encryption is enabled (as it is by default since v3.36.0), the wizard ensures that the encryption layer is properly initialized. Every AI tool sees its own permission boundary inline, and the governance layer is sealed against both write bypass and read-path side effects.
## Demo Integration
A demonstration script is provided in the repository that showcases the setup workflow programmatically.
**File:** [demos/setup_engram.py](https://github.com/Patdolitse/piia-engram/blob/main/demos/setup_engram.py)
This demo illustrates how to:
1. Initialize an Engram instance programmatically
2. Configure MCP clients for specific IDEs
3. Verify the setup was successful
## First-Run Confidence
Since v3.40.0, the Setup Wizard contributes to first-run confidence by ensuring that all configuration is properly established before users begin adding knowledge. The `engram status` command provides a summary of the setup health after initialization.
After running `engram setup`, users are encouraged to run:
```bash
engram status
```
This displays the current configuration state including MCP client configuration health (with redacted metadata) and recommended next steps.
## Troubleshooting
If the Setup Wizard encounters issues, the following diagnostic commands are available:
| Command | Purpose |
|---------|---------|
| `engram doctor` | Run terminal encoding diagnostics and verify setup integrity |
| `engram status` | Display current configuration state and health |
| `engram repair-encoding` | Repair mojibake in Engram data files (dry-run by default) |
### Common Issues
**Issue:** MCP clients not appearing after setup
**Resolution:** Verify the MCP config file was created in the correct location for your client. Run `engram doctor` to check configuration health.
**Issue:** Unicode/encoding warnings during setup
**Resolution:** Ensure your terminal encoding is set to UTF-8. The `engram doctor` command can distinguish between terminal display mojibake and actual data corruption (v3.39.1).
## Configuration Options
The Setup Wizard respects several environment variables that influence its behavior:
| Variable | Description | Default |
|----------|-------------|---------|
| `ENGRAM_ROOT` | Override the default Engram root directory | `~/.engram/` |
| `ENGRAM_EMBED_MODEL` | Specify the embedding model for semantic search | System default |
| `PYTHONIOENCODING` | Terminal encoding (recommended: `utf-8`) | Platform default |
## See Also
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) - Complete command-line interface documentation
- [MCP Integration](https://github.com/Patdolitse/piia-engram/wiki/MCP-Integration) - Detailed MCP client configuration guide
- [Security Model](https://github.com/Patdolitse/piia-engram/wiki/Security-Model) - Corpus encryption and governance documentation
- [Troubleshooting Guide](https://github.com/Patdolitse/piia-engram/wiki/Troubleshooting) - Common issues and resolutions
---
## System Architecture
### Related Pages
Related topics: [Core Components](#components), [Security and Encryption](#security-encryption)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/__init__.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/__init__.py)
- [src/piia_engram/core.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py)
- [src/piia_engram/mcp_server.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py)
- [src/piia_engram/storage.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/storage.py)
- [src/piia_engram/search.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [src/piia_engram/governance.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
- [src/piia_engram/encryption.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/encryption.py)
- [src/piia_engram/models.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/models.py)
# System Architecture
## Overview
Piia Engram (piia-engram) is a local-first knowledge memory system designed for AI developers and agents. The architecture emphasizes **privacy by default**, **zero external dependencies**, and **structured knowledge management** through the Model Context Protocol (MCP). Source: [src/piia_engram/__init__.py:1-20]()
The core philosophy is that all knowledge content remains local, encrypted at rest, with every AI tool operating within its own permission boundary. This design enables AI assistants and agents to maintain persistent context across sessions while preserving data sovereignty.
## High-Level Architecture
```mermaid
graph TD
subgraph "Client Layer"
CLI[CLI: engram]
MCP[MCP Clients]
GUI[GUI Tools]
end
subgraph "Service Layer"
MCPS[MCP Server]
Core[Core Engine]
Search[Search Engine]
Gov[Governance Layer]
end
subgraph "Data Layer"
Storage[(Storage Backend)]
Index[(Index Store)]
Crypto[(Encrypted Corpus)]
Config[(Config/State)]
end
CLI --> Core
MCP --> MCPS
MCPS --> Core
GUI --> MCP
Core --> Search
Core --> Gov
Core --> Storage
Search --> Index
Storage --> Crypto
Gov --> Config
Storage -.-> Index
```
## Core Components
Piia Engram comprises several interconnected subsystems that work together to provide persistent memory capabilities.
### Component Overview
| Component | File | Responsibility |
|-----------|------|----------------|
| Core Engine | [core.py](src/piia_engram/core.py) | Central orchestration, session management, knowledge CRUD |
| MCP Server | [mcp_server.py](src/piia_engram/mcp_server.py) | Model Context Protocol interface for AI tool integration |
| Storage | [storage.py](src/piia_engram/storage.py) | File-based persistence with encryption at rest |
| Search | [search.py](src/piia_engram/search.py) | Hybrid keyword and vector semantic search |
| Governance | [governance.py](src/piia_engram/governance.py) | Trust-tier enforcement, permission profiles |
| Encryption | [encryption.py](src/piia_engram/encryption.py) | Corpus encryption and identity-layer security |
| CLI | [cli.py](src/piia_engram/cli.py) | Command-line interface for all operations |
| Models | [models.py](src/piia_engram/models.py) | Data models for decisions, sessions, permissions |
## Data Layer Architecture
### Storage Subsystem
The storage layer provides the foundation for persistent knowledge management. It implements a file-based storage model with encryption at rest.
```mermaid
graph LR
A[Knowledge Content] --> B[Encryption Layer]
B --> C[File System]
C --> D[Storage Index]
D --> E[Query Interface]
```
**Key Storage Features:**
- **Corpus Encryption**: All knowledge content is encrypted at rest using identity-layer security. Source: [encryption.py](src/piia_engram/encryption.py)
- **Freshness Fingerprinting**: Storage tracks content changes and embedding model changes to trigger index rebuilds when necessary. Source: [storage.py](src/piia_engram/storage.py)
- **Backup System**: Encoding repairs create timestamped backups before applying fixes. Source: [src/piia_engram/repair.py](src/piia_engram/repair.py)
### Data Models
The system uses structured data models for organizing knowledge:
| Model | Purpose | Key Attributes |
|-------|---------|----------------|
| Decision | Tracked decisions with evolution history | `id`, `content`, `supersedes`, `created_at` |
| Session | Cross-tool agent session continuity | `id`, `tools`, `context`, `metadata` |
| Permission Profile | Access control per AI tool | `tool_id`, `trust_tier`, `boundaries` |
| Knowledge Unit | Atomic knowledge content | `id`, `content`, `embedding`, `tags` |
Source: [models.py](src/piia_engram/models.py)
## Search Architecture
### Hybrid Search Implementation
Piia Engram implements a hybrid search strategy combining keyword matching with semantic vector search.
```mermaid
graph TD
Q[Query] --> KW[Keyword Search]
Q --> SEM[Semantic Search]
KW --> RRF[RRF Reranking]
SEM --> RRF
RRF --> FINAL[Final Results]
RRF -.->|Preserve| KWHits[Keyword Hits]
```
**Search Features:**
| Feature | Description |
|---------|-------------|
| Keyword Search | Full-text search using FTS (Full-Text Search) indexing |
| Semantic Search | Vector-based KNN search using embeddings |
| Hybrid RRF | Reciprocal Rank Fusion for result merging |
| Recall Guarantee | Keyword hits are preserved before RRF filling |
Source: [search.py](src/piia_engram/search.py)
**Index Freshness Management:**
- Content changes trigger index updates
- Embedding model changes force complete index rebuild
- Vector backend availability is tracked in freshness fingerprints
Source: [storage.py](src/piia_engram/storage.py) - "freshness fingerprint"
## MCP Integration
### Model Context Protocol Server
The MCP server provides a standardized interface for AI tools to interact with Engram's memory capabilities.
```mermaid
graph LR
AI[AI Tool] --> MCP[MCP Client]
MCP -->|JSON-RPC| Server[MCP Server]
Server --> Core[Core Engine]
Core --> Response[Tool Response]
Response --> AI
```
**MCP Entry Points:**
| Entry Point | Command | Use Case |
|-------------|---------|----------|
| CLI | `engram` | Terminal operations |
| MCP Server | `piia-engram-mcp` | GUI AI tools integration |
| Zero-Install | `uvx --from piia-engram piia-engram-mcp` | Quick MCP setup |
Source: [mcp_server.py](src/piia_engram/mcp_server.py)
**Supported GUI Tools:**
- Trae (`~/.trae/mcp.json`)
- Tencent platforms
Source: [cli.py](src/piia_engram/cli.py) - "setup" command
### MCP Client Configuration
The system provides diagnostic capabilities for MCP client health:
```bash
engram status
engram status --html
```
The status command summarizes MCP client configuration with redacted metadata, helping users understand their current setup without exposing sensitive information. Source: [core.py](src/piia_engram/core.py)
## Security Layer
### Identity-Layer Security
The security architecture implements multiple layers of protection:
```mermaid
graph TD
subgraph "Security Layers"
E[Encryption at Rest]
P[Permission Profiles]
G[Governance]
T[Trust Tiers]
end
E --> P
P --> G
G --> T
```
**Security Components:**
| Layer | Implementation | File |
|-------|----------------|------|
| Encryption | Corpus-level encryption for all stored content | [encryption.py](src/piia_engram/encryption.py) |
| Permission Profiles | Per-tool access boundaries | [governance.py](src/piia_engram/governance.py) |
| Trust Tiers | Read-path enforcement on every access | [governance.py](src/piia_engram/governance.py) |
| Governance Sealing | Protection against write bypass and read-path side effects | [governance.py](src/piia_engram/governance.py) |
Source: [v3.36.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0)
### Governance Architecture
The governance layer ensures consistent permission enforcement:
1. **Trust-tier checks** execute on every read path
2. **File-side-effect harness** validates all governed operations
3. **Playbook policies** define allowed operations per trust level
Source: [governance.py](src/piia_engram/governance.py)
## CLI Architecture
### Command Structure
The CLI provides comprehensive commands for all operations:
| Command | Purpose |
|---------|---------|
| `engram status` | Show MCP client configuration health |
| `engram doctor` | Terminal encoding diagnostics |
| `engram sessions` | List saved cross-tool agent sessions |
| `engram review` | Staged knowledge review |
| `engram repair-encoding` | Encoding repair with dry-run support |
| `engram reindex` | Force index rebuild |
| `engram setup` | Configure MCP for GUI tools |
Source: [cli.py](src/piia_engram/cli.py)
### Status Command Output
The `engram status` command provides:
- MCP Clients table with redacted metadata
- Next recommended commands (`engram doctor`, `engram review`, `engram sessions`)
- Configuration health summary
Source: [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
## Decision Management
### Decision Threads
The system tracks decision evolution through decision threads:
```mermaid
graph LR
D1[Decision v1] -->|supersedes| D2[Decision v2]
D2 -->|supersedes| D3[Decision v3]
```
**Features:**
- **Auto-supersedes**: New decisions automatically create `supersedes` edges
- **History tracing**: Users can trace how decisions evolved
- **Thread continuity**: Decision context persists across sessions
Source: [v3.35.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0)
## Configuration
### Environment Variables
| Variable | Purpose | Default |
|----------|---------|---------|
| `ENGRAM_ROOT` | Engram data directory | `~/.engram` |
| `ENGRAM_EMBED_MODEL` | Embedding model for semantic search | Auto-detected |
### Configuration Files
| File | Purpose |
|------|---------|
| `~/.engram/config.json` | Main configuration |
| `~/.engram/corpus.db` | Encrypted knowledge storage |
| `~/.engram/index/` | Search indexes |
| `~/.trae/mcp.json` | Trae MCP configuration |
Source: [storage.py](src/piia_engram/storage.py)
## Data Flow: Knowledge Addition
```mermaid
sequenceDiagram
participant User as CLI/MCP Client
participant Core as Core Engine
participant Crypto as Encryption Layer
participant Storage as Storage Backend
participant Index as Search Index
participant Gov as Governance
User->>Core: add_knowledge(content)
Core->>Gov: check_permission(tool_id, write)
Gov-->>Core: allowed
Core->>Crypto: encrypt(content)
Crypto-->>Core: encrypted_blob
Core->>Storage: persist(encrypted_blob)
Storage->>Index: update_index(content, embedding)
Core-->>User: success
```
## Common Failure Modes
### Encoding Issues
| Symptom | Cause | Resolution |
|---------|-------|------------|
| Mojibake in display | Terminal encoding mismatch | Run `engram doctor` |
| Data corruption | Real encoding damage | `engram repair-encoding --apply` |
| Windows cp65001 | UTF-8 not recognized | Set `PYTHONIOENCODING=utf-8` |
Source: [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### Index Freshness Issues
| Symptom | Cause | Resolution |
|---------|-------|------------|
| Empty semantic results | Stale index after model change | `engram reindex` |
| Hybrid search missing results | RRF truncating keyword hits | Upgraded to guarantee keyword preservation |
Source: [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
### MCP Connection Issues
| Symptom | Cause | Resolution |
|---------|-------|------------|
| MCP client not detected | Missing configuration | `engram status` to diagnose |
| Permission denied | Trust tier too low | Check governance settings |
Source: [core.py](src/piia_engram/core.py)
## Local-First Principles
Piia Engram's architecture strictly follows local-first principles:
1. **No cloud dependencies**: All data stays on the local filesystem
2. **Zero third-party services**: No external API calls for core functionality
3. **Encryption at rest**: Knowledge corpus is always encrypted
4. **Permission isolation**: Each AI tool sees only its permitted content
This design was explicitly confirmed when declining third-party API partnership proposals, emphasizing that recommending specific API relay services contradicts the local-first positioning. Source: [Issue #8](https://github.com/Patdolitse/piia-engram/issues/8)
## See Also
- [Installation Guide](../Installation) — Setting up piia-engram locally
- [CLI Reference](../CLI-Reference) — Complete command documentation
- [MCP Integration](../MCP-Integration) — Connecting AI tools to Engram
- [Security Model](../Security-Model) — Deep dive into encryption and governance
- [Search Configuration](../Search-Configuration) — Tuning hybrid search performance
---
## Core Components
### Related Pages
Related topics: [System Architecture](#architecture), [MCP Tools Reference](#mcp-tools), [Security and Encryption](#security-encryption)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/core.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py)
- [src/piia_engram/mcp_server.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py)
- [src/piia_engram/storage.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/storage.py)
- [src/piia_engram/crypto.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py)
- [src/piia_engram/governance.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
- [src/piia_engram/governance_runtime.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance_runtime.py)
- [src/piia_engram/hooks/__init__.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/__init__.py)
- [src/piia_engram/search.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [src/piia_engram/sessions.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
# Core Components
Piia Engram is a local-first knowledge management system designed for AI agents and developers who need persistent, privacy-preserving memory capabilities. The system provides encrypted storage, governance-based access control, and Model Context Protocol (MCP) integration for seamless AI tool connectivity.
## System Architecture Overview
The architecture follows a layered design with clear separation of concerns between storage, security, governance, and interface layers.
```mermaid
graph TD
subgraph "Interface Layer"
CLI[CLI: engram]
MCP[MCP Server]
API[Python API]
end
subgraph "Governance Layer"
GR[Governance Runtime]
GOV[Governance Engine]
GCONF[Governance Config]
end
subgraph "Core Layer"
CORE[Core Engine]
HOOKS[Hooks System]
SEARCH[Search Engine]
end
subgraph "Security Layer"
CRYPTO[Encryption]
AUTH[Trust Tiers]
end
subgraph "Storage Layer"
KB[Knowledge Base]
SESSIONS[Session Store]
IDX[Index Store]
end
CLI --> CORE
MCP --> CORE
API --> CORE
CORE --> GR
CORE --> HOOKS
CORE --> SEARCH
CORE --> CRYPTO
GR --> AUTH
CORE --> KB
CORE --> SESSIONS
SEARCH --> IDX
KB --> CRYPTO
SESSIONS --> CRYPTO
GR --> GOV
GOV --> GCONF
```
**Source:** [src/piia_engram/core.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py), [src/piia_engram/governance_runtime.py:1-30](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance_runtime.py)
---
## Core Engine (`core.py`)
The core engine serves as the central orchestrator for all Engram operations. It initializes subsystems, manages configuration, and provides the primary API for knowledge management operations.
### Purpose and Scope
The core engine handles:
- System initialization and configuration loading
- Command routing and execution
- Integration between storage, governance, and search subsystems
- Trust tier validation for AI tool access
**Source:** [src/piia_engram/core.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py)
### Key Responsibilities
| Responsibility | Description | Module |
|----------------|-------------|--------|
| Configuration Management | Loads and validates `engram.yaml` settings | `core.py` |
| Trust Tier Validation | Enforces access control based on tool identity | `governance_runtime.py` |
| Knowledge Operations | Add, retrieve, update, delete knowledge entries | `core.py`, `storage.py` |
| Session Management | Maintains cross-tool agent session continuity | `sessions.py` |
**Source:** [src/piia_engram/core.py:50-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py)
---
## Storage Layer (`storage.py`)
The storage layer provides persistent data management for knowledge content, sessions, and indexes. All data is encrypted at rest using corpus-level encryption introduced in v3.36.0.
### Data Models
```mermaid
erDiagram
KNOWLEDGE_ENTRY ||--o| ENCRYPTED_BLOCK : contains
KNOWLEDGE_ENTRY {
string id PK
string content
string embed_model
timestamp created_at
timestamp updated_at
string tool_identity FK
}
ENCRYPTED_BLOCK {
blob ciphertext
string algorithm
string key_id
}
SESSION ||--o{ KNOWLEDGE_ENTRY : references
SESSION {
string id PK
string name
timestamp created_at
json metadata
}
```
**Source:** [src/piia_engram/storage.py:1-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/storage.py)
### Storage Operations
| Operation | Method | Description |
|-----------|--------|-------------|
| Add Knowledge | `add_knowledge(content, metadata)` | Stores new knowledge with encryption |
| Retrieve | `get_knowledge(id)` | Decrypts and returns knowledge entry |
| Update | `update_knowledge(id, content)` | Modifies existing entry, re-encrypts |
| Delete | `delete_knowledge(id)` | Removes entry and associated indexes |
| Search | `search(query, mode)` | Performs hybrid/keyword/vector search |
**Source:** [src/piia_engram/storage.py:100-200](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/storage.py)
### Encryption at Rest
Starting with v3.36.0, all knowledge content is encrypted at rest. The encryption system ensures that:
- Each knowledge corpus has its own encryption keys
- AI tools see only their permission boundaries inline
- Governance layer is sealed against write bypass and read-path side effects
**Source:** [src/piia_engram/crypto.py:1-60](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py), [v3.36.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0)
---
## Security Layer (`crypto.py`)
The crypto module implements identity-layer security features including encryption at rest and key management.
### Encryption Features
| Feature | Description | Trust Tier Required |
|---------|-------------|---------------------|
| Corpus Encryption | AES-256-GCM encryption for all stored content | All tiers |
| Key Derivation | PBKDF2-based key derivation from master secret | All tiers |
| Per-Tool Keys | Separate encryption contexts per AI tool | High Trust |
| Backup Encryption | Timestamped encrypted backups during repair | All tiers |
**Source:** [src/piia_engram/crypto.py:50-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py)
### Key Management Flow
```mermaid
graph LR
A[Master Secret] --> B[Key Derivation]
B --> C[Corpus Master Key]
C --> D[Content Encryption Key]
D --> E[Encrypted Data]
F[Tool Identity] --> G[Trust Validation]
G --> H[Key Access]
H --> D
```
**Source:** [src/piia_engram/crypto.py:100-130](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py)
---
## Governance Layer (`governance.py`, `governance_runtime.py`)
The governance layer implements access control and permission boundaries. This was fully enforced across all read paths starting with v3.34.0.
### Governance Architecture
```mermaid
graph TD
subgraph "Governance Engine"
G1[Read Path Check]
G2[Write Path Check]
G3[Trust Tier Evaluation]
end
subgraph "Trust Tiers"
T1[Low Trust]
T2[Medium Trust]
T3[High Trust]
end
subgraph "Policy Enforcement"
P1[Permission Profile]
P2[Decision Threads]
P3[Playbook Policy]
end
G1 --> G3
G2 --> G3
G3 --> T1
G3 --> T2
G3 --> T3
T1 --> P1
T2 --> P2
T3 --> P3
```
**Source:** [src/piia_engram/governance.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py), [src/piia_engram/governance_runtime.py:1-40](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance_runtime.py)
### Governance Components
| Component | File | Purpose |
|-----------|------|---------|
| Policy Engine | `governance.py` | Defines permission rules and access policies |
| Runtime Enforcer | `governance_runtime.py` | Validates trust tiers on every operation |
| Decision Threads | `governance.py` | Tracks decision history and supersession chains |
| Permission Profile | `governance.py` | User-controllable access boundaries |
**Source:** [src/piia_engram/governance.py:50-120](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py), [v3.35.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0)
### Decision Thread Auto-Supersedes
When the same question receives a new answer, the governance system automatically creates a `supersedes` edge linking the new decision to the previous one. This enables tracing how decisions evolved over time.
```python
# Example: Decision thread structure
{
"decision_id": "d123",
"question": "How to configure MCP?",
"answer": "Use engram setup --mcp",
"supersedes": "d122", # Auto-created edge
"timestamp": "2025-01-15T10:30:00Z"
}
```
**Source:** [src/piia_engram/governance.py:150-200](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py), [v3.35.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0)
---
## MCP Server Integration (`mcp_server.py`)
The MCP server provides Model Context Protocol integration, enabling AI tools like Claude Desktop, Cursor, and Trae to access Engram's knowledge base.
### MCP Configuration
Zero-install MCP configuration is available via `uvx`:
```bash
uvx --from piia-engram piia-engram-mcp
```
**Source:** [src/piia_engram/mcp_server.py:1-60](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py), [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0)
### MCP Setup Support
| AI Tool | Config File | Command |
|---------|-------------|---------|
| Claude Desktop | `~/.claude/mcp.json` | `engram setup --mcp claude` |
| Cursor | `~/.cursor/mcp.json` | `engram setup --mcp cursor` |
| Trae | `~/.trae/mcp.json` | `engram setup --mcp trae` |
| Tencent Hunyuan | `~/.config/hunyuan/mcp.json` | `engram setup --mcp hunyuan` |
**Source:** [src/piia_engram/mcp_server.py:60-120](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py)
### MCP Server Responsibilities
| Responsibility | Description |
|----------------|-------------|
| Tool Discovery | Advertises available MCP resources and tools |
| Context Injection | Provides knowledge context to AI tools on request |
| Trust Validation | Enforces governance rules before returning data |
| Status Reporting | Reports MCP client configuration health via `engram status` |
**Source:** [src/piia_engram/mcp_server.py:120-200](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py), [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
---
## Search Engine (`search.py`)
The search engine provides hybrid search combining keyword and vector-based semantic search.
### Search Modes
| Mode | Description | Use Case |
|------|-------------|----------|
| `keyword` | Full-text search with FTS | Exact phrase matching |
| `vector` | Semantic similarity search | Concept-based retrieval |
| `hybrid` | Combined RRF ranking | Best of both approaches |
**Source:** [src/piia_engram/search.py:1-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
### Hybrid Search Implementation
```mermaid
graph TD
A[Query] --> B[Keyword Search]
A --> C[Vector Embedding]
B --> D[BM25 Scores]
C --> E[KNN Scores]
D --> F[RRF Ranking]
E --> F
F --> G[Preserve Keyword Hits]
G --> H[Final Results]
style G fill:#90EE90
```
**Source:** [src/piia_engram/search.py:80-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
### Index Freshness Management
The search engine tracks index freshness using a fingerprint that includes:
- Content hash
- Embed model identifier
- Vector backend availability
When the embedding model changes, the system automatically triggers index rebuild to ensure dimension consistency.
**Source:** [src/piia_engram/search.py:150-220](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py), [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
---
## Session Management (`sessions.py`)
Sessions enable cross-tool agent continuity, allowing AI agents to maintain context across multiple interactions.
### Session Features
| Feature | Description | CLI Command |
|---------|-------------|-------------|
| Session Listing | List all saved sessions | `engram sessions` |
| Session Resume | Continue from a previous session | `engram sessions resume ` |
| Session Export | Export session history | `engram sessions export` |
**Source:** [src/piia_engram/sessions.py:1-60](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py), [v3.39.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.0)
### Session Data Flow
```mermaid
graph LR
A[AI Agent] -->|Tool Calls| B[Session Recorder]
B --> C[Session Store]
D[AI Agent] -->|Resume| E[Session Loader]
E --> C
E --> F[Knowledge Context]
C --> G[Session History]
```
**Source:** [src/piia_engram/sessions.py:60-120](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
---
## Hooks System (`hooks/__init__.py`)
The hooks system provides extensibility for custom processing at key lifecycle points.
### Available Hook Points
| Hook Point | Trigger | Use Case |
|------------|---------|----------|
| `pre_add` | Before knowledge is stored | Content validation, transformation |
| `post_add` | After knowledge is stored | Notifications, external sync |
| `pre_search` | Before search execution | Query modification, logging |
| `post_search` | After search returns | Result filtering, ranking adjustment |
**Source:** [src/piia_engram/hooks/__init__.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/__init__.py)
---
## CLI Interface (`cli.py`)
The command-line interface provides user-facing commands for all core operations.
### Primary Commands
| Command | Description |
|---------|-------------|
| `engram add` | Add knowledge content |
| `engram search` | Search knowledge base |
| `engram status` | Show system status and MCP health |
| `engram doctor` | Run diagnostics including terminal encoding |
| `engram sessions` | Manage agent sessions |
| `engram setup` | Configure MCP integration |
| `engram reindex` | Rebuild search indexes |
| `engram repair-encoding` | Fix mojibake in stored files |
| `engram review` | Review staged knowledge |
| `engram doctor` | Terminal encoding diagnostics |
**Source:** [src/piia_engram/cli.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
### Status Command Output
The `engram status` command provides:
- MCP client configuration health with redacted metadata
- MCP Clients table (with `--html` flag)
- Next recommended commands
**Source:** [src/piia_engram/cli.py:100-180](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py), [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
### Encoding Repair
The `engram repair-encoding` command:
- Dry-runs detection of high-confidence mojibake
- `--apply` flag repairs reversible cases with timestamped backup
- Reports lossy/suspicious cases for manual review
**Source:** [src/piia_engram/cli.py:180-250](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py), [v3.38.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.38.0)
---
## Configuration
Engram is configured via `engram.yaml` in the Engram root directory.
### Key Configuration Options
| Option | Default | Description |
|--------|---------|-------------|
| `engram_root` | `~/.engram` | Root directory for all data |
| `embed_model` | `auto` | Embedding model for vector search |
| `encryption.enabled` | `true` | Enable encryption at rest |
| `governance.trust_tier` | `medium` | Default trust tier for new tools |
| `search.mode` | `hybrid` | Default search mode |
**Source:** [src/piia_engram/core.py:200-280](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/core.py)
---
## Common Failure Modes and Troubleshooting
### Terminal Encoding Issues
If you see garbled characters (mojibake), run:
```bash
engram doctor
```
This distinguishes between:
- Terminal display issues (environment variable problems)
- Actual data corruption in stored files
**Source:** [src/piia_engram/cli.py:250-300](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py), [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### Index Freshness Issues
If semantic search returns empty results after changing the embedding model:
```bash
engram reindex
```
This ensures the vector index is rebuilt with the correct dimensions.
**Source:** [src/piia_engram/search.py:220-280](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py), [v3.33.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1)
### MCP Client Configuration
If MCP clients fail to connect:
1. Check configuration health: `engram status`
2. Re-run setup: `engram setup --mcp `
3. Verify JSON config files in tool-specific locations
**Source:** [src/piia_engram/cli.py:300-350](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py), [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
---
## See Also
- [Installation Guide](https://github.com/Patdolitse/piia-engram/wiki/Installation) — Setup instructions and dependencies
- [MCP Integration](https://github.com/Patdolitse/piia-engram/wiki/MCP-Integration) — Detailed MCP client configuration
- [Governance Model](https://github.com/Patdolitse/piia-engram/wiki/Governance-Model) — Access control and permission profiles
- [Search Reference](https://github.com/Patdolitse/piia-engram/wiki/Search-Reference) — Search modes and optimization
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) — Complete command documentation
- [Troubleshooting](https://github.com/Patdolitse/piia-engram/wiki/Troubleshooting) — Common issues and solutions
---
## MCP Tools Reference
### Related Pages
Related topics: [Knowledge Management](#knowledge-management), [Search and Retrieval](#search-retrieval), [Core Components](#components)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/mcp_server.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp_server.py)
- [src/piia_engram/mcp/__init__.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/mcp/__init__.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [src/piia_engram/__main__.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/__main__.py)
- [README.md](https://github.com/Patdolitse/piia-engram/blob/main/README.md)
- [pyproject.toml](https://github.com/Patdolitse/piia-engram/blob/main/pyproject.toml)
# MCP Tools Reference
This page documents the MCP (Model Context Protocol) tools exposed by piia-engram, enabling AI agents and GUI clients to interact with Engram's knowledge corpus, decision tracking, and session management capabilities.
## Overview
piia-engram exposes a comprehensive set of MCP tools that allow AI agents to persistently store, retrieve, and manage knowledge artifacts. The MCP server acts as the interface layer between AI clients and Engram's local-first knowledge infrastructure. Source: [src/piia_engram/mcp_server.py:1-50]()
```mermaid
graph TD
A["MCP Client
(Claude Desktop, Cursor, Trae, etc.)"] --> B["piia-engram-mcp
Server"]
B --> C["Knowledge Corpus
SQLite + FTS + Vector"]
B --> D["Decision Graph
(edges & vertices)"]
B --> E["Session Store
(JSONL)"]
B --> F["Governance Layer
(trust-tier enforcement)"]
G["CLI: engram doctor"] --> B
H["CLI: engram status"] --> B
I["CLI: engram sessions"] --> B
```
### Tool Categories
| Category | Purpose | Primary Tool Prefix |
|----------|---------|---------------------|
| Knowledge Management | Store and retrieve knowledge artifacts | `add_*`, `search_*` |
| Decision Tracking | Record and traverse decision histories | `add_decision`, `get_decisions` |
| Session Continuity | Track cross-tool agent sessions | `sessions_*` |
| Diagnostics | Health checks and encoding repair | `engram doctor`, `engram repair-encoding` |
## MCP Server Architecture
The MCP server is implemented as a Python package with a console entry point. Source: [src/piia_engram/mcp/__init__.py:1-30]()
### Entry Points
| Entry Point | Command | Use Case |
|-------------|---------|----------|
| `piia-engram-mcp` | Direct invocation | MCP clients requiring Python runtime |
| `uvx` | `uvx --from piia-engram piia-engram-mcp` | Zero-install MCP configuration |
| `engram` | CLI wrapper | Local diagnostics and administration |
Source: [pyproject.toml:1-50]() (console_scripts configuration)
### Zero-Install MCP Configuration
For GUI-based AI tools that support MCP, the recommended setup uses `uvx` for zero-installation:
```json
{
"mcpServers": {
"engram": {
"command": "uvx",
"args": ["--from", "piia-engram", "piia-engram-mcp"]
}
}
}
```
This configuration is automatically generated by `engram setup` for supported GUI clients. Source: [v3.37.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0)
### Supported GUI Clients
| Client | Config File | Auto-setup Support |
|--------|-------------|---------------------|
| Claude Desktop | `~/.claude/mcp.json` | Native |
| Cursor | `~/.cursor/mcp.json` | Native |
| Trae | `~/.trae/mcp.json` | Added in v3.37.0 |
| Tencent products | `~/.trae/mcp.json` | Added in v3.37.0 |
## Tool Reference
### Knowledge Tools
#### add_knowledge
Stores new knowledge artifacts in the Engram corpus.
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `content` | string | Yes | The knowledge content to store |
| `metadata` | object | No | Optional metadata (tags, source, etc.) |
| `embedding_model` | string | No | Override default embedding model |
**Returns**: Knowledge record ID and confirmation.
**Example Request**:
```json
{
"tool": "add_knowledge",
"parameters": {
"content": "The authentication service uses JWT tokens with 1-hour expiry",
"metadata": {
"source": "api-docs",
"component": "auth"
}
}
}
```
Source: [src/piia_engram/mcp_server.py:100-150]()
#### search_knowledge
Performs hybrid search combining keyword and semantic retrieval.
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `query` | string | Yes | Search query text |
| `mode` | string | No | `hybrid` (default), `keyword`, or `semantic` |
| `limit` | integer | No | Maximum results (default: 10) |
**Hybrid Search Behavior**:
- Keyword results are preserved and ranked using RRF (Reciprocal Rank Fusion)
- Semantic (vector) results supplement keyword matches
- Ensures hybrid recall ≥ keyword-only recall
Source: [v3.33.2 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.2)
### Decision Tools
#### add_decision
Records a decision in the decision graph with automatic threading.
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `question` | string | Yes | The question or context that led to the decision |
| `answer` | string | Yes | The decision/answer made |
| `supersedes` | string | No | Previous decision ID this supersedes |
**Auto-supersedes Behavior**: When the same question receives a new answer, `add_decision` automatically creates a `supersedes` edge to the previous decision. This enables tracing decision evolution over time. Source: [v3.35.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0)
#### get_decisions
Retrieves decision history, optionally filtered by question or time range.
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `question` | string | No | Filter by question text |
| `include_superseded` | boolean | No | Include superseded decisions (default: false) |
| `limit` | integer | No | Maximum results |
### Session Tools
#### sessions_list
Lists saved cross-tool agent sessions.
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `limit` | integer | No | Maximum sessions to return |
| `tool_filter` | string | No | Filter by specific tool |
Sessions are stored as JSONL files in the Engram root directory, enabling AI agents to resume previous contexts. Source: [v3.39.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.0)
## CLI Diagnostic Tools
While MCP tools target AI agents, the CLI provides human-accessible diagnostics:
### engram status
Reports MCP client configuration health with redacted metadata.
```bash
engram status
engram status --html
```
| Flag | Description |
|------|-------------|
| `--html` | Output HTML format with MCP Clients table |
The `--html` output includes:
- MCP Clients table with configuration status
- Next Commands suggestions: `engram doctor`, `engram review`, `engram sessions`
Source: [v3.40.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0)
### engram doctor
Performs terminal encoding diagnostics and health checks.
```bash
engram doctor
```
**Diagnostics Performed**:
- UTF-8 encoding verification (Windows `cp65001` recognized)
- Mojibake detection distinguishes display issues from data corruption
- Warning suppression when stdout/stderr are already UTF-8
Source: [v3.39.1 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1)
### engram repair-encoding
Repairs detected mojibake in knowledge files.
```bash
engram repair-encoding # Dry-run mode
engram repair-encoding --apply # Apply repairs with backup
```
**Repair Guardrails**:
- Reversible cases are repaired with timestamped backups
- Lossy/suspicious cases are reported for manual review
- Scans JSON, JSONL, Markdown, and text files
Source: [v3.38.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.38.0)
## Configuration
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `ENGRAM_ROOT` | `~/.engram` | Engram data directory |
| `ENGRAM_EMBED_MODEL` | (built-in) | Embedding model for semantic search |
| `ENGRAM_DB_PATH` | `$ENGRAM_ROOT/engram.db` | SQLite database path |
Source: [src/piia_engram/cli.py:50-100]()
### Trust Tier Configuration
The governance layer enforces permission profiles based on trust tiers:
```mermaid
graph LR
A["Local CLI"] --> B["R20: Full Access"]
C["MCP Client"] --> D["R15: Controlled Access"]
E["External Export"] --> F["R10: Read-Only"]
B --> G["Corpus Encryption"]
D --> G
F --> G
```
Source: [v3.34.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.34.0)
## Security Model
### Corpus Encryption
Knowledge content is encrypted at rest. Each AI tool sees its own permission boundary inline. Source: [v3.36.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0)
### Governance Closure
The governance layer is sealed against:
- Write bypass attempts
- Read-path side effects
All governed/export tools enforce trust-tier checks on every read path. Source: [v3.34.0 Release Notes](https://github.com/Patdolitse/piia-engram/releases/tag/v3.34.0)
## Common Failure Modes
| Issue | Symptom | Resolution |
|-------|---------|------------|
| Wrong embedding model | Semantic search returns empty | Ensure `ENGRAM_EMBED_MODEL` matches; run `engram reindex` |
| Index not rebuilt after model change | Vector results missing | Run `engram reindex`; v3.33.1 fixed this automatically |
| Terminal mojibake | Garbled characters displayed | Run `engram doctor` to distinguish display vs data issues |
| MCP connection refused | AI client cannot reach server | Check `engram status` for MCP client health |
### Encoding Repair Workflow
```mermaid
graph TD
A["engram repair-encoding"] --> B{Dry-run or Apply?}
B -->|Dry-run| C["List suspicious files"]
B -->|Apply| D["Create timestamped backup"]
D --> E{Repairable?}
E -->|Yes| F["Apply repair"]
E -->|No| G["Report for manual review"]
C --> H["Review list"]
F --> I["Verify restored content"]
```
## Data Flow
```mermaid
sequenceDiagram
participant Client as MCP Client
participant Server as piia-engram-mcp
participant Storage as SQLite/JSONL
participant Vector as Embedding Engine
Client->>Server: add_knowledge(content)
Server->>Vector: generate_embedding(content)
Server->>Storage: Store with embedding
Server-->>Client: record_id
Client->>Server: search_knowledge(query, mode=hybrid)
Server->>Vector: generate_embedding(query)
Server->>Storage: FTS + KNN query
Storage-->>Server: ranked_results
Server-->>Client: results
```
## See Also
- [Home](README) — Project overview and quick start
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) — Full CLI command documentation
- [Architecture Overview](https://github.com/Patdolitse/piia-engram/wiki/Architecture) — System architecture details
- [Governance & Security](https://github.com/Patdolitse/piia-engram/wiki/Governance) — Trust tiers and encryption
- [Troubleshooting](https://github.com/Patdolitse/piia-engram/wiki/Troubleshooting) — Common issues and solutions
---
## Knowledge Management
### Related Pages
Related topics: [MCP Tools Reference](#mcp-tools), [Search and Retrieval](#search-retrieval)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/decision_thread.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/decision_thread.py)
- [src/piia_engram/reconcile.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/reconcile.py)
- [src/piia_engram/hooks/auto_absorb_compact.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_absorb_compact.py)
- [src/piia_engram/hooks/auto_inject_resume_brief.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_inject_resume_brief.py)
- [src/piia_engram/hooks/auto_save_on_stop.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_save_on_stop.py)
- [src/piia_engram/status.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/status.py)
- [src/piia_engram/sessions.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
- [src/piia_engram/doctor.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/doctor.py)
- [src/piia_engram/search.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
- [src/piia_engram/corpus.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/corpus.py)
# Knowledge Management
## Overview
Knowledge Management in piia-engram is the core system for capturing, storing, organizing, and retrieving information that AI agents and users need to maintain context across sessions. As a local-first, privacy-focused knowledge management system, Engram provides a structured corpus where decisions, session histories, and absorbed knowledge are stored with encryption at rest and fine-grained access controls.
The knowledge management system is designed to support cross-tool agent workflows, enabling AI assistants to remember previous interactions, decisions made, and context gathered over time without relying on external cloud services. Source: [src/piia_engram/corpus.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/corpus.py)
## Architecture Overview
The knowledge management architecture consists of several interconnected components that work together to provide a comprehensive memory system for AI agents.
```mermaid
graph TD
A[User / AI Agent] -->|add_decision| B[Decision Thread Manager]
A -->|create_session| C[Session Manager]
A -->|absorb_knowledge| D[Auto Absorb Hook]
B -->|stores| E[(Decision Corpus)]
C -->|stores| F[(Session Storage)]
D -->|compacts| G[(Knowledge Corpus)]
H[Search Engine] -->|queries| E
H -->|queries| G
H -->|hybrid search| F
I[Governance Layer] -->|enforces permissions| E
I -->|enforces permissions| G
I -->|enforces permissions| F
J[Corpus Encryption] -->|encrypts at rest| E
J -->|encrypts at rest| G
J -->|encrypts at rest| F
```
### Core Components
| Component | Purpose | Storage Location |
|-----------|---------|------------------|
| Decision Thread Manager | Tracks decision history and supersession chains | `~/.engram/decisions/` |
| Session Manager | Maintains cross-tool agent session continuity | `~/.engram/sessions/` |
| Knowledge Corpus | Stores absorbed knowledge with semantic embeddings | `~/.engram/corpus/` |
| Governance Layer | Enforces trust-tier permissions on read/write paths | All storage locations |
| Search Engine | Provides hybrid FTS + vector search across knowledge | `~/.engram/search_index/` |
Source: [src/piia_engram/decision_thread.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/decision_thread.py), [src/piia_engram/sessions.py:1-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
## Decision Threads
Decision threads are the primary mechanism for tracking how decisions evolved over time within the knowledge management system. Each decision records not only the decision itself but also the context, rationale, and relationships to previous decisions.
### Decision Thread Structure
```mermaid
classDiagram
class Decision {
+str id
+str question
+str answer
+str rationale
+datetime timestamp
+list~str~ supersedes
+str thread_id
+dict metadata
}
class DecisionThread {
+str thread_id
+str topic
+list~Decision~ decisions
+datetime created_at
+datetime updated_at
}
DecisionThread "1" --> "*" Decision : contains
Decision "1" --> "*" Decision : supersedes
```
### Auto-Supersession Behavior
When a new decision is added that addresses the same question as an existing decision, the system automatically creates a `supersedes` edge. This ensures that decision history remains traceable while making the most recent decision the authoritative one. Source: [src/piia_engram/decision_thread.py:45-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/decision_thread.py)
### Decision Attributes
| Attribute | Type | Description |
|-----------|------|-------------|
| `id` | string | Unique identifier for the decision |
| `question` | string | The question or problem being addressed |
| `answer` | string | The decision made or answer provided |
| `rationale` | string | Reasoning behind the decision |
| `thread_id` | string | Groups related decisions into a thread |
| `supersedes` | list | IDs of decisions this decision supersedes |
| `metadata` | dict | Additional context and tags |
## Session Management
Sessions represent cross-tool agent workflows that span multiple interactions. The session management system provides continuity by storing session state and enabling resumption of interrupted workflows.
### Session Lifecycle
```mermaid
stateDiagram-v2
[*] --> Active: create_session
Active --> Staged: auto_save_on_stop hook
Active --> Suspended: timeout or manual suspend
Suspended --> Active: resume_session
Staged --> Archived: review_complete
Staged --> Active: resume_session
Archived --> [*]: session_prune
```
### Session Storage
Sessions are stored with the following structure:
| Field | Description |
|-------|-------------|
| `session_id` | Unique session identifier |
| `created_at` | Session creation timestamp |
| `tools_used` | List of MCP tools invoked during session |
| `decisions_made` | Decisions recorded in this session |
| `knowledge_absorbed` | Knowledge entries added |
| `state` | Current session state (active/staged/archived) |
Source: [src/piia_engram/sessions.py:20-60](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
## Knowledge Lifecycle Hooks
Engram provides automated hooks that manage the knowledge lifecycle without requiring manual intervention.
### Auto-Absorb and Compact
The auto-absorb hook captures knowledge from agent interactions and automatically compacts redundant entries to maintain an efficient corpus. Source: [src/piia_engram/hooks/auto_absorb_compact.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_absorb_compact.py)
### Auto-Inject Resume Brief
When resuming a session, this hook automatically injects relevant context from the knowledge corpus to help the agent regain context quickly. Source: [src/piia_engram/hooks/auto_inject_resume_brief.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_inject_resume_brief.py)
### Auto-Save on Stop
This hook ensures session state is persisted when an agent session ends, preventing data loss from unexpected interruptions. Source: [src/piia_engram/hooks/auto_save_on_stop.py:1-60](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/hooks/auto_save_on_stop.py)
### Hook Execution Flow
```mermaid
sequenceDiagram
participant Agent
participant HookManager
participant KnowledgeCorpus
participant SessionStorage
Agent->>HookManager: Session ends
HookManager->>HookManager: Trigger auto_save_on_stop
HookManager->>SessionStorage: Persist session state
HookManager->>HookManager: Trigger auto_absorb_compact
HookManager->>KnowledgeCorpus: Compact redundant entries
HookManager->>HookManager: Update freshness fingerprint
KnowledgeCorpus-->>HookManager: Index updated
```
## Knowledge Retrieval
### Hybrid Search
Engram supports hybrid search combining full-text search (FTS) with semantic vector search to maximize recall. The system uses Reciprocal Rank Fusion (RRF) to merge result sets while ensuring keyword-matched results are never dropped. Source: [src/piia_engram/search.py:1-120](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search.py)
### Search Configuration
| Parameter | Default | Description |
|-----------|---------|-------------|
| `ENGRAM_EMBED_MODEL` | auto-detected | Embedding model for semantic search |
| `ENGRAM_SEARCH_MODE` | hybrid | Search mode: fts, vector, or hybrid |
| `ENGRAM_RRF_K` | 60 | RRF parameter for result fusion |
| `ENGRAM_MAX_RESULTS` | 20 | Maximum results to return |
### Index Freshness
The system maintains a freshness fingerprint that tracks:
- Content modifications
- Embedding model changes
- Vector backend availability
When the embedding model changes, the index is automatically rebuilt to ensure vector dimensions match the new model. Source: [src/piia_engram/reconcile.py:30-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/reconcile.py)
## Security and Governance
### Corpus Encryption
All knowledge content is encrypted at rest. The encryption system ensures that each AI tool sees only its own permission boundary inline, maintaining data isolation between different agents and users. Source: [src/piia_engram/corpus.py:100-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/corpus.py)
### Trust-Tier Enforcement
The governance layer enforces trust-tier checks on every read and write path, closing file-side-effect leak classes and preventing unauthorized access to knowledge entries.
```mermaid
graph LR
A[Read Request] --> B{Trust Tier Check}
B -->|Passed| C[Return Knowledge Entry]
B -->|Failed| D[Access Denied]
E[Write Request] --> F{Governance Check}
F -->|Passed| G[Persist to Corpus]
F -->|Failed| H[Write Blocked]
```
### Permission Profile
Users can control who accesses their Engram data through permission profiles that define trust tiers for different agents and tools.
## CLI Commands for Knowledge Management
### Status Command
The `engram status` command provides a comprehensive overview of the knowledge management system:
```bash
engram status # Show basic status
engram status --html # Generate HTML report with MCP clients table
```
Source: [src/piia_engram/status.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/status.py)
### Doctor Command
The `engram doctor` command runs diagnostics including terminal encoding checks:
```bash
engram doctor # Run full diagnostic suite
```
Source: [src/piia_engram/doctor.py:1-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/doctor.py)
### Sessions Command
The `engram sessions` command lists saved cross-tool agent sessions:
```bash
engram sessions # List all sessions
engram sessions --active # Show only active sessions
```
Source: [src/piia_engram/sessions.py:80-120](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sessions.py)
### Repair Encoding Command
For repairing mojibake in knowledge files:
```bash
engram repair-encoding # Dry-run for encoding issues
engram repair-encoding --apply # Apply repairs with backup
```
## Common Failure Modes
| Issue | Symptoms | Resolution |
|-------|----------|------------|
| Encoding corruption | Mojibake in JSON/Markdown files | Run `engram repair-encoding --apply` |
| Index staleness | Missing semantic search results | Run `engram reindex` |
| Vector dimension mismatch | Empty KNN results after model change | Index rebuilds automatically |
| Trust-tier violations | Access denied errors | Check permission profile configuration |
| Session corruption | Cannot resume session | Use `engram sessions` to find staged sessions |
## Data Storage Structure
```
~/.engram/
├── corpus/ # Knowledge entries with embeddings
├── decisions/ # Decision threads and history
├── sessions/ # Cross-tool agent sessions
├── search_index/ # FTS and vector search indexes
├── governance/ # Permission profiles and policies
└── config.yaml # Local configuration
```
## See Also
- [MCP Setup Guide](https://github.com/Patdolitse/piia-engram/wiki/MCP-Setup) - Configuring Engram as an MCP server
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) - Complete CLI command documentation
- [Security Architecture](https://github.com/Patdolitse/piia-engram/wiki/Security-Architecture) - Encryption and governance details
- [Release Notes](https://github.com/Patdolitse/piia-engram/releases) - Latest features and fixes
---
## Search and Retrieval
### Related Pages
Related topics: [MCP Tools Reference](#mcp-tools), [Knowledge Management](#knowledge-management)
```details
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/search_index.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
- [src/piia_engram/retrieval.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/retrieval.py)
- [src/piia_engram/embeddings.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/embeddings.py)
- [src/piia_engram/config.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/config.py)
- [src/piia_engram/corpus.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/corpus.py)
- [src/piia_engram/database.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/database.py)
- [src/piia_engram/cli.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
- [docs/hybrid-search.md](https://github.com/Patdolitse/piia-engram/blob/main/docs/hybrid-search.md)
- [docs/embeddings.md](https://github.com/Patdolitse/piia-engram/blob/main/docs/embeddings.md)
```
# Search and Retrieval
This page documents the search and retrieval system in Engram, covering architecture, configuration, usage patterns, and implementation details for knowledge content discovery.
## Overview
Engram provides a multi-modal search and retrieval system that enables users and AI agents to locate relevant knowledge content across the local corpus. The system supports three primary search strategies:
1. **Keyword Search (FTS)** — Exact term matching using SQLite Full-Text Search (FTS5)
2. **Semantic Search** — Vector-based similarity search using embedding models
3. **Hybrid Search** — Combines keyword and semantic results using Reciprocal Rank Fusion (RRF)
The search system is designed for local-first operation, processing all data on-device without external API dependencies. Source: [src/piia_engram/search_index.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
## Architecture
```mermaid
graph TD
A[User Query] --> B[Search Entry Point]
B --> C{Search Mode}
C -->|keyword| D[FTS5 Query Engine]
C -->|semantic| E[Embedding Model]
C -->|hybrid| F[Hybrid Orchestrator]
E --> G[Vector Similarity Search]
D --> H[Ranked Results]
F --> I[RRF Fusion]
G --> I
I --> J[Post-Processing]
H --> J
J --> K[Filtered Results]
K --> L[Trust-Tier Gating]
L --> M[Final Results]
N[(Corpus Database)] -.-> D
O[(Vector Index)] -.-> G
P[Embedding Cache] -.-> E
```
### Core Components
| Component | File | Purpose |
|-----------|------|---------|
| `SearchIndex` | `search_index.py` | Main search orchestrator managing FTS, semantic, and hybrid modes |
| `RetrievalEngine` | `retrieval.py` | Query execution and result ranking |
| `EmbeddingService` | `embeddings.py` | Text-to-vector conversion for semantic search |
| `CorpusManager` | `corpus.py` | Knowledge content storage and metadata |
| `DatabaseManager` | `database.py` | SQLite connection and schema management |
Source: [src/piia_engram/search_index.py:50-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
## Search Modes
### Keyword Search (FTS5)
Full-Text Search provides exact term matching using SQLite's FTS5 extension. This mode is optimal for:
- Known exact terms or phrases
- Code identifiers and technical keywords
- Users who prefer deterministic results
The FTS index supports prefix matching (`term*`), phrase matching (`"exact phrase"`), and boolean operators (AND, OR, NOT). Source: [src/piia_engram/database.py:200-280](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/database.py)
**Configuration Options:**
| Option | Environment Variable | Default | Description |
|--------|---------------------|---------|-------------|
| FTS Index | Auto-created | — | Built on `chunks.content` table |
| Stemming | `ENGRAM_FTS_STEMMING` | `true` | Enable Porter stemmer |
| Tokenizer | `ENGRAM_FTS_TOKENIZER` | `unicode61` | FTS5 tokenizer selection |
### Semantic Search
Semantic search converts queries and content into dense vector embeddings, enabling similarity-based retrieval. This mode excels at:
- Conceptual and semantic matches
- Paraphrased queries
- Finding related content without exact keyword overlap
Source: [src/piia_engram/embeddings.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/embeddings.py)
**Embedding Configuration:**
| Parameter | Environment Variable | Default | Description |
|-----------|---------------------|---------|-------------|
| Model | `ENGRAM_EMBED_MODEL` | `sentence-transformers/all-MiniLM-L6-v2` | HuggingFace model identifier |
| Dimension | Computed | Auto | Vector dimensionality |
| Device | `ENGRAM_EMBED_DEVICE` | `cpu` | Compute device (cpu/cuda/mps) |
| Batch Size | `ENGRAM_EMBED_BATCH_SIZE` | `32` | Processing batch size |
```mermaid
graph LR
A[Query Text] --> B[Embedding Service]
B --> C[Query Vector]
C --> D[KNN Search]
D --> E[Content Vectors]
E --> F[Top-K Results]
F --> G[Score Threshold]
G --> H[Semantic Matches]
```
**Important Index Freshness Behavior:**
When the embedding model is changed via `ENGRAM_EMBED_MODEL`, the vector index is automatically flagged for rebuild. The freshness fingerprint includes the model identifier, ensuring that changing models triggers index reconstruction. Source: [src/piia_engram/search_index.py:300-350](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
### Hybrid Search
Hybrid search combines keyword and semantic results using Reciprocal Rank Fusion (RRF), providing both precision and recall. The algorithm:
1. Executes FTS query and semantic query in parallel
2. Normalizes scores from each result set
3. Applies RRF formula: `score = Σ(1 / (k + rank))` where k=60
4. Returns merged, re-ranked results
**Hybrid Recall Guarantee:**
Engram's hybrid search ensures that keyword matches are never lost during fusion. The implementation preserves all FTS results before applying RRF, addressing a known issue where RRF truncation could displace keyword hits. Source: [docs/hybrid-search.md:1-50](https://github.com/Patdolitse/piia-engram/blob/main/docs/hybrid-search.md)
```python
# Hybrid search result fusion (simplified)
def rrf_fusion(fst_results, semantic_results, k=60):
fused_scores = {}
# Preserve all keyword results
for rank, (chunk_id, fts_score) in enumerate(fst_results):
fused_scores[chunk_id] = fts_score
# Apply RRF for semantic results
for rank, (chunk_id, vec_score) in enumerate(semantic_results):
rrf_score = 1 / (k + rank)
if chunk_id in fused_scores:
fused_scores[chunk_id] += vec_score + rrf_score
else:
fused_scores[chunk_id] = rrf_score
return sorted(fused_scores.items(), key=lambda x: -x[1])
```
## Search Configuration
### Environment Variables
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `ENGRAM_SEARCH_MODE` | `keyword\|semantic\|hybrid` | `hybrid` | Default search mode |
| `ENGRAM_EMBED_MODEL` | string | `all-MiniLM-L6-v2` | Sentence-transformer model |
| `ENGRAM_EMBED_DEVICE` | `cpu\|cuda\|mps` | `cpu` | Compute device |
| `ENGRAM_SEARCH_LIMIT` | int | `20` | Default result limit |
| `ENGRAM_SEARCH_THRESHOLD` | float | `0.0` | Minimum relevance score |
| `ENGRAM_FTS_TOKENIZER` | string | `unicode61` | FTS5 tokenizer |
| `ENGRAM_VECTOR_BACKEND` | `numpy\|torch\|auto` | `auto` | Vector compute backend |
Source: [src/piia_engram/config.py:1-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/config.py)
### Vector Backend Selection
The system auto-selects the optimal vector backend based on available dependencies:
| Priority | Backend | Requirement |
|----------|---------|-------------|
| 1 | PyTorch | `torch` package installed |
| 2 | NumPy | Default fallback |
| 3 | Auto | Dependency detection |
Source: [src/piia_engram/embeddings.py:100-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/embeddings.py)
## Usage Patterns
### CLI Search Commands
```bash
# Basic keyword search
engram search "exact phrase"
# Semantic search
engram search --mode semantic "conceptual query"
# Hybrid search (default)
engram search --mode hybrid "combined query"
# Limit results
engram search --limit 10 "query"
# Filter by metadata
engram search --source "github" --tag "bug" "query"
```
Source: [src/piia_engram/cli.py:200-300](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
### Python API
```python
from piia_engram import SearchIndex, RetrievalEngine
# Initialize search index
index = SearchIndex(corpus_path="./engram_data")
# Execute hybrid search
results = index.search(
query="how to configure MCP",
mode="hybrid",
limit=10,
trust_tier="user"
)
for result in results:
print(f"{result.chunk_id}: {result.score:.3f}")
print(f" Content: {result.content[:100]}...")
```
Source: [src/piia_engram/search_index.py:400-450](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
### MCP Tool Integration
The MCP server exposes search functionality for AI agents:
```json
{
"name": "engram_search",
"description": "Search Engram knowledge base",
"inputSchema": {
"type": "object",
"properties": {
"query": {"type": "string"},
"mode": {"enum": ["keyword", "semantic", "hybrid"]},
"limit": {"type": "integer", "default": 10}
}
}
}
```
Source: [src/piia_engram/cli.py:50-100](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
## Index Management
### Index States
```mermaid
graph TD
A[New Corpus] --> B[Initial Index Build]
B --> C[Index Ready]
C --> D{Content Change}
C --> E{Model Change}
D --> F[Incremental Update]
E --> G[Full Rebuild]
F --> C
G --> C
```
### Rebuild Commands
```bash
# Full index rebuild
engram reindex
# FTS-only rebuild
engram reindex --type fts
# Vector-only rebuild
engram reindex --type vector
# Force rebuild regardless of freshness
engram reindex --force
```
**Vector Backend Availability Detection:**
When vector dependencies are installed after FTS-only indexing, the system detects missing vector tables and automatically triggers a rebuild to include semantic search capabilities. Source: [src/piia_engram/search_index.py:250-300](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/search_index.py)
### Freshness Fingerprint
The index freshness is determined by a fingerprint that includes:
| Factor | Included | Notes |
|--------|----------|-------|
| Content hash | Yes | Detects content changes |
| Embedding model | Yes | Triggers rebuild on model change |
| Vector backend | Yes | Detects backend changes |
| FTS tokenizer | Yes | Detects tokenizer config changes |
| Schema version | Yes | Ensures compatibility |
Source: [src/piia_engram/corpus.py:150-200](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/corpus.py)
## Trust-Tier Gating
All search results pass through trust-tier enforcement, ensuring that AI tools access only permitted content based on their permission profile. Source: [src/piia_engram/retrieval.py:100-150](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/retrieval.py)
```mermaid
graph TD
A[Search Results] --> B{Trust-Tier Check}
B -->|chunk.trust_tier <= tool.trust_tier| C[Include in Results]
B -->|chunk.trust_tier > tool.trust_tier| D[Exclude]
C --> E[Post-Processing]
D --> F[Audit Log Entry]
```
| Trust Tier | Level | Description |
|------------|-------|-------------|
| `public` | 0 | Visible to all tools and users |
| `user` | 10 | Visible to user-trusted tools |
| `internal` | 20 | Internal tools only |
| `restricted` | 30 | Explicitly permitted tools only |
## Common Issues and Solutions
### Symptom: Empty Semantic Results After Model Change
**Cause:** The vector index was built with a different embedding model, causing dimension mismatch.
**Solution:** The system automatically detects model changes via freshness fingerprint. If results are empty after a model change, run:
```bash
engram reindex --type vector --force
```
This ensures the vector table is rebuilt with the new model's dimensions. Source: [docs/hybrid-search.md:80-100](https://github.com/Patdolitse/piia-engram/blob/main/docs/hybrid-search.md)
### Symptom: Hybrid Search Missing Expected Keyword Matches
**Cause:** RRF fusion with strict result limits could displace keyword matches.
**Solution:** Engram v3.33.2+ preserves all keyword results before applying RRF fusion. If using an older version, upgrade or run:
```bash
engram reindex
```
### Symptom: Slow Search on Large Corpus
**Diagnostic:** Check vector backend selection:
```bash
engram doctor
```
**Solutions:**
1. Enable GPU acceleration: `ENGRAM_EMBED_DEVICE=cuda`
2. Increase batch size: `ENGRAM_EMBED_BATCH_SIZE=64`
3. Limit search scope with filters: `--source`, `--tag`, `--date-range`
### Symptom: Encoding Issues in Search Results
The `engram doctor` command includes terminal encoding diagnostics to distinguish between display mojibake and actual data corruption. Source: [src/piia_engram/cli.py:300-350](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/cli.py)
```bash
# Run encoding diagnostics
engram doctor --encoding
```
## Performance Considerations
| Operation | Typical Latency |影响因素 |
|-----------|-----------------|---------|
| FTS query | 10-50ms | Index size, query complexity |
| Semantic query | 50-500ms | Embedding time, vector size |
| Hybrid query | 100-600ms | Both above, plus fusion |
| Index rebuild | 1-10 min | Corpus size, model |
**Optimization Tips:**
1. **Batch operations** — When adding multiple chunks, use batch insert for better throughput
2. **Cache embeddings** — Repeated queries benefit from embedding cache
3. **Filter early** — Apply metadata filters before search to reduce result set
4. **Warm-up queries** — First semantic query may be slower due to model loading
## Related Documentation
- [Hybrid Search Implementation](docs/hybrid-search.md) — Detailed RRF algorithm documentation
- [Embedding Configuration](docs/embeddings.md) — Model selection and optimization
- [Trust and Governance](docs/governance.md) — Access control and permission profiles
- [CLI Reference](docs/cli.md) — Complete command documentation
## See Also
- [Home](Home) — Project overview
- [Architecture](Architecture) — System design overview
- [MCP Integration](MCP-Integration) — MCP server setup and tools
- [Configuration Reference](Configuration) — Full environment variable documentation
---
## Security and Encryption
### Related Pages
Related topics: [System Architecture](#architecture)
Related Source Files
The following source files were used to generate this page:
- [src/piia_engram/crypto.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py)
- [src/piia_engram/sensitivity.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sensitivity.py)
- [src/piia_engram/governance.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
- [src/piia_engram/permission.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/permission.py)
- [SECURITY.md](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md)
- [src/piia_engram/trust.py](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/trust.py)
# Security and Encryption
## Overview
Engram implements a comprehensive identity-layer security system designed to protect knowledge content at rest while maintaining clear permission boundaries between AI tools and users. The security architecture is built around three core principles: **encryption at rest**, **trust-tier enforcement**, and **governance isolation**.
The system ensures that every AI tool sees only its own permission boundary inline, and the governance layer is sealed against both write bypass and read-path side effects. This approach aligns with Engram's local-first privacy model, where data remains under user control without dependence on third-party cloud services.
Source: [SECURITY.md:1-20](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md)
## Architecture
### High-Level Security Model
The security architecture consists of layered components that work together to provide defense-in-depth protection for knowledge content:
```mermaid
graph TD
A[User Data] --> B[Encryption Layer
crypto.py]
B --> C[Permission Layer
permission.py]
C --> D[Governance Layer
governance.py]
D --> E[Trust Tier Engine
trust.py]
F[AI Tool Request] --> C
G[Export Request] --> D
H[Read Request] --> E
style B fill:#e1f5fe
style C fill:#e8f5e8
style D fill:#fff3e0
style E fill:#fce4ec
```
### Encryption at Rest
Knowledge content stored in the Engram corpus is encrypted at rest using cryptographic primitives defined in the crypto module. This ensures that even if storage media are compromised, the content remains protected.
Source: [src/piia_engram/crypto.py:1-50](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/crypto.py)
### Trust-Tier System
The trust-tier system enforces access controls on every read path. All governed and export tools check trust-tier levels before granting access to knowledge content, closing the file-side-effect leak class that was identified in earlier versions.
Source: [src/piia_engram/trust.py:1-30](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/trust.py)
## Core Components
| Component | File | Purpose |
|-----------|------|---------|
| Crypto Module | `src/piia_engram/crypto.py` | Handles encryption/decryption of corpus content at rest |
| Sensitivity Module | `src/piia_engram/sensitivity.py` | Manages sensitivity labels and content classification |
| Permission Module | `src/piia_engram/permission.py` | Enforces permission boundaries for AI tools |
| Governance Module | `src/piia_engram/governance.py` | Seals the governance layer against write bypass and read-path side effects |
| Trust Module | `src/piia_engram/trust.py` | Implements trust-tier checks on read paths |
Source: [src/piia_engram/governance.py:1-40](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
## Permission Profile
Engram v3.35.0 introduced permission profiles that give users control over who accesses their Engram data. Each AI tool receives its own permission boundary, preventing cross-tool data leakage.
### Permission Boundary Enforcement
Every AI tool sees only its own permission boundary inline. This isolation ensures that:
1. Tools cannot read content outside their authorized scope
2. Write operations are restricted to permitted content areas
3. Access attempts are logged for audit purposes
Source: [src/piia_engram/permission.py:1-35](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/permission.py)
## Governance Layer
The governance layer acts as the final enforcement point for security policies. It was hardened in v3.34.0 and v3.36.0 to seal against both write bypass and read-path side effects.
```mermaid
graph LR
A[Request] --> B{Governance Check}
B -->|Write| C[Write Path Guard]
B -->|Read| D[Trust Tier Check]
C -->|Bypass Attempt| E[Blocked]
D -->|Insufficient Trust| E
D -->|Pass| F[Content Access]
```
### Governance Sealing
The governance layer implements the following protections:
- **Write Bypass Prevention**: All write operations must pass through governance validation
- **Read-Path Side Effect Prevention**: Read operations cannot trigger unintended state changes
- **Trust-Tier Enforcement**: Every read path enforces trust-tier checks
Source: [src/piia_engram/governance.py:40-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
## Sensitivity Classification
Content sensitivity is managed through the sensitivity module, which classifies knowledge content and applies appropriate protection levels.
### Sensitivity Levels
| Level | Description | Protection Applied |
|-------|-------------|-------------------|
| Public | Non-sensitive information | Standard encryption |
| Internal | Internal use only | Encryption + access logging |
| Confidential | Sensitive business data | Encryption + strict ACL |
| Restricted | Highly sensitive content | Full encryption + audit trail |
Source: [src/piia_engram/sensitivity.py:1-45](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/sensitivity.py)
## Configuration Options
### Environment Variables for Security
| Variable | Purpose | Default |
|----------|---------|---------|
| `ENGRAM_ENCRYPTION_KEY` | Master encryption key for corpus | Required |
| `ENGRAM_TRUST_TIER_DEFAULT` | Default trust tier for new tools | `internal` |
| `ENGRAM_GOVERNANCE_STRICT` | Enable strict governance mode | `true` |
| `ENGRAM_AUDIT_LOG_PATH` | Path for audit log output | `./engram-audit.log` |
Source: [src/piia_engram/governance.py:15-25](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/governance.py)
## Usage Patterns
### Initializing Secure Storage
```python
from piia_engram import crypto, governance
# Initialize encryption with provided key
crypto.initialize(key_path="/path/to/key")
# Verify governance layer is sealed
governance.verify_integrity()
```
### Trust-Tier Checking
```python
from piia_engram import trust, permission
# Check if a tool has sufficient trust tier for read access
tool_trust = trust.get_tool_trust_level("mcp-client-name")
required_tier = "internal"
if trust.check_access(tool_trust, required_tier):
# Grant read access
pass
else:
# Deny access and log attempt
trust.log_denial("mcp-client-name", "read", required_tier)
```
Source: [src/piia_engram/trust.py:50-80](https://github.com/Patdolitse/piia-engram/blob/main/src/piia_engram/trust.py)
## Common Failure Modes
### Encryption Key Issues
- **Missing Key**: Engram will refuse to decrypt corpus without valid key
- **Corrupted Key**: Key validation fails; re-initialization may be required
- **Key Rotation**: Unrotated keys may cause access issues after policy updates
### Trust-Tier Violations
- **Insufficient Trust**: Tools with low trust tiers cannot access protected content
- **Trust Tier Drift**: Tools may accumulate permissions over time (use `engram status` to audit)
### Governance Blockages
- **Write Bypass Attempt**: Blocked and logged; prevents unauthorized content modification
- **Read-Path Side Effects**: Detected and prevented by governance layer
Source: [SECURITY.md:30-60](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md)
## Security Best Practices
1. **Key Management**: Store encryption keys securely using a secrets manager
2. **Trust Tier Auditing**: Regularly audit tool trust tiers using `engram status`
3. **Governance Verification**: Periodically run `engram doctor` to verify governance integrity
4. **Audit Log Review**: Review audit logs for suspicious access patterns
5. **Principle of Least Privilege**: Assign minimum necessary trust tiers to tools
Source: [SECURITY.md:60-90](https://github.com/Patdolitse/piia-engram/blob/main/SECURITY.md)
## See Also
- [CLI Reference](https://github.com/Patdolitse/piia-engram/wiki/CLI-Reference) - Security-related commands
- [Installation Guide](https://github.com/Patdolitse/piia-engram/wiki/Installation) - Secure installation procedures
- [Architecture Overview](https://github.com/Patdolitse/piia-engram/wiki/Architecture) - System architecture details
- [Governance Configuration](https://github.com/Patdolitse/piia-engram/wiki/Governance-Configuration) - Governance layer setup
- [Release Notes v3.36.0](https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0) - Identity-layer security release
- [Release Notes v3.34.0](https://github.com/Patdolitse/piia-engram/releases/tag/v3.34.0) - Governance cutover details
---
---
## Pitfall Log
Project: Patdolitse/piia-engram
Summary: Found 18 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.
## 1. Installation risk - Installation risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this installation risk before relying on the project: v3.34.0 — Governance cutover + Universal harness + Playbook policy
- User impact: Upgrade or migration may change expected behavior: v3.34.0 — Governance cutover + Universal harness + Playbook policy
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.34.0 — Governance cutover + Universal harness + Playbook policy. Context: Observed when using python
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_76925401e068c1b5f3b688fe03df685f | https://github.com/Patdolitse/piia-engram/releases/tag/v3.34.0
## 2. Installation risk - Installation risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this installation risk before relying on the project: v3.35.0 — Decision Threads & Permission Profile
- User impact: Upgrade or migration may change expected behavior: v3.35.0 — Decision Threads & Permission Profile
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.35.0 — Decision Threads & Permission Profile. Context: Observed when using python
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_77e4f97aad82e83fe9ef3312294eec6b | https://github.com/Patdolitse/piia-engram/releases/tag/v3.35.0
## 3. Installation risk - Installation risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this installation risk before relying on the project: v3.37.0 - GUI entry adoption: piia-engram-mcp + uvx
- User impact: Upgrade or migration may change expected behavior: v3.37.0 - GUI entry adoption: piia-engram-mcp + uvx
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.37.0 - GUI entry adoption: piia-engram-mcp + uvx. Context: Observed during installation or first-run setup.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_75eede3829cc5d9e70aae6672a96605c | https://github.com/Patdolitse/piia-engram/releases/tag/v3.37.0
## 4. Installation risk - Installation risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this installation risk before relying on the project: v3.39.0 - Local Workflow Visibility
- User impact: Upgrade or migration may change expected behavior: v3.39.0 - Local Workflow Visibility
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.39.0 - Local Workflow Visibility. Context: Observed during installation or first-run setup.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_500227db57a8529c37cfce701ae84bdc | https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.0
## 5. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: capability.host_targets | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 6. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: v3.38.0 - Encoding repair guardrails
- User impact: Upgrade or migration may change expected behavior: v3.38.0 - Encoding repair guardrails
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.38.0 - Encoding repair guardrails. Context: Observed when using windows
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_42be567d8e4b76432faa58f51baccf59 | https://github.com/Patdolitse/piia-engram/releases/tag/v3.38.0
## 7. Configuration risk - Configuration risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: v3.40.0 - First-run confidence
- User impact: Upgrade or migration may change expected behavior: v3.40.0 - First-run confidence
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.40.0 - First-run confidence. Context: Observed during installation or first-run setup.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_e8bb2627f41868837723a394026965a3 | https://github.com/Patdolitse/piia-engram/releases/tag/v3.40.0
## 8. Capability evidence risk - Capability evidence risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: capability.assumptions | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 9. Maintenance risk - Maintenance risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: evidence.maintainer_signals | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 10. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: downstream_validation.risk_items | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 11. Security or permission risk - Security or permission risk requires verification
- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: risks.scoring_risks | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 12. Capability evidence risk - Capability evidence risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Project evidence flags a capability evidence risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_issue | fmev_2c63fe4051ac808c161d261e8e1fd5e9 | https://github.com/Patdolitse/piia-engram/issues/8
## 13. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: evidence.maintainer_signals | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 14. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: evidence.maintainer_signals | github_repo:1242620513 | https://github.com/Patdolitse/piia-engram
## 15. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_cfb0ee212bf2a234918f898536a05bf3 | https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.1
## 16. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Suggested check: Reproduce the official install and quickstart path in an isolated environment.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_1fcadca841756b4c403761fc6d2267a6 | https://github.com/Patdolitse/piia-engram/releases/tag/v3.33.2
## 17. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v3.36.0 — Identity-layer security: corpus encryption + governance closure
- User impact: Upgrade or migration may change expected behavior: v3.36.0 — Identity-layer security: corpus encryption + governance closure
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.36.0 — Identity-layer security: corpus encryption + governance closure. Context: Source discussion did not expose a precise runtime context.
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_d385dce779f00a690121395cadcd4eed | https://github.com/Patdolitse/piia-engram/releases/tag/v3.36.0
## 18. Maintenance risk - Maintenance risk requires verification
- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v3.39.1 - Terminal Encoding Diagnostics
- User impact: Upgrade or migration may change expected behavior: v3.39.1 - Terminal Encoding Diagnostics
- Suggested check: Before packaging this project, run the relevant install/config/quickstart check for: v3.39.1 - Terminal Encoding Diagnostics. Context: Observed when using python, windows
- Guardrail: State this as source-backed community evidence, not as Doramagic reproduction.
- Evidence: failure_mode_cluster:github_release | fmev_190c5a24ab97bf159a934181221eea0c | https://github.com/Patdolitse/piia-engram/releases/tag/v3.39.1