# https://github.com/aiming-lab/SimpleMem Project Manual

Generated at: 2026-07-10 23:39:39 UTC

## Table of Contents

- [Getting Started with SimpleMem](#page-1)
- [Architecture and Memory Backends](#page-2)
- [MCP Server, Docker Deployment & Security](#page-3)
- [Benchmark Reproduction & Common Issues](#page-4)

<a id='page-1'></a>

## Getting Started with SimpleMem

### Related Pages

Related topics: [Architecture and Memory Backends](#page-2), [MCP Server, Docker Deployment & Security](#page-3)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [README.md](https://github.com/aiming-lab/SimpleMem/blob/main/README.md)
- [simplemem/__init__.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/__init__.py)
- [simplemem/router.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/router.py)
- [config.py.example](https://github.com/aiming-lab/SimpleMem/blob/main/config.py.example)
- [requirements.txt](https://github.com/aiming-lab/SimpleMem/blob/main/requirements.txt)
- [examples/quickstart.py](https://github.com/aiming-lab/SimpleMem/blob/main/examples/quickstart.py)
- [setup.py](https://github.com/aiming-lab/SimpleMem/blob/main/setup.py)
</details>

# Getting Started with SimpleMem

SimpleMem is a memory-augmented framework for LLM applications that provides semantic lossless compression of conversational history, intent-aware retrieval, and (via the Omni-SimpleMem backend) multimodal memory support. Starting with **v0.3.0**, the three prior packages — `SimpleMem`, `Omni-SimpleMem`, and `EvolveMem` — are consolidated into a single `simplemem` package with an auto-routing facade, so new users no longer need to choose an import path manually. Source: [README.md:1-40]()

## What "Getting Started" Covers

This page walks a new user from a clean environment to a working memory object that can `add` and `query`. It intentionally focuses on the unified entry point rather than the older `OmniSimpleMem` or `EvolveMem` repositories, which remain in the tree for reproducibility but are no longer the recommended path. Source: [simplemem/__init__.py:1-25]()

## Installation

The package is installed from source via the standard `pyproject.toml`/`setup.py` workflow. Clone the repository and install dependencies declared in `requirements.txt` before importing the library. Source: [setup.py:1-30]()

```bash
git clone https://github.com/aiming-lab/SimpleMem.git
cd SimpleMem
pip install -r requirements.txt
pip install -e .
```

The `requirements.txt` file pins the LLM, embedding, and vector-store backends that SimpleMem relies on (typically `openai`, `tiktoken`, `lancedb`, and a sentence-transformer client). Users who only need the textual backend can install a minimal subset; users running the multimodal path need the vision extras referenced by the OmniSimpleMem subpackage. Source: [requirements.txt:1-30]()

## Configuration

SimpleMem reads runtime configuration from a project-level `config.py`. The repository ships a `config.py.example` template that should be copied and edited with your own API credentials and model identifiers. Source: [config.py.example:1-40]()

```bash
cp config.py.example config.py
```

The example file documents the fields that `OmniMemoryConfig` and related dataclasses (such as embedding model selection and LLM provider URLs) expect. Several users have reported `ModuleNotFoundError: No module named 'omni_memory.core.config'` after pulling the repo — this is the same config object, just relocated under the new unified package. Issues #49 and #60 track this migration; the practical fix is to copy `config.py.example` to `config.py` at the repository root rather than looking for a legacy `omni_memory/core/config.py` file. Source: [config.py.example:1-40]()

The relevant environment variables typically include:

| Variable | Purpose |
|----------|---------|
| `OPENAI_API_KEY` | LLM and embedding calls |
| `LLM_MODEL` | Chat model name (e.g. `gpt-4o-mini`) |
| `EMBED_MODEL` | Embedding model (e.g. `text-embedding-3-small`) |
| `LANCEDB_URI` | Local vector-store path |

## Your First Memory Session

The recommended entry point is the top-level `SimpleMem` symbol. Because of the auto-routing facade, the same class is returned for textual and multimodal use — the router inspects the first method you call and dispatches to the right backend. Source: [simplemem/__init__.py:1-25]()

```python
from simplemem import SimpleMem

mem = SimpleMem()
mem.add("User prefers concise answers and dark-mode UIs.")
mem.add("User is allergic to peanuts.")

answer = mem.query("What UI theme should I use?", k=4)
print(answer)
```

The `add` call compresses the input into structured memory entries (lossless at the semantic level), while `query` performs intent-aware retrieval before generating a response. A runnable version of this snippet lives in `examples/quickstart.py` and is the suggested starting point for experimentation. Source: [examples/quickstart.py:1-30]()

## Backend Routing

The router exposes one Python class for two backends:

```mermaid
flowchart LR
    A[from simplemem import SimpleMem] --> B{Router}
    B -- first call: add text --> C[Text Backend: SimpleMem]
    B -- first call: add image/audio --> D[Multimodal Backend: OmniSimpleMem]
    C --> E[(LanceDB Vector Store)]
    D --> E
```

The router lives in `simplemem/router.py` and selects the backend based on the modality of the first interaction. Switching mid-session is not supported; users who need both modalities should instantiate two `SimpleMem` objects with distinct persistence paths. Source: [simplemem/router.py:1-30]()

## Common First-Run Issues

The issues most frequently encountered on a fresh setup, drawn from community reports:

1. **Missing `omni_memory.core.config` module** (#49, #60): the file was renamed/relocated when the unified package shipped. Copy `config.py.example` to `config.py` and import the new path. Source: [config.py.example:1-40]()
2. **Reproduction drift on LoCoMo** (#58, #47): scores higher than the paper often come from different model versions or evaluation prompts. Pin model names explicitly in `config.py`. Source: [README.md:1-40]()
3. **Token-counting caveats** (#31): published token totals cover memory generation plus retrieval, not the answering pass alone; instrument your calls if you need a per-stage breakdown.
4. **Multimodal adapters fall back to text** (#64): some benchmark adapters under `OmniSimpleMem/benchmarks/memgallery/` drop image content during preprocessing. If you need pixel-faithful reproduction, audit the adapter before trusting its numbers. Source: [README.md:1-40]()

## Next Steps

After the quickstart works, the typical progression is:

- Run the LoCoMo and LongMemEval benchmarks under `benchmarks/` to validate your configuration.
- Tune retrieval depth (`k`), compression aggressiveness, and the embedding model.
- If you need agent integration, look at the MCP HTTP server in `OmniSimpleMem/`, but be aware of the security advisories filed against earlier CORS and auth configurations (#51, #54) and update the server before exposing it.

For deeper architectural detail, see the per-module wiki pages on the memory compressor, the vector store, and the evaluation harness.

---

<a id='page-2'></a>

## Architecture and Memory Backends

### Related Pages

Related topics: [Getting Started with SimpleMem](#page-1), [Benchmark Reproduction & Common Issues](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [simplemem/__init__.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/__init__.py)
- [simplemem/text/system.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/text/system.py)
- [simplemem/multimodal/orchestrator.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/multimodal/orchestrator.py)
- [simplemem/config.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/config.py)
- [simplemem/core/database/vector_store.py](https://github.com/aiming-lab/SimpleMem/blob/main/simplemem/core/database/vector_store.py)
- [OmniSimpleMem/omni_memory/core/config.py](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/omni_memory/core/config.py)
</details>

# Architecture and Memory Backends

## Overview

SimpleMem v0.3.0 unified three previously separate memory projects—SimpleMem, Omni-SimpleMem, and EvolveMem—into a single `simplemem` Python package. The architecture exposes one `SimpleMem` class that auto-routes between a text backend and a multimodal backend based on the first method invoked by the caller. This page describes the high-level module layout, the responsibilities of each backend, the routing mechanism, and the shared infrastructure (configuration and vector store) consumed by both.

## Package Layout and Auto-Routing

The unified package follows a backend-fanout layout: `simplemem.text` houses the textual memory system, `simplemem.multimodal` houses Omni-SimpleMem, and the top-level `__init__.py` re-exports a single `SimpleMem` facade.

`Source: [simplemem/__init__.py:1-80]()`

The `SimpleMem` class inspects the first method called on the instance (for example `.add_text()` versus `.add_image()`) and lazily constructs the appropriate backend system object on first use. Callers therefore do not need to import a separate `OmniSimpleMem` class; the same `from simplemem import SimpleMem` import serves both text-only and multimodal workloads. This design directly addresses the friction reported in issues #49 and #60, where users imported `from omni_memory.core.config import OmniMemoryConfig` from a path that did not exist in the legacy layout—the v0.3.0 re-export consolidates those imports behind the unified facade.

## Text Backend (`simplemem.text`)

The textual backend implements semantic lossless compression and intent-aware retrieval. Its main entry point is `SimpleMemSystem` in `simplemem/text/system.py`, which orchestrates ingestion, compression, indexing, and query handling.

`Source: [simplemem/text/system.py:1-120]()`

Key responsibilities delegated to sub-modules:

- **Memory construction** via `simplemem.core.memory_builder`, which prompts an LLM to extract entities, facts, and intent from dialog turns and produces structured memory entries.
- **Hybrid retrieval** via `simplemem.core.hybrid_retriever`, which combines dense similarity over embeddings with structured filters (persons, entities, timestamps) issued against the vector store.
- **Answer generation** via `simplemem.core.answer_generator`, which re-ranks retrieved entries and prompts the LLM to produce a grounded response.

This is the backend exercised by the LoCoMo benchmark. As raised in issue #31, token-cost accounting on LoCoMo covers both the memory-generation phase (add path) and the retrieval-plus-answer phase (query path); the text backend exposes both phases through the same `SimpleMemSystem` so users can attribute tokens to each stage.

## Multimodal Backend (`simplemem.multimodal`)

The multimodal backend—originally developed as OmniSimpleMem—handles images, audio, and interleaved multimodal dialog. Its entry point is `OmniOrchestrator` in `simplemem/multimodal/orchestrator.py`, which coordinates captioning, visual entity extraction, and cross-modal retrieval.

`Source: [simplemem/multimodal/orchestrator.py:1-150]()`

When the unified `SimpleMem` facade detects an image-bearing input, it routes to this orchestrator instead of the text pipeline. The MemGallery benchmark exercises this path; issue #64 raised a question about whether the MemGallery adapter operates on original images or only textual surrogates, and the multimodal orchestrator is the component responsible for that distinction.

The orchestrator depends on the same `VectorStore` primitive as the text backend, so embedding indices and filter syntax are shared across both backends rather than duplicated.

## Shared Infrastructure

Two modules are consumed by both backends and deserve explicit treatment.

### Configuration

The legacy `omni_memory.core.config` module path referenced in issues #49 and #60 has been consolidated in v0.3.0. Configuration classes (`OmniMemoryConfig`, `EmbeddingConfig`, and related dataclasses) are now exposed through `simplemem.config`, so a single import works regardless of which backend is selected. The legacy module is retained for backward compatibility but is no longer the canonical entry point.

`Source: [simplemem/config.py:1-200]()`

`Source: [OmniSimpleMem/omni_memory/core/config.py:1-180]()` (legacy path, retained for backward compatibility)

### Vector Store

`VectorStore` in `simplemem/core/database/vector_store.py` wraps LanceDB and exposes both `similarity_search` (dense vector lookup) and `structured_search` (filter-based lookup). The latter constructs `.where()` clauses by interpolating user-derived values into filter strings—an interface flagged in security issue #53 as a potential filter-injection vector. Sanitization of the `persons`, `entities`, and `timestamp` fields is therefore a shared backend responsibility, not a text-only one.

`Source: [simplemem/core/database/vector_store.py:1-220]()`

## Routing Workflow

```mermaid
flowchart TD
    User["User code: from simplemem import SimpleMem"] --> Facade[SimpleMem facade]
    Facade -->|add_text / query| Text["simplemem.text.system"]
    Facade -->|add_image / query_multimodal| MM["simplemem.multimodal.orchestrator"]
    Text --> VS["VectorStore / LanceDB"]
    MM --> VS
    Text --> Cfg["simplemem.config"]
    MM --> Cfg
```

The diagram shows the single-import, dual-backend pattern. Once a backend is locked in by the first method call, subsequent calls dispatch to the same orchestrator instance; mixed workloads (text plus image in one session) are handled by the multimodal orchestrator for the remainder of the session to avoid mid-session backend switching.

## Choosing a Backend

- **Text-only workloads** (LoCoMo, long-chat summarization) should use the default `SimpleMem` and call `add_text()` first to lock in the text backend; this minimizes per-session token overhead as discussed in issue #31.
- **Image-bearing or multimodal workloads** (MemGallery, MMLongBench-Doc) should invoke an image-aware method on first use so that `OmniOrchestrator` is selected. Note that the MMLongBench-Doc loader uses `eval()` to parse `evidence_pages`, flagged in security issue #52—downstream callers should sanitize benchmark data before ingestion.
- **Reproducibility**: community reports (#47, #58) show that LoCoMo numbers can drift by several points across model checkpoints, so pinning both the LLM model identifier and the embedding model in `simplemem.config` is required when matching published tables.

---

<a id='page-3'></a>

## MCP Server, Docker Deployment & Security

### Related Pages

Related topics: [Getting Started with SimpleMem](#page-1), [Benchmark Reproduction & Common Issues](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [MCP/README.md](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/README.md)
- [MCP/server/http_server.py](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/server/http_server.py)
- [MCP/server/mcp_handler.py](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/server/mcp_handler.py)
- [MCP/server/auth/token_manager.py](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/server/auth/token_manager.py)
- [MCP/server/database/vector_store.py](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/server/database/vector_store.py)
- [MCP/server/integrations/openrouter.py](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/server/integrations/openrouter.py)
- [MCP/Dockerfile](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/Dockerfile)
- [MCP/docker-compose.yml](https://github.com/aiming-lab/SimpleMem/blob/main/MCP/docker-compose.yml)
</details>

# MCP Server, Docker Deployment & Security

The SimpleMem project ships a standalone **Model Context Protocol (MCP)** server under the `MCP/` directory. It exposes SimpleMem's memory operations (add, search, update, delete) over HTTP/SSE so that MCP-compatible clients (Claude Desktop, IDE agents, custom agents) can use a long-term memory backend. The server is built on FastAPI/Starlette and is designed to run either locally or inside a container.

## Architecture

The MCP server is composed of three cooperating layers:

| Layer | Module | Responsibility |
|-------|--------|----------------|
| Transport | `MCP/server/http_server.py` | FastAPI app, route registration, CORS, lifespan |
| Protocol | `MCP/server/mcp_handler.py` | MCP JSON-RPC dispatch, session lifecycle |
| Backend | `MCP/server/database/vector_store.py` + `auth/token_manager.py` | LanceDB storage, per-session token issuance |

`Source: [MCP/server/http_server.py]()`

The HTTP server registers both a modern authenticated `/mcp` route and a legacy `/mcp/message` endpoint. The legacy route exists for backward compatibility with older clients and accepts a `session_id` query parameter as a fallback to bearer-token auth.

`Source: [MCP/server/mcp_handler.py]()`

### Request flow

```mermaid
sequenceDiagram
    participant Client as MCP Client
    participant HTTP as http_server.py
    participant Auth as token_manager.py
    participant DB as vector_store.py
    Client->>HTTP: POST /mcp (Bearer token)
    HTTP->>Auth: validate_token()
    Auth-->>HTTP: session context
    HTTP->>HTTP: mcp_handler.dispatch()
    HTTP->>DB: structured_search() / add()
    DB-->>Client: JSON-RPC response
```

`Source: [MCP/server/http_server.py](), [MCP/server/mcp_handler.py](), [MCP/server/auth/token_manager.py](), [MCP/server/database/vector_store.py]()`

## Authentication and Sessions

`token_manager.py` issues short-lived bearer tokens bound to a `session_id`. Tokens are validated on every request to `/mcp`. The legacy `/mcp/message` route retains a fallback that accepts only `session_id` (no token) — see the **Security** section.

`Source: [MCP/server/auth/token_manager.py]()`

Sessions are stored alongside their memories in the LanceDB-backed `VectorStore`, which keys records by `session_id` so that retrieval is automatically scoped per session.

`Source: [MCP/server/database/vector_store.py]()`

## Docker Deployment

The repository provides a containerized deployment under `MCP/` consisting of a `Dockerfile` and `docker-compose.yml`. The typical workflow is:

1. Build the image: `docker compose -f MCP/docker-compose.yml build`
2. Configure environment variables (LLM provider keys, embedding model, auth secret).
3. Launch: `docker compose -f MCP/docker-compose.yml up -d`
4. The container exposes the FastAPI server (default port defined in `http_server.py`) and a persistent volume for the LanceDB data directory.

Because the server binds to a network socket, deployment manifests must mount the data directory as a named volume to survive container restarts, and environment variables for OpenRouter or other LLM providers are injected at runtime rather than baked into the image.

`Source: [MCP/Dockerfile](), [MCP/docker-compose.yml](), [MCP/server/integrations/openrouter.py]()`

## Security

Security has been an active topic in the project's issue tracker, with five distinct advisories filed against the MCP server and its supporting modules. Operators deploying the server should be aware of the following:

**1. Unauthenticated legacy endpoint** (issue #54). `http_server.py` contains a fallback in the `/mcp/message` route (around lines 782–786) that grants session access using only a `session_id` query parameter when no bearer token is supplied. If an attacker guesses or obtains a valid session ID, they bypass authentication entirely. Recommended mitigations: disable the legacy route behind a feature flag, or require token validation regardless of the `session_id` parameter.

**Source: [MCP/server/http_server.py](), issue #54]()`

**2. Permissive CORS** (issue #51). The MCP HTTP server configures CORS with `allow_origins=["*"]` combined with `allow_credentials=True`. Starlette reflects the requesting `Origin` back as `Access-Control-Allow-Origin` in this configuration, which permits cross-origin credential theft from any domain. The fix is to enumerate explicit allowed origins or drop `allow_credentials`.

`Source: [MCP/server/http_server.py](), issue #51]()`

**3. Filter injection in `structured_search`** (issue #53). `VectorStore.structured_search` builds LanceDB `.where()` clauses by f-string interpolation of `persons`, `entities`, and timestamps. User-derived values reach the filter unsanitized, enabling SQL/filter injection. Inputs must be quoted and escaped, or replaced with parameterized predicates.

`Source: [MCP/server/database/vector_store.py](), issue #53]()`

**4. `eval()` on benchmark data** (issue #52). The MMLongBench-Doc loader uses `eval()` to parse `evidence_pages` and `evidence_sources`. A tampered dataset file leads to arbitrary code execution. Replace `eval` with `ast.literal_eval` and verify dataset integrity via a checksum.

`Source: issue #52]()`

**5. SSRF via DNS rebinding** (issue #57). The pinned `langchain-openai` version used for image-token counting contains a TOCTOU vulnerability where SSRF protections can be bypassed by DNS rebinding. Upgrade the dependency to a patched release, or pin and validate hostnames before fetching.

`Source: issue #57]()`

### Hardening checklist for production

- Remove or gate the unauthenticated fallback in `/mcp/message`.
- Replace wildcard CORS with an allowlist.
- Parameterize LanceDB filters and sanitize entity/person names.
- Pin and upgrade `langchain-openai` to a non-vulnerable release.
- Use `ast.literal_eval` everywhere benchmark data is parsed.
- Run the container as a non-root user and mount the LanceDB volume read-write only for the data directory.
- Rotate `session_id` and tokens via `token_manager.py` on a regular cadence.

`Source: [MCP/server/auth/token_manager.py](), [MCP/Dockerfile]()`

---

<a id='page-4'></a>

## Benchmark Reproduction & Common Issues

### Related Pages

Related topics: [Architecture and Memory Backends](#page-2), [MCP Server, Docker Deployment & Security](#page-3)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [test_locomo10.py](https://github.com/aiming-lab/SimpleMem/blob/main/test_locomo10.py)
- [test_ref/test_advanced.py](https://github.com/aiming-lab/SimpleMem/blob/main/test_ref/test_advanced.py)
- [test_ref/load_dataset.py](https://github.com/aiming-lab/SimpleMem/blob/main/test_ref/load_dataset.py)
- [test_ref/README.md](https://github.com/aiming-lab/SimpleMem/blob/main/test_ref/README.md)
- [OmniSimpleMem/README.md](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/README.md)
- [OmniSimpleMem/configs/locomo_config.yaml](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/configs/locomo_config.yaml)
- [OmniSimpleMem/configs/memgallery_config.yaml](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/configs/memgallery_config.yaml)
- [OmniSimpleMem/configs/mmlongbenchdoc_config.yaml](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/configs/mmlongbenchdoc_config.yaml)
- [OmniSimpleMem/benchmarks/memgallery/adapter.py](https://github.com/aiming-lab/SimpleMem/blob/main/OmniSimpleMem/benchmarks/memgallery/adapter.py)
- [README.md](https://github.com/aiming-lab/SimpleMem/blob/main/README.md)
</details>

# Benchmark Reproduction & Common Issues

## Supported Benchmarks and Reproduction Workflow

SimpleMem evaluates three primary long-context memory benchmarks: **LoCoMo** (multi-session conversational memory), **MemGallery** (long-image/document multimodal memory), and **MMLongBench-Doc** (long-document QA). Reproduction is driven by lightweight driver scripts that configure the target model and dataset, then invoke the unified `SimpleMem` class introduced in v0.3.0. Source: [README.md]()

The canonical text benchmark driver is `test_locomo10.py`, which is run from the repository root and loads the LoCoMo JSON dataset. It sets the chat model and embedding model via environment variables before instantiating `SimpleMem` and invoking `add()`, `search()`, and response generation. Source: [test_locomo10.py]()

Configuration for each benchmark is centralized under `OmniSimpleMem/configs/`. For example, `locomo_config.yaml` exposes dataset paths, model identifiers, and evaluation toggles, while `memgallery_config.yaml` and `mmlongbenchdoc_config.yaml` carry benchmark-specific fields such as image-token thresholds and retrieval depth. Source: [OmniSimpleMem/configs/locomo_config.yaml](), [OmniSimpleMem/configs/memgallery_config.yaml](), [OmniSimpleMem/configs/mmlongbenchdoc_config.yaml]()

## Dataset Loading and Adapter Layer

`test_ref/load_dataset.py` provides reusable loaders that fetch the raw benchmark artifacts (LoCoMo JSON, MMLongBench-Doc splits) and shape them into the schema expected by `SimpleMem.add`. Source: [test_ref/load_dataset.py]()

`OmniSimpleMem/benchmarks/memgallery/adapter.py` adapts the multimodal benchmark into a uniform ingestion format. Community discussion has highlighted that the current adapter normalizes images into textual captions rather than passing raw pixels; users reproducing vision-grounded metrics should verify whether textual-only adaptation matches the protocol used to produce the reported numbers. Source: [OmniSimpleMem/benchmarks/memgallery/adapter.py](), referenced in [#64]()

The advanced evaluation routine in `test_ref/test_advanced.py` adds intent-aware retrieval, scoring against expected answers using both LLM-as-judge and F1 heuristics. Source: [test_ref/test_advanced.py]()

## Common Reproduction Issues

**Missing `omni_memory.core.config` module.** Several pre-v0.3.0 imports reference `omni_memory.core.config` symbols such as `OmniMemoryConfig`. The submodule was removed in the unified release. Users still importing it will hit `ModuleNotFoundError`; the fix is to migrate to `from simplemem import SimpleMem` and to reload any YAML config via the new loader helpers. Source: issues [#49](), [#60]()

**Score drift on LoCoMo with GPT-4o.** A maintainer-reported reproduction obtains approximately 44.6 (vs the paper's 39.06) when running GPT-4o end-to-end. The discrepancy correlates with judging-strategy changes and updated API versions; users should pin model snapshots and reconcile scoring prompts before claiming paper-matching numbers. Source: [#58]()

**Table 3 divergence with small Qwen models.** Reported LoCoMo numbers for Qwen2.5-1.5B/3B and Qwen3-1.7B/8B are sensitive to chat-template and tokenizer settings. Community requests have asked for the exact `generation_config`, `max_new_tokens`, and chat formatting used in the original runs; until published, treat the small-model row as a lower bound. Source: [#47]()

**MemGallery textual-only adapter.** As noted above, the current adapter does not feed raw images into the memory encoder, so any multimodal gain is unmeasured in the shipped benchmark driver. Source: [#64]()

**Token accounting ambiguity.** Token counts quoted in the paper (for example 572 tokens on LoCoMo with Qwen2.5-3B) include both write-time compression and read-time retrieval; users reproducing efficiency claims should sum tokens across both phases. Source: [#31]()

## Security Considerations When Running Benchmarks

Reproducing public benchmarks involves executing untrusted or semi-trusted dataset files. Reviewers have flagged several concrete risks:

- `eval()` in dataset loaders (notably MMLongBench-Doc) can execute arbitrary Python if the upstream artifact is tampered with. Pin dataset checksums and prefer `json.loads` over `eval` whenever possible. Source: [#52]()
- `VectorStore.structured_search` interpolates `persons`, `entities`, and timestamps into LanceDB `.where()` clauses via f-strings, enabling filter injection. Sanitize identifiers or use parameterized filters before ingesting benchmark-derived metadata. Source: [#53]()
- The MCP HTTP server sets `allow_origins=["*"]` together with `allow_credentials=True`, which Starlette reflects back as the requesting origin. Disable wildcard credentials when exposing benchmark runners over the network. Source: [#51]()
- A legacy `/mcp/message` fallback authorizes sessions with a `session_id` query parameter only, bypassing bearer tokens. Restrict or remove this route before running evaluations against a remotely reachable host. Source: [#54]()

## Practical Reproduction Checklist

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | Use the unified `simplemem` package and the YAML in `OmniSimpleMem/configs/` | Avoids the legacy `omni_memory.core.config` import path |
| 2 | Pin model versions (LLM and embeddings) and the dataset commit hash | Reproduces Table 3 numbers within a known delta |
| 3 | Confirm whether multimodal adapters ingest raw pixels or captions | Determines whether MemGallery scores are comparable |
| 4 | Sanitize dataset loaders and disable legacy MCP endpoints | Mitigates the security issues listed above |
| 5 | Sum read- and write-time tokens when comparing efficiency claims | Matches the methodology behind the paper's token budgets |

Following the checklist above produces a pipeline that closely tracks the paper's reported numbers within the variance bands documented by the maintainers, while avoiding the most common environment and configuration pitfalls. Source: [OmniSimpleMem/README.md](), [test_ref/README.md]()

---

<!-- evidence_pipeline_checked: true -->
<!-- evidence_injected: true -->

---

## Pitfall Log

Project: aiming-lab/SimpleMem

Summary: Found 19 structured pitfall item(s), including 1 high/blocking item(s). Top priority: Security or permission risk - Security or permission risk requires verification.

## 1. Security or permission risk - Security or permission risk requires verification

- Severity: high
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/58

## 2. Installation risk - Installation risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/64

## 3. Installation risk - Installation risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/63

## 4. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.host_targets | https://github.com/aiming-lab/SimpleMem

## 5. Capability evidence risk - Capability evidence risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://github.com/aiming-lab/SimpleMem

## 6. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/aiming-lab/SimpleMem

## 7. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://github.com/aiming-lab/SimpleMem

## 8. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://github.com/aiming-lab/SimpleMem

## 9. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/60

## 10. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/47

## 11. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/50

## 12. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/54

## 13. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/51

## 14. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/53

## 15. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/52

## 16. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/57

## 17. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/aiming-lab/SimpleMem/issues/62

## 18. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/aiming-lab/SimpleMem

## 19. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/aiming-lab/SimpleMem

<!-- canonical_name: aiming-lab/SimpleMem; human_manual_source: deepwiki_human_wiki -->