# https://github.com/experientiallabs/world-model-optimizer Project Manual

Generated at: 2026-07-27 00:27:50 UTC

## Table of Contents

- [Introduction and Getting Started](#page-1)
- [System Architecture and Core Modules](#page-2)
- [World Models, Optimization, and Distillation](#page-3)
- [Data Ingestion, Providers, and Platform Integrations](#page-4)

<a id='page-1'></a>

## Introduction and Getting Started

### Related Pages

Related topics: [System Architecture and Core Modules](#page-2)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [README.md](https://github.com/experientiallabs/world-model-optimizer/blob/main/README.md)
- [pyproject.toml](https://github.com/experientiallabs/world-model-optimizer/blob/main/pyproject.toml)
- [wmo/__init__.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/__init__.py)
- [wmo/cli/app.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/cli/app.py)
- [wmo/providers/registry.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/providers/registry.py)
</details>

# Introduction and Getting Started

## Overview

`world-model-optimizer` is a Python toolkit for working with world models in machine learning workflows. It exposes a single importable package (`wmo`) and a console entry point (`wmo`) that together provide the public surface for running, configuring, and extending the tool. The package is published to PyPI as `world-model-optimizer`, and version `0.2.0` marks the first release under this name. `Source: [README.md:1-40]()`

The project was previously distributed under the name `world-model-harness`. The rename is total: the PyPI distribution, the importable package, the CLI command, and the project metadata were all updated together in `v0.2.0`. Users upgrading from a previous install must reinstall under the new name rather than upgrading in place. `Source: [README.md:1-60]()`

The repository is organized around a small, predictable layout:

- `wmo/` — the importable Python package, including the package marker `wmo/__init__.py`.
- `wmo/cli/app.py` — the CLI application registered under the `wmo` console script.
- `wmo/providers/registry.py` — provider registration machinery that backends plug into.
- `pyproject.toml` — build, dependency, and entry-point configuration.

`Source: [pyproject.toml:1-80]()`

## Installation

The recommended install path is from PyPI:

```bash
pip install world-model-optimizer
```

This installs the `wmo` importable package and registers the `wmo` console script in the active environment. `Source: [README.md:1-30]()`

Once installed, the CLI can be invoked directly:

```bash
wmo --help
```

This prints the top-level command tree exposed by `wmo/cli/app.py`, which is the canonical way to confirm that the install succeeded and to discover subcommands. `Source: [wmo/cli/app.py:1-40]()`

The package is configured as a standard PEP 621 project in `pyproject.toml`, which declares the `wmo` console script entry point that resolves to the application object defined in `wmo/cli/app.py`. `Source: [pyproject.toml:1-80]()`

## Quick Start

After installation, the typical first interaction is to inspect the CLI help and the package marker:

```bash
wmo --help
python -c "import wmo; print(wmo.__file__)"
```

The `import wmo` form resolves to `wmo/__init__.py`, which is the canonical package marker. Importing it is the supported way to verify that the package is installed in the current interpreter and that no import-time errors have been introduced by provider plugins. `Source: [wmo/__init__.py:1-20]()`

Backends are wired in through the provider registry rather than being hard-coded into the CLI. The registry lives in `wmo/providers/registry.py` and acts as the single point where new model providers can be registered, looked up, and selected at runtime. This means that adding a new backend does not require changes to `wmo/cli/app.py`; registering the provider is sufficient. `Source: [wmo/providers/registry.py:1-60]()`

The CLI itself is a thin layer over the registry and the core library: it parses commands and arguments, then delegates to the appropriate provider via the registry. Because of that separation, the same backends that the CLI uses are also accessible programmatically through `import wmo`. `Source: [wmo/cli/app.py:1-60]()`

## Migration from `world-model-harness`

If you have an existing install from before the rename, the names changed across the entire stack:

| | before | after |
|---|---|---|
| PyPI package | `world-model-harness` | `world-model-optimizer` |
| Importable package | `harness` / `world_model_harness` | `wmo` |
| Console script | `harness` / equivalent | `wmo` |

`Source: [README.md:1-60]()`

The supported migration is a clean reinstall under the new name rather than an in-place upgrade:

```bash
pip uninstall world-model-harness
pip install world-model-optimizer
```

After reinstall, any code or shell aliases that referenced the old import path or old console script must be updated to `wmo` and `world-model-optimizer` respectively. The CLI's own `--help` output, generated from `wmo/cli/app.py`, is the authoritative reference for the current command set under the new name. `Source: [wmo/cli/app.py:1-40]()`

For users tracking the project, `v0.2.0` is the cutoff: releases before it shipped under `world-model-harness`, and `v0.2.0` and later ship under `world-model-optimizer`. Pinning to the new distribution name when declaring dependencies in downstream projects is required to receive future updates. `Source: [pyproject.toml:1-80]()`

## Next Steps

From a fresh install, the productive next moves are:

1. Run `wmo --help` to enumerate available subcommands. `Source: [wmo/cli/app.py:1-40]()`
2. Open `wmo/providers/registry.py` to see which providers are registered out of the box and how to register additional ones. `Source: [wmo/providers/registry.py:1-60]()`
3. Read the README for project-specific workflows and example invocations. `Source: [README.md:1-40]()`

These three entry points — the CLI help, the provider registry, and the README — cover the surface area a new user needs in order to understand what `world-model-optimizer` offers, how it is wired together, and how to extend it.

---

<a id='page-2'></a>

## System Architecture and Core Modules

### Related Pages

Related topics: [Introduction and Getting Started](#page-1), [World Models, Optimization, and Distillation](#page-3)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [wmo/engine/world_model.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/engine/world_model.py)
- [wmo/engine/loader.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/engine/loader.py)
- [wmo/engine/play.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/engine/play.py)
- [wmo/engine/build.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/engine/build.py)
- [wmo/harness/create.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/harness/create.py)
- [wmo/harness/mutate.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/harness/mutate.py)
</details>

# System Architecture and Core Modules

`world-model-optimizer` (importable as `wmo`) is a Python framework for constructing, mutating, and interacting with simulated world models used to evaluate and optimize agents or prompts. The package exposes a CLI entry point (`wmo`) and a programmatic Python API, both of which delegate to two collaborating subpackages: `wmo.engine` for runtime simulator mechanics and `wmo.harness` for scenario authoring and search.

## High-Level Architecture

The codebase enforces a clean separation between **engine** primitives (which implement the simulator/runtime) and **harness** primitives (which describe test scenarios, mutations, and generation pipelines). External callers interact either through the CLI or by importing the public Python API; both paths ultimately route through the engine and harness modules rather than touching internal state directly.

```mermaid
flowchart LR
    CLI[wmo CLI] --> Engine
    API[Python API] --> Engine
    API --> Harness
    Engine --> WM[world_model.py]
    Engine --> Loader[loader.py]
    Engine --> Play[play.py]
    Engine --> Build[build.py]
    Harness --> Create[create.py]
    Harness --> Mutate[mutate.py]
    Create --> Build
    Mutate --> Build
    Build --> WM
    Loader --> WM
```

## Engine Module: Runtime Mechanics

The `wmo/engine/` package contains the core simulator logic. It is responsible for instantiating a world model, loading its state or configuration, building a fresh instance, and executing the interaction loop against a given agent.

### World Model Core

`world_model.py` defines the central domain object that represents the simulated environment. It encapsulates state, transition rules, and the contract for stepping the simulator forward in response to agent actions. Any external code that wishes to evaluate behavior in a world model either constructs a `WorldModel` directly or obtains one through `loader.py`. Source: [wmo/engine/world_model.py:1-80]()

### Loader

`loader.py` materializes a `WorldModel` from a serialized representation (such as a configuration file or directory layout produced by `build.py`). Centralizing deserialization in the loader keeps `WorldModel` free of I/O concerns and lets the CLI and harness modules share a single materialization path. Source: [wmo/engine/loader.py:1-60]()

### Build

`build.py` is the counterpart to the loader: it constructs and serializes a new `WorldModel` together with any associated fixtures, prompts, or initial state. Calls originating in `wmo/harness/create.py` typically flow through `build.py` to produce a runnable artifact. Source: [wmo/engine/build.py:1-80]()

### Play

`play.py` implements the interaction loop between an agent (or policy) and a `WorldModel`. Given a loaded world model and an agent, it drives the simulation forward, collects observations and actions, and emits results. This is the primary execution entry point used by evaluators and the CLI. Source: [wmo/engine/play.py:1-100]()

## Harness Module: Scenario Authoring

The `wmo/harness/` package sits one level above the engine. It supplies primitives for generating and transforming the scenarios that the engine consumes, which is what enables the "optimizer" portion of the package name.

### Create

`create.py` produces new harness scenarios from scratch. It composes `build.py` to assemble a fresh `WorldModel` along with the agent configuration, scoring criteria, and metadata required for a reproducible run. Source: [wmo/harness/create.py:1-80]()

### Mutate

`mutate.py` transforms an existing scenario into a related variant. This is the mechanism that powers the optimizer: given a parent scenario, `mutate.py` produces child scenarios whose outcomes can be compared against the parent. Repeated application of `mutate` (typically driven from the CLI) implements an iterative or evolutionary search. Source: [wmo/harness/mutate.py:1-100]()

## Interaction Between Engine and Harness

The harness layer does not duplicate engine logic; it composes it. A typical workflow is:

1. `create.py` calls `build.py` to produce a world-model artifact.
2. The artifact is fed to `loader.py` for deserialization on subsequent runs.
3. `play.py` runs the simulation against an agent and returns results.
4. `mutate.py` consumes those results (or the parent scenario) to generate new candidates.

This keeps the engine focused on simulation fidelity while the harness handles search, variation, and orchestration. The strict directional flow (harness → engine, never the reverse) is what makes the engine reusable independent of any particular search strategy. Source: [wmo/harness/create.py:1-80](), [wmo/harness/mutate.py:1-100]()

## Community Notes on the v0.2.0 Rename

The v0.2.0 release renamed the project from `world-model-harness` to `world-model-optimizer`, and the rename extends throughout the package layout. Anyone migrating from a pre-0.2.0 install should update imports from `world_model_harness.*` to `wmo.*`. The PyPI distribution is now published as `world-model-optimizer`, installable with `pip install world-model-optimizer` and invocable as `wmo --help`. This rename is the most common source of breakage reported by users upgrading across the 0.1.x → 0.2.0 boundary, so it is worth verifying both the package name and the import path after upgrading.

---

<a id='page-3'></a>

## World Models, Optimization, and Distillation

### Related Pages

Related topics: [System Architecture and Core Modules](#page-2), [Data Ingestion, Providers, and Platform Integrations](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [wmo/optimize/routing.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/routing.py)
- [wmo/optimize/knn.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/knn.py)
- [wmo/optimize/judge.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/judge.py)
- [wmo/optimize/gepa.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/gepa.py)
- [wmo/optimize/policy.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/policy.py)
- [wmo/optimize/judge_quality.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/optimize/judge_quality.py)
</details>

# World Models, Optimization, and Distillation

The `world-model-optimizer` package (installed via `pip install world-model-optimizer` and invoked with the `wmo` CLI) provides a modular framework for steering, evaluating, and compressing world-model behavior. It separates concerns into discrete optimization strategies under `wmo/optimize/` and pairs them with judge-based evaluation components that make distillation tractable. As of release `v0.2.0`, this is the first release published under the `world-model-optimizer` name, replacing the prior `world-model-harness` distribution.

## Purpose and Scope

The framework's purpose is to take a base world model and produce a smaller, faster, or higher-quality policy without retraining the model from scratch. It does this by composing three layers:

1. **Optimization strategies** that decide which action, prompt, or trajectory variant to emit (`routing`, `knn`, `gepa`, `policy`).
2. **Judges** that score candidate outputs against quality criteria (`judge`, `judge_quality`).
3. A **distillation loop** in which the optimizer's high-scoring outputs are recorded as supervision signals for the smaller student model.

The scope is deliberately narrow: the optimizer never touches model weights directly. It operates on candidates produced by the world model, ranking them with the strategies and judges, and writes artifacts (logs, distilled datasets, routing tables) that downstream training jobs consume. Source: [wmo/optimize/policy.py]().

## Optimization Strategies

Four strategy modules live side by side under `wmo/optimize/`, each implementing a different selection rule:

| Strategy | Module | Selection Rule |
|---|---|---|
| Routing | `routing.py` | Dispatches a request to a configured handler or model endpoint based on rules. |
| kNN | `knn.py` | Retrieves the nearest historical trajectories by embedding similarity and reuses their actions. |
| GEPA | `gepa.py` | Genetic/Pareto-style evolutionary search over prompt or trajectory variants. |
| Policy | `policy.py` | Applies a learned or configured decision policy over candidate outputs. |

These strategies are designed to be swappable. A user composing a pipeline picks one strategy per request, or stacks them — for example, `knn` to narrow candidates, then `policy` to rank the shortlist, then `gepa` to mutate the top entries. Source: [wmo/optimize/routing.py](), [wmo/optimize/knn.py](), [wmo/optimize/gepa.py](), [wmo/optimize/policy.py]().

```mermaid
flowchart LR
    A[World Model Output] --> B[Routing]
    B --> C[kNN Retrieval]
    C --> D[GEPA Search]
    D --> E[Policy Rank]
    E --> F[Judge Quality]
    F --> G[Distilled Artifact]
```

## Judging and Quality Scoring

`judge.py` and `judge_quality.py` form the evaluation backbone. The base `judge` module exposes a general scoring interface used to compare two or more candidate trajectories, while `judge_quality.py` specializes in assessing output quality along dimensions relevant to distillation (faithfulness, consistency, task success). Their outputs feed back into the optimization strategies so that later iterations bias toward higher-scoring candidates. Source: [wmo/optimize/judge.py](), [wmo/optimize/judge_quality.py]().

## Distillation Workflow

Distillation in this project is an offline, judgment-driven process rather than an online gradient step. The loop is:

1. The world model generates multiple candidate trajectories for a prompt.
2. An optimization strategy (typically `gepa` for exploration or `policy` for exploitation) selects and mutates candidates.
3. `judge` and `judge_quality` score each candidate.
4. High-scoring candidates are written as `(prompt, response)` pairs into a distillation dataset.
5. A separate training job consumes the dataset to fit a smaller student model.

Because strategies and judges are decoupled, the same distillation loop can be re-run with different strategies (for example, swapping `gepa` for `knn`) without changing the judge code, and vice versa. Source: [wmo/optimize/policy.py](), [wmo/optimize/gepa.py](), [wmo/optimize/judge_quality.py]().

## Practical Notes

- The CLI entry point is `wmo`, and `wmo --help` lists the available optimize subcommands corresponding to the modules above. Source: [wmo/optimize/routing.py]().
- The v0.2.0 rename from `world-model-harness` to `world-model-optimizer` is reflected throughout `wmo/optimize/`, so any user scripts importing the old name must be updated.
- Strategy modules are intended to be composable; if a pipeline needs both retrieval and search, instantiate `knn` before `gepa` in the chain.

---

<a id='page-4'></a>

## Data Ingestion, Providers, and Platform Integrations

### Related Pages

Related topics: [System Architecture and Core Modules](#page-2), [World Models, Optimization, and Distillation](#page-3)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [wmo/ingest/adapter.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/adapter.py)
- [wmo/ingest/base.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/base.py)
- [wmo/ingest/braintrust.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/braintrust.py)
- [wmo/ingest/langfuse.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/langfuse.py)
- [wmo/ingest/langsmith.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/langsmith.py)
- [wmo/ingest/mastra.py](https://github.com/experientiallabs/world-model-optimizer/blob/main/wmo/ingest/mastra.py)
</details>

# Data Ingestion, Providers, and Platform Integrations

The `wmo/ingest` package is the entry point for pulling trace and execution data from external LLM-observability platforms into `world-model-optimizer`. It defines a uniform ingestion contract, a single registry-style adapter, and per-provider implementations that normalize proprietary schemas into the optimizer's internal representation. This page documents the structure of that subsystem, the abstractions it relies on, and the four currently supported platforms.

## Architecture and Data Flow

Ingestion follows a two-layer design: a **base abstraction** that defines what an "ingestion source" is, and a **registry adapter** that maps a string identifier (such as `"braintrust"` or `"langfuse"`) to a concrete provider implementation. The remaining modules each subclass the base to talk to one specific platform.

```mermaid
flowchart LR
    A[CLI / Python API] --> B[adapter.py<br/>registry]
    B -->|provider name| C[base.py<br/>IngestSource]
    C --> D[braintrust.py]
    C --> E[langfuse.py]
    C --> F[langsmith.py]
    C --> G[mastra.py]
    D --> H[Normalized<br/>Trace Records]
    E --> H
    F --> H
    G --> H
    H --> I[Optimizer core]
```

The CLI command `wmo` resolves a user-supplied provider string through the adapter, instantiates the corresponding class, and streams normalized records downstream. Source: [wmo/ingest/adapter.py:1-40]().

## Base Abstraction

`wmo/ingest/base.py` defines the contract that every provider implementation must satisfy. It exposes the `IngestSource` base class along with the normalized record types (`Trace`, `Span`, `Generation`, `ToolCall`, and associated metadata fields) that downstream optimization stages consume. Provider modules never expose platform-specific shapes to the rest of the system; they convert raw responses into these canonical types. Source: [wmo/ingest/base.py:1-60]().

Key responsibilities encoded in the base layer include:

- Declaring authentication parameters (API keys, project IDs, host URLs) and validating them at construction time.
- Defining `fetch_traces()` / `iter_traces()` style methods so callers can paginate through large trace corpora without loading everything into memory.
- Standardizing timestamp handling, token accounting, and latency fields so cross-provider analytics remain comparable.

By centralizing the schema, the optimizer can add a new provider by writing only the platform-specific fetcher and mapping logic, leaving the rest of the pipeline untouched. Source: [wmo/ingest/base.py:30-80]().

## Provider Implementations

Four provider modules live alongside `base.py`, each isolating the idiosyncrasies of a single observability platform.

- **Braintrust** — `wmo/ingest/braintrust.py` wraps the Braintrust SDK/API to pull experiments, spans, and scored outputs. It maps Braintrust's notion of "experiments" and "spans" onto the canonical `Trace`/`Span` records, preserving scores when present. Source: [wmo/ingest/braintrust.py:1-50]().
- **Langfuse** — `wmo/ingest/langfuse.py` talks to Langfuse's `/api/public/` endpoints. It uses the project's public/secret key pair and supports both hosted and self-hosted Langfuse instances by accepting a configurable `host` parameter. Pagination is implemented against Langfuse's cursor-based API. Source: [wmo/ingest/langfuse.py:1-60]().
- **LangSmith** — `wmo/ingest/langsmith.py` integrates with LangSmith's `list_runs` API. It filters by project, time range, and run type, then flattens the nested run tree into the linear span representation expected by the optimizer. Source: [wmo/ingest/langsmith.py:1-55]().
- **Mastra** — `wmo/ingest/mastra.py` provides ingestion for Mastra traces. It supports the storage backends Mastra exposes and applies the same normalization rules as the other providers. Source: [wmo/ingest/mastra.py:1-45]().

Each module is intentionally small and self-contained: if a user only needs one platform, they pay the import cost for just that provider's transitive dependencies.

## The Adapter and Registry

`wmo/ingest/adapter.py` is the thin glue that ties the providers together. It typically exposes a `get_provider(name: str, **config) -> IngestSource` factory and a `list_providers()` helper so callers (including the `wmo` CLI) can enumerate supported integrations without importing each module eagerly.

The adapter performs three jobs:

1. **Lookup** — resolve the textual provider name to a class via a `dict` or `EntryPoint`-based registry. Source: [wmo/ingest/adapter.py:10-25]().
2. **Configuration** — pass through credentials and project identifiers supplied by the caller, deferring validation to the provider's constructor. Source: [wmo/ingest/adapter.py:25-40]().
3. **Lazy import** — import provider modules on demand so that optional SDK dependencies are only required when the corresponding provider is actually used. Source: [wmo/ingest/adapter.py:40-60]().

This pattern keeps the core package lightweight and lets users install only the provider SDKs they need.

## Selecting a Provider

In practice, a user invokes the CLI with a `--provider` flag (or its equivalent in the Python API), supplies credentials via environment variables or flags, and lets the adapter materialize the right class. The renaming from `world-model-harness` to `world-model-optimizer` (the v0.2.0 breaking change described in the community context) applies uniformly across these modules, so all import paths, entry points, and configuration keys now use the `wmo.` prefix. Source: [wmo/ingest/adapter.py:1-15]().

Once traces are pulled, they are streamed into the optimizer's core, where they are used to build world models and drive prompt/parameter optimization. The ingestion layer's sole responsibility is to deliver a consistent, normalized stream — every provider-specific quirk is contained within its own module.

---

<!-- evidence_pipeline_checked: true -->
<!-- evidence_injected: true -->

---

## Pitfall Log

Project: experientiallabs/world-model-optimizer

Summary: Found 6 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification.

## 1. Capability evidence risk - Capability evidence risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://news.ycombinator.com/item?id=49063454

## 2. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

## 3. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://news.ycombinator.com/item?id=49063454

## 4. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://news.ycombinator.com/item?id=49063454

## 5. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

## 6. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

<!-- canonical_name: experientiallabs/world-model-optimizer; human_manual_source: deepwiki_human_wiki -->