Related Pages

Related topics: System Architecture and Core Modules

Introduction and Getting Started

Overview

world-model-optimizer is a Python toolkit for working with world models in machine learning workflows. It exposes a single importable package (wmo) and a console entry point (wmo) that together provide the public surface for running, configuring, and extending the tool. The package is published to PyPI as world-model-optimizer, and version 0.2.0 marks the first release under this name. Source: README.md:1-40

The project was previously distributed under the name world-model-harness. The rename is total: the PyPI distribution, the importable package, the CLI command, and the project metadata were all updated together in v0.2.0. Users upgrading from a previous install must reinstall under the new name rather than upgrading in place. Source: README.md:1-60

The repository is organized around a small, predictable layout:

• wmo/ — the importable Python package, including the package marker wmo/__init__.py.
• wmo/cli/app.py — the CLI application registered under the wmo console script.
• wmo/providers/registry.py — provider registration machinery that backends plug into.
• pyproject.toml — build, dependency, and entry-point configuration.

Source: pyproject.toml:1-80

Installation

The recommended install path is from PyPI:

pip install world-model-optimizer

This installs the wmo importable package and registers the wmo console script in the active environment. Source: README.md:1-30

Once installed, the CLI can be invoked directly:

wmo --help

This prints the top-level command tree exposed by wmo/cli/app.py, which is the canonical way to confirm that the install succeeded and to discover subcommands. Source: wmo/cli/app.py:1-40

The package is configured as a standard PEP 621 project in pyproject.toml, which declares the wmo console script entry point that resolves to the application object defined in wmo/cli/app.py. Source: pyproject.toml:1-80

Quick Start

After installation, the typical first interaction is to inspect the CLI help and the package marker:

wmo --help
python -c "import wmo; print(wmo.__file__)"

The import wmo form resolves to wmo/__init__.py, which is the canonical package marker. Importing it is the supported way to verify that the package is installed in the current interpreter and that no import-time errors have been introduced by provider plugins. Source: wmo/__init__.py:1-20

Backends are wired in through the provider registry rather than being hard-coded into the CLI. The registry lives in wmo/providers/registry.py and acts as the single point where new model providers can be registered, looked up, and selected at runtime. This means that adding a new backend does not require changes to wmo/cli/app.py; registering the provider is sufficient. Source: wmo/providers/registry.py:1-60

The CLI itself is a thin layer over the registry and the core library: it parses commands and arguments, then delegates to the appropriate provider via the registry. Because of that separation, the same backends that the CLI uses are also accessible programmatically through import wmo. Source: wmo/cli/app.py:1-60

Migration from `world-model-harness`

If you have an existing install from before the rename, the names changed across the entire stack:

Source: README.md:1-60

The supported migration is a clean reinstall under the new name rather than an in-place upgrade:

pip uninstall world-model-harness
pip install world-model-optimizer

After reinstall, any code or shell aliases that referenced the old import path or old console script must be updated to wmo and world-model-optimizer respectively. The CLI's own --help output, generated from wmo/cli/app.py, is the authoritative reference for the current command set under the new name. Source: wmo/cli/app.py:1-40

For users tracking the project, v0.2.0 is the cutoff: releases before it shipped under world-model-harness, and v0.2.0 and later ship under world-model-optimizer. Pinning to the new distribution name when declaring dependencies in downstream projects is required to receive future updates. Source: pyproject.toml:1-80

Next Steps

From a fresh install, the productive next moves are:

1. Run wmo --help to enumerate available subcommands. Source: wmo/cli/app.py:1-40
2. Open wmo/providers/registry.py to see which providers are registered out of the box and how to register additional ones. Source: wmo/providers/registry.py:1-60
3. Read the README for project-specific workflows and example invocations. Source: README.md:1-40

These three entry points — the CLI help, the provider registry, and the README — cover the surface area a new user needs in order to understand what world-model-optimizer offers, how it is wired together, and how to extend it.

Related Pages

Related topics: Introduction and Getting Started, World Models, Optimization, and Distillation

System Architecture and Core Modules

world-model-optimizer (importable as wmo) is a Python framework for constructing, mutating, and interacting with simulated world models used to evaluate and optimize agents or prompts. The package exposes a CLI entry point (wmo) and a programmatic Python API, both of which delegate to two collaborating subpackages: wmo.engine for runtime simulator mechanics and wmo.harness for scenario authoring and search.

High-Level Architecture

The codebase enforces a clean separation between engine primitives (which implement the simulator/runtime) and harness primitives (which describe test scenarios, mutations, and generation pipelines). External callers interact either through the CLI or by importing the public Python API; both paths ultimately route through the engine and harness modules rather than touching internal state directly.

flowchart LR
    CLI[wmo CLI] --> Engine
    API[Python API] --> Engine
    API --> Harness
    Engine --> WM[world_model.py]
    Engine --> Loader[loader.py]
    Engine --> Play[play.py]
    Engine --> Build[build.py]
    Harness --> Create[create.py]
    Harness --> Mutate[mutate.py]
    Create --> Build
    Mutate --> Build
    Build --> WM
    Loader --> WM

Engine Module: Runtime Mechanics

The wmo/engine/ package contains the core simulator logic. It is responsible for instantiating a world model, loading its state or configuration, building a fresh instance, and executing the interaction loop against a given agent.

World Model Core

world_model.py defines the central domain object that represents the simulated environment. It encapsulates state, transition rules, and the contract for stepping the simulator forward in response to agent actions. Any external code that wishes to evaluate behavior in a world model either constructs a WorldModel directly or obtains one through loader.py. Source: wmo/engine/world_model.py:1-80

Loader

loader.py materializes a WorldModel from a serialized representation (such as a configuration file or directory layout produced by build.py). Centralizing deserialization in the loader keeps WorldModel free of I/O concerns and lets the CLI and harness modules share a single materialization path. Source: wmo/engine/loader.py:1-60

Build

build.py is the counterpart to the loader: it constructs and serializes a new WorldModel together with any associated fixtures, prompts, or initial state. Calls originating in wmo/harness/create.py typically flow through build.py to produce a runnable artifact. Source: wmo/engine/build.py:1-80

Play

play.py implements the interaction loop between an agent (or policy) and a WorldModel. Given a loaded world model and an agent, it drives the simulation forward, collects observations and actions, and emits results. This is the primary execution entry point used by evaluators and the CLI. Source: wmo/engine/play.py:1-100

Harness Module: Scenario Authoring

The wmo/harness/ package sits one level above the engine. It supplies primitives for generating and transforming the scenarios that the engine consumes, which is what enables the "optimizer" portion of the package name.

Create

create.py produces new harness scenarios from scratch. It composes build.py to assemble a fresh WorldModel along with the agent configuration, scoring criteria, and metadata required for a reproducible run. Source: wmo/harness/create.py:1-80

Mutate

mutate.py transforms an existing scenario into a related variant. This is the mechanism that powers the optimizer: given a parent scenario, mutate.py produces child scenarios whose outcomes can be compared against the parent. Repeated application of mutate (typically driven from the CLI) implements an iterative or evolutionary search. Source: wmo/harness/mutate.py:1-100

Interaction Between Engine and Harness

The harness layer does not duplicate engine logic; it composes it. A typical workflow is:

1. create.py calls build.py to produce a world-model artifact.
2. The artifact is fed to loader.py for deserialization on subsequent runs.
3. play.py runs the simulation against an agent and returns results.
4. mutate.py consumes those results (or the parent scenario) to generate new candidates.

This keeps the engine focused on simulation fidelity while the harness handles search, variation, and orchestration. The strict directional flow (harness → engine, never the reverse) is what makes the engine reusable independent of any particular search strategy. Source: wmo/harness/create.py:1-80, wmo/harness/mutate.py:1-100

Community Notes on the v0.2.0 Rename

The v0.2.0 release renamed the project from world-model-harness to world-model-optimizer, and the rename extends throughout the package layout. Anyone migrating from a pre-0.2.0 install should update imports from world_model_harness.* to wmo.*. The PyPI distribution is now published as world-model-optimizer, installable with pip install world-model-optimizer and invocable as wmo --help. This rename is the most common source of breakage reported by users upgrading across the 0.1.x → 0.2.0 boundary, so it is worth verifying both the package name and the import path after upgrading.

Related Pages

Related topics: System Architecture and Core Modules, Data Ingestion, Providers, and Platform Integrations

World Models, Optimization, and Distillation

The world-model-optimizer package (installed via pip install world-model-optimizer and invoked with the wmo CLI) provides a modular framework for steering, evaluating, and compressing world-model behavior. It separates concerns into discrete optimization strategies under wmo/optimize/ and pairs them with judge-based evaluation components that make distillation tractable. As of release v0.2.0, this is the first release published under the world-model-optimizer name, replacing the prior world-model-harness distribution.

Purpose and Scope

The framework's purpose is to take a base world model and produce a smaller, faster, or higher-quality policy without retraining the model from scratch. It does this by composing three layers:

1. Optimization strategies that decide which action, prompt, or trajectory variant to emit (routing, knn, gepa, policy).
2. Judges that score candidate outputs against quality criteria (judge, judge_quality).
3. A distillation loop in which the optimizer's high-scoring outputs are recorded as supervision signals for the smaller student model.

The scope is deliberately narrow: the optimizer never touches model weights directly. It operates on candidates produced by the world model, ranking them with the strategies and judges, and writes artifacts (logs, distilled datasets, routing tables) that downstream training jobs consume. Source: wmo/optimize/policy.py.

Optimization Strategies

Four strategy modules live side by side under wmo/optimize/, each implementing a different selection rule:

These strategies are designed to be swappable. A user composing a pipeline picks one strategy per request, or stacks them — for example, knn to narrow candidates, then policy to rank the shortlist, then gepa to mutate the top entries. Source: wmo/optimize/routing.py, wmo/optimize/knn.py, wmo/optimize/gepa.py, wmo/optimize/policy.py.

flowchart LR
    A[World Model Output] --> B[Routing]
    B --> C[kNN Retrieval]
    C --> D[GEPA Search]
    D --> E[Policy Rank]
    E --> F[Judge Quality]
    F --> G[Distilled Artifact]

Judging and Quality Scoring

judge.py and judge_quality.py form the evaluation backbone. The base judge module exposes a general scoring interface used to compare two or more candidate trajectories, while judge_quality.py specializes in assessing output quality along dimensions relevant to distillation (faithfulness, consistency, task success). Their outputs feed back into the optimization strategies so that later iterations bias toward higher-scoring candidates. Source: wmo/optimize/judge.py, wmo/optimize/judge_quality.py.

Distillation Workflow

Distillation in this project is an offline, judgment-driven process rather than an online gradient step. The loop is:

1. The world model generates multiple candidate trajectories for a prompt.
2. An optimization strategy (typically gepa for exploration or policy for exploitation) selects and mutates candidates.
3. judge and judge_quality score each candidate.
4. High-scoring candidates are written as (prompt, response) pairs into a distillation dataset.
5. A separate training job consumes the dataset to fit a smaller student model.

Because strategies and judges are decoupled, the same distillation loop can be re-run with different strategies (for example, swapping gepa for knn) without changing the judge code, and vice versa. Source: wmo/optimize/policy.py, wmo/optimize/gepa.py, wmo/optimize/judge_quality.py.

Practical Notes

• The CLI entry point is wmo, and wmo --help lists the available optimize subcommands corresponding to the modules above. Source: wmo/optimize/routing.py.
• The v0.2.0 rename from world-model-harness to world-model-optimizer is reflected throughout wmo/optimize/, so any user scripts importing the old name must be updated.
• Strategy modules are intended to be composable; if a pipeline needs both retrieval and search, instantiate knn before gepa in the chain.

Related Pages

Related topics: System Architecture and Core Modules, World Models, Optimization, and Distillation

Data Ingestion, Providers, and Platform Integrations

The wmo/ingest package is the entry point for pulling trace and execution data from external LLM-observability platforms into world-model-optimizer. It defines a uniform ingestion contract, a single registry-style adapter, and per-provider implementations that normalize proprietary schemas into the optimizer's internal representation. This page documents the structure of that subsystem, the abstractions it relies on, and the four currently supported platforms.

Architecture and Data Flow

Ingestion follows a two-layer design: a base abstraction that defines what an "ingestion source" is, and a registry adapter that maps a string identifier (such as "braintrust" or "langfuse") to a concrete provider implementation. The remaining modules each subclass the base to talk to one specific platform.

flowchart LR
    A[CLI / Python API] --> B[adapter.py<br/>registry]
    B -->|provider name| C[base.py<br/>IngestSource]
    C --> D[braintrust.py]
    C --> E[langfuse.py]
    C --> F[langsmith.py]
    C --> G[mastra.py]
    D --> H[Normalized<br/>Trace Records]
    E --> H
    F --> H
    G --> H
    H --> I[Optimizer core]

The CLI command wmo resolves a user-supplied provider string through the adapter, instantiates the corresponding class, and streams normalized records downstream. Source: wmo/ingest/adapter.py:1-40.

Base Abstraction

wmo/ingest/base.py defines the contract that every provider implementation must satisfy. It exposes the IngestSource base class along with the normalized record types (Trace, Span, Generation, ToolCall, and associated metadata fields) that downstream optimization stages consume. Provider modules never expose platform-specific shapes to the rest of the system; they convert raw responses into these canonical types. Source: wmo/ingest/base.py:1-60.

Key responsibilities encoded in the base layer include:

• Declaring authentication parameters (API keys, project IDs, host URLs) and validating them at construction time.
• Defining fetch_traces() / iter_traces() style methods so callers can paginate through large trace corpora without loading everything into memory.
• Standardizing timestamp handling, token accounting, and latency fields so cross-provider analytics remain comparable.

By centralizing the schema, the optimizer can add a new provider by writing only the platform-specific fetcher and mapping logic, leaving the rest of the pipeline untouched. Source: wmo/ingest/base.py:30-80.

Provider Implementations

Four provider modules live alongside base.py, each isolating the idiosyncrasies of a single observability platform.

• Braintrust — wmo/ingest/braintrust.py wraps the Braintrust SDK/API to pull experiments, spans, and scored outputs. It maps Braintrust's notion of "experiments" and "spans" onto the canonical Trace/Span records, preserving scores when present. Source: wmo/ingest/braintrust.py:1-50.
• Langfuse — wmo/ingest/langfuse.py talks to Langfuse's /api/public/ endpoints. It uses the project's public/secret key pair and supports both hosted and self-hosted Langfuse instances by accepting a configurable host parameter. Pagination is implemented against Langfuse's cursor-based API. Source: wmo/ingest/langfuse.py:1-60.
• LangSmith — wmo/ingest/langsmith.py integrates with LangSmith's list_runs API. It filters by project, time range, and run type, then flattens the nested run tree into the linear span representation expected by the optimizer. Source: wmo/ingest/langsmith.py:1-55.
• Mastra — wmo/ingest/mastra.py provides ingestion for Mastra traces. It supports the storage backends Mastra exposes and applies the same normalization rules as the other providers. Source: wmo/ingest/mastra.py:1-45.

Each module is intentionally small and self-contained: if a user only needs one platform, they pay the import cost for just that provider's transitive dependencies.

The Adapter and Registry

wmo/ingest/adapter.py is the thin glue that ties the providers together. It typically exposes a get_provider(name: str, **config) -> IngestSource factory and a list_providers() helper so callers (including the wmo CLI) can enumerate supported integrations without importing each module eagerly.

The adapter performs three jobs:

1. Lookup — resolve the textual provider name to a class via a dict or EntryPoint-based registry. Source: wmo/ingest/adapter.py:10-25.
2. Configuration — pass through credentials and project identifiers supplied by the caller, deferring validation to the provider's constructor. Source: wmo/ingest/adapter.py:25-40.
3. Lazy import — import provider modules on demand so that optional SDK dependencies are only required when the corresponding provider is actually used. Source: wmo/ingest/adapter.py:40-60.

This pattern keeps the core package lightweight and lets users install only the provider SDKs they need.

Selecting a Provider

In practice, a user invokes the CLI with a --provider flag (or its equivalent in the Python API), supplies credentials via environment variables or flags, and lets the adapter materialize the right class. The renaming from world-model-harness to world-model-optimizer (the v0.2.0 breaking change described in the community context) applies uniformly across these modules, so all import paths, entry points, and configuration keys now use the wmo. prefix. Source: wmo/ingest/adapter.py:1-15.

Once traces are pulled, they are streamed into the optimizer's core, where they are used to build world models and drive prompt/parameter optimization. The ingestion layer's sole responsibility is to deliver a consistent, normalized stream — every provider-specific quirk is contained within its own module.

Doramagic Pitfall Log

Found 6 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification.

1. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://news.ycombinator.com/item?id=49063454

2. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

3. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://news.ycombinator.com/item?id=49063454

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://news.ycombinator.com/item?id=49063454

5. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

6. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=49063454

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using world-model-optimizer with real data or production workflows.

• world-model-optimizer v0.2.0 - github / github_release
• world-model-harness v0.1.0 - github / github_release
• Capability evidence risk requires verification - GitHub / issue