# https://github.com/andrii-su/datoon Project Manual

Generated at: 2026-07-07 23:44:23 UTC

## Table of Contents

- [Project Overview and Conversion Decision Engine](#page-1)
- [Multi-Format Readers and Data Ingestion](#page-2)
- [Token Estimation and Encoding Configuration](#page-3)
- [CLI, MCP Server, and Agent Plugin Surfaces](#page-4)

<a id='page-1'></a>

## Project Overview and Conversion Decision Engine

### Related Pages

Related topics: [Multi-Format Readers and Data Ingestion](#page-2), [Token Estimation and Encoding Configuration](#page-3), [CLI, MCP Server, and Agent Plugin Surfaces](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/datoon/__init__.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/__init__.py)
- [src/datoon/analyzer.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/analyzer.py)
- [src/datoon/converter.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/converter.py)
- [src/datoon/models.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/models.py)
- [src/datoon/errors.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/errors.py)
- [src/datoon/readers/__init__.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/__init__.py)
- [src/datoon/cli.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/cli.py)
- [README.md](https://github.com/andrii-su/datoon/blob/main/README.md)
</details>

# Project Overview and Conversion Decision Engine

## Purpose and Scope

Datoon is a data preparation tool oriented toward LLM training and evaluation workloads. Its role is to ingest heterogeneous source files (CSV, JSONL, YAML, XML, Excel, Parquet, Avro, ORC, and Apple Numbers) and emit a single, model-friendly target artifact, with associated token accounting and quality signals. The CLI is the primary entry point and is exposed under `src/datoon/cli.py`, while the core orchestration lives inside the `converter` and `analyzer` modules (Source: [README.md](); Source: [src/datoon/cli.py]()). The project's distribution surface also includes an MCP server bundle, so the same decision engine powers both interactive CLI runs and agent-driven invocations.

The codebase is deliberately split into three concentric rings: a thin presentation layer (`cli.py`, MCP wiring), a pure decision layer (`analyzer.py`, `converter.py`, `models.py`), and pluggable I/O adapters under `readers/`. This separation lets the format-detection and target-selection logic remain testable independently of file parsing (Source: [src/datoon/analyzer.py](); Source: [src/datoon/converter.py]()).

## Architecture and Module Layout

The conversion pipeline traverses the repository in a fixed order: CLI argument parsing → reader dispatch → analyzer scoring → converter emission. Each stage owns its own error type so callers can branch on the failure mode.

| Stage | Module | Responsibility |
|-------|--------|----------------|
| Entry | `src/datoon/cli.py` | argparse surface, exit-code mapping (Source: [src/datoon/cli.py]()) |
| Dispatcher | `src/datoon/readers/__init__.py` | Selects a reader by extension and routes `ValueError`s upward (Source: [src/datoon/readers/__init__.py]()) |
| Readers | `readers/{csv,jsonl,yaml,xml,…}.py` | Normalize format-specific quirks into a list-of-objects (Source: [issue #43]()) |
| Analyzer | `src/datoon/analyzer.py` | Computes per-format suitability, token counts, schema heuristics |
| Converter | `src/datoon/converter.py` | Serializes the chosen target, owns token-encoder lifecycle |
| Errors | `src/datoon/errors.py` | Defines `DatoonError`; sits alongside built-in `ValueError`s (Source: [issue #41]()) |
| Models | `src/datoon/models.py` | Dataclasses describing normalized records and conversion reports |

The decision engine itself is the analyzer/converter pair. The analyzer answers "given this input, what target format is safest and most efficient?" and the converter answers "given that target, how do we serialize and count tokens?". The split lets token-encoding changes (such as the v1.9.0 switch from `cl100k_base` to `o200k_base`) be made without touching scoring rules (Source: [release v1.9.0](); Source: [src/datoon/converter.py]()).

## Conversion Decision Engine

### How the target is chosen

The analyzer inspects the normalized records produced by the reader layer and weighs factors such as nesting depth, scalar density, presence of metadata fields, and approximate token footprint. Heuristics steer toward JSONL when records are flat and toward richer formats when the input carries nested structure that would lose fidelity under JSONL flattening.

```mermaid
flowchart LR
    A[Raw file] --> B[Reader dispatcher]
    B --> C[Normalized records]
    C --> D[Analyzer scoring]
    D --> E{Target format}
    E -->|flat| F[JSONL]
    E -->|nested| G[Parquet / Avro]
    E -->|spreadsheet-like| H[CSV / Excel]
    F --> I[Converter + token encoder]
    G --> I
    H --> I
    I --> J[Output artifact + report]
```

The dispatcher raises `ValueError` for unparseable inputs in formats that don't define a richer error, while `readers/jsonl.py` raises `DatoonError` for the same condition. This asymmetry is acknowledged as technical debt and is the motivation behind issue #41, which proposes consolidating around a single taxonomy (Source: [issue #41](); Source: [src/datoon/readers/__init__.py]()).

### Token accounting

`converter._load_token_encoder` instantiates a `tiktoken` encoding used both for budget reporting and for any prompt-template stitching the tool performs. Prior to v1.9.0 the function hard-coded `cl100k_base`, which mis-estimated token counts for GPT-4o/o-series (which expect `o200k_base`) and for Claude-family models (which use Anthropic's tokenizer). Version 1.9.0 makes the encoding configurable and switches the default to `o200k_base`, resolving issue #42 (Source: [issue #42](); Source: [release v1.9.0](); Source: [src/datoon/converter.py]()).

### Safety checks

The XML reader refuses DTD/DOCTYPE declarations to block entity-expansion denial of service (v1.7.4), and the converter rejects non-finite JSON constants such as `NaN` and `Infinity` rather than silently passing them through (v1.7.3). Both safeguards sit at the boundary between the reader and the converter, which means a future reader that fails to validate list-item shape (the open gap in `readers/yaml.py::_normalize` per issue #43) is the kind of regression these boundary checks are designed to surface (Source: [release v1.7.3](); Source: [release v1.7.4](); Source: [issue #43]()).

## Evolution and Community Context

The release cadence has been a sequence of small, well-scoped fixes rather than large refactors. v1.6.0 introduced the multi-format readers, v1.7.x hardened the readers and converter, v1.8.0 added `--sheet` and `--table` flags for Excel and Numbers inputs, and v1.9.x made the token encoder configurable (Source: [release v1.6.0](); Source: [release v1.8.0](); Source: [release v1.9.0](); Source: [release v1.9.1]()). The current open issues cluster around hygiene: a deprecated `typing.Sequence` import in `cli.py` (Source: [issue #44]()), a missing YAML list-item guard (Source: [issue #43]()), and the unification of reader error types (Source: [issue #41]()). None of these change the decision engine's public contract; they all tighten the boundaries around it.

For new contributors, the practical takeaway is that any change touching output selection should be made in `analyzer.py` with corresponding tests, while any change touching serialization or token counting belongs in `converter.py`. Keeping that boundary intact is what allows the tool to evolve its reader set and its tokenizer choice independently.

---

<a id='page-2'></a>

## Multi-Format Readers and Data Ingestion

### Related Pages

Related topics: [Project Overview and Conversion Decision Engine](#page-1), [Token Estimation and Encoding Configuration](#page-3), [CLI, MCP Server, and Agent Plugin Surfaces](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/datoon/readers/__init__.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/__init__.py)
- [src/datoon/readers/_coerce.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/_coerce.py)
- [src/datoon/readers/_tabular.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/_tabular.py)
- [src/datoon/readers/csv.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/csv.py)
- [src/datoon/readers/jsonl.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/jsonl.py)
- [src/datoon/readers/yaml.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/yaml.py)
- [src/datoon/readers/xml.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/readers/xml.py)
- [src/datoon/errors.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/errors.py)
- [src/datoon/cli.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/cli.py)
</details>

# Multi-Format Readers and Data Ingestion

The `datoon.readers` subsystem is the entry point for turning heterogeneous files on disk into a uniform Python list of records that downstream stages (tokenization, conversion, MCP exposure) can consume. It was introduced as the headline feature of v1.6.0 and expanded with spreadsheet/sheet awareness in v1.8.0 (`--sheet`, `--table`). Source: [src/datoon/readers/__init__.py:1-80]()

## Purpose and Scope

Read responsibility is split into two layers:

- A format-detection dispatcher in `__init__.py` maps file extensions and explicit `--input-format` overrides to the appropriate reader.
- Format-specific modules under `readers/` own I/O quirks for CSV, JSONL, YAML, XML, Excel, Parquet, Avro, ORC, and Numbers.

All readers return a single shape: a `list[dict[str, Any]]` (or, for YAML where the source is naturally list-shaped, the bare list and let coercion normalize later). Source: [src/datoon/readers/yaml.py:1-40]()

## Internal Architecture

```mermaid
flowchart LR
    A[CLI / MCP entrypoint] --> B[readers/__init__.py<br/>dispatcher]
    B -->|extension or --input-format| C[csv.py]
    B --> D[jsonl.py]
    B --> E[yaml.py]
    B --> F[xml.py]
    B --> G[excel/numbers readers]
    C --> H[readers/_coerce.py<br/>scalar coercion]
    C --> I[readers/_tabular.py<br/>header + row utils]
    D --> H
    E --> H
    F --> H
    H --> J[(list[dict] ready for converter)]
    I --> J
```

The dashed lines emphasize that `_coerce.py` and `_tabular.py` are shared helpers rather than readers themselves. Every format-specific module imports coercion helpers to keep semantic types intact. This was reinforced in v1.7.2 ("preserve semantic values during scalar and row coercion", #36). Source: [src/datoon/readers/_coerce.py:1-60](), [src/datoon/readers/_tabular.py:1-80]()

## Format Dispatch and CLI Integration

The dispatcher inspects the file suffix and falls back to a manual override. The CLI wires this together with options such as `--sheet` and `--table` for spreadsheet formats, added in v1.8.0 (issue #39). The CLI catches a broad `(OSError, ValueError, DatoonError, ...)` tuple because readers raise inconsistent exception classes today. Source: [src/datoon/cli.py:1-200]()

### Reader → Error Type Mapping (current state)

| Format / module | Malformed input raises | Notes |
|---|---|---|
| `readers/jsonl.py` | `DatoonError` | Consistent with project error type. |
| `readers/xml.py` | `ValueError` | Refuses DTD/DOCTYPE per v1.7.4 (#38). |
| `readers/yaml.py` | `ValueError` | `_normalize` does not validate list-item types (#43). |
| `readers/__init__.py` (dispatcher) | `ValueError` | Wraps lookup/unsupported-format failures. |

This table reflects issue #41 ("Unify reader error taxonomy") and motivates the open discussion of standardizing on `DatoonError` everywhere. Source: [src/datoon/errors.py:1-40]()

## Format-Specific Behavior

**CSV** leans on the shared tabular helpers for header sniffing and row normalization, then defers scalar coercion to `_coerce.py`. Source: [src/datoon/readers/csv.py:1-120]()

**JSONL** is strict line-delimited JSON; it raises `DatoonError` on parse failure rather than letting `json.JSONDecodeError` leak, which lets the dispatcher treat all reader failures uniformly once #41 is resolved. Source: [src/datoon/readers/jsonl.py:1-100]()

**YAML** accepts three input shapes via `_normalize`: a top-level list (returned as-is), a single-key dict whose value is a list (unwrapped), or other shapes (rejected). Open issue #43 notes that the list path does not currently verify that each element is a mapping, so a YAML file like `- a\n- b` slips through and later fails in coercion. Source: [src/datoon/readers/yaml.py:1-60]()

**XML** uses `xml.etree` with `defusedxml`-equivalent hardening: v1.7.4 added an explicit refusal of `<!DOCTYPE>` declarations to block entity-expansion DoS. Source: [src/datoon/readers/xml.py:1-140]()

**Spreadsheet readers** (Excel/Numbers) accept `--sheet NAME` and `--table NAME` through the CLI dispatcher; the reader layer interprets those names to pick a worksheet or a named table region before delegating to `_tabular.py` for header detection. Source: [src/datoon/readers/_tabular.py:1-120](), [src/datoon/cli.py:1-200]()

## Known Limitations and Open Issues

- **Inconsistent error taxonomy (#41):** mixing `ValueError` and `DatoonError` forces callers to catch broad tuples.
- **YAML list-item validation (#43):** `_normalize` returns the parsed list without checking that items are dicts.
- **Scalars in coercion (#36, fixed in v1.7.2):** previously, scalar coercion could stringify numbers/bools; now semantics are preserved before downstream stages.
- **XML DoS hardening (#38, fixed in v1.7.4):** DOCTYPE declarations are rejected outright.

These are tracked in the repository issues and inform near-term refactors of the reader layer. Source: [src/datoon/readers/__init__.py:1-80](), [src/datoon/readers/yaml.py:1-60](), [src/datoon/readers/xml.py:1-140]()

---

<a id='page-3'></a>

## Token Estimation and Encoding Configuration

### Related Pages

Related topics: [Project Overview and Conversion Decision Engine](#page-1), [Multi-Format Readers and Data Ingestion](#page-2)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/datoon/converter.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/converter.py)
- [src/datoon/models.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/models.py)
- [src/datoon/cli.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/cli.py)
- [src/datoon/__init__.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/__init__.py)
- [pyproject.toml](https://github.com/andrii-su/datoon/blob/main/pyproject.toml)
- [README.md](https://github.com/andrii-su/datoon/blob/main/README.md)
- [CHANGELOG.md](https://github.com/andrii-su/datoon/blob/main/CHANGELOG.md)
</details>

# Token Estimation and Encoding Configuration

## Overview

Datoon estimates token counts for records as they flow through the conversion pipeline. Because datoon's primary use case is preparing data for large language model (LLM) workloads, token counts are reported alongside record counts so that users can budget context windows, plan cost, and verify that a dataset will fit a target model's limit. Token estimation is implemented through `tiktoken`, OpenAI's open-source BPE tokenizer library, and is integrated into the central converter module that powers every format reader and writer.

Source: [src/datoon/converter.py:1-40]()
Source: [README.md:1-60]()

The encoder is loaded once per process and then reused for every record, so the choice of encoding has a direct, predictable effect on every total that appears in conversion output.

## The `_load_token_encoder` Helper

The encoder is constructed by `_load_token_encoder` inside the converter module. As originally shipped, the function unconditionally called `tiktoken.get_encoding("cl100k_base")`, the BPE encoding used by GPT-3.5 and GPT-4 era models.

Source: [src/datoon/converter.py:80-120]()

The `cl100k_base` choice was a reasonable default while datoon's user base was predominantly working with OpenAI models, but it produces token counts that diverge from the encodings actually used by newer or non-OpenAI target models. Issue #42 documents this mismatch in detail: GPT-4o and the o-series use `o200k_base`, and Claude uses its own tokenizer that is not exposed via `tiktoken` at all.

Source: [README.md:60-140]()

Because the encoder is cached and reused rather than reconstructed per row, a single process-wide configuration switch is sufficient to redirect every subsequent estimate, which is what made the v1.9.0 patch low-risk.

## Configurable Encoding in v1.9.0

Release v1.9.0 (PR #46, commit `8483eb7`) made the token encoding configurable and changed the default to `o200k_base`. The patch preserves the public surface of `_load_token_encoder` while extending it to accept an encoding name, and adds a fall-back path so that missing or invalid names surface a clear error rather than silently reverting to `cl100k_base`.

Source: [src/datoon/converter.py:80-160]()
Source: [CHANGELOG.md:1-40]()

The supported set of encodings is whatever `tiktoken` itself exposes at the installed version. The version metadata that drives this default switch is recorded in `pyproject.toml`.

Source: [pyproject.toml:1-80]()

| datoon version | Default encoding | Configurable | Source |
|----------------|------------------|--------------|--------|
| < v1.9.0       | `cl100k_base`    | No           | Issue #42, CHANGELOG v1.8.0 entry |
| v1.9.0+        | `o200k_base`     | Yes          | PR #46, CHANGELOG v1.9.0 entry |

Models whose tokenizers are not exposed through `tiktoken` — notably the Claude family — will always show approximated counts even after this change, because `tiktoken` is the sole encoder backend.

## CLI Integration

The encoding choice is exposed through the CLI so that operators do not need to touch Python code to align estimates with their target model. The CLI option is parsed in `cli.py` and threaded into the converter's call to `_load_token_encoder`.

Source: [src/datoon/cli.py:1-120]()

The CLI also handles the related error taxonomy: conversions raise a mix of `OSError`, `ValueError`, and the project-defined `DatoonError`, and the CLI entry point catches the union so that encoding errors, format errors, and I/O errors all surface as a single user-friendly failure. Issue #41 tracks a longer-term cleanup to unify these error types across the reader modules (`readers/jsonl.py`, `readers/xml.py`, `readers/yaml.py`, and the dispatcher in `readers/__init__.py`).

Source: [src/datoon/cli.py:120-220]()
Source: [src/datoon/__init__.py:1-40]()

## Known Limitations and Future Work

Even with the v1.9.0 default change, datoon still ships a `tiktoken`-only encoder loader. The most likely direction for closing the Claude-tokenizer gap is a plug-in or adapter interface similar in spirit to the format-reader plug-ins added in v1.6.0 (commit `ca8de4e`).

Source: [CHANGELOG.md:40-120]()

The default of `o200k_base` is also a moving target: future model families may introduce their own encodings, and datoon will need to revisit both the default and the configurability surface. Until then, users preparing data for non-OpenAI models should treat the reported token counts as a budget estimate rather than a precise bill. Issue #42 remains the canonical reference for this caveat and any follow-up design discussion.

Source: [src/datoon/converter.py:120-200]()

## Summary

Token estimation in datoon is a single helper, `_load_token_encoder`, wired through the converter and exposed via the CLI. The v1.9.0 release turned a hard-coded `cl100k_base` into a configurable parameter with an `o200k_base` default, addressing the most visible mismatch called out in issue #42. The remaining limitation is the `tiktoken`-only backend, which the project has not yet abstracted behind a plug-in interface.

---

<a id='page-4'></a>

## CLI, MCP Server, and Agent Plugin Surfaces

### Related Pages

Related topics: [Project Overview and Conversion Decision Engine](#page-1), [Multi-Format Readers and Data Ingestion](#page-2)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/datoon/cli.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/cli.py)
- [src/datoon/mcp_server.py](https://github.com/andrii-su/datoon/blob/main/src/datoon/mcp_server.py)
- [.claude-plugin/marketplace.json](https://github.com/andrii-su/datoon/blob/main/.claude-plugin/marketplace.json)
- [.claude-plugin/plugin.json](https://github.com/andrii-su/datoon/blob/main/.claude-plugin/plugin.json)
- [.agents/plugins/marketplace.json](https://github.com/andrii-su/datoon/blob/main/.agents/plugins/marketplace.json)
- [plugins/datoon/.codex-plugin/plugin.json](https://github.com/andrii-su/datoon/blob/main/plugins/datoon/.codex-plugin/plugin.json)
</details>

# CLI, MCP Server, and Agent Plugin Surfaces

## Overview

datoon is an LLM-data toolkit that ships three external surfaces wrapping the same core converter and reader pipeline:

- A **CLI** invoked as `datoon` from a shell.
- An **MCP server** that exposes the same operations as tools over the Model Context Protocol.
- **Agent plugin manifests** for Claude (`.claude-plugin/`), generic agent hosts (`.agents/plugins/`), and Codex (`plugins/datoon/.codex-plugin/`).

All three surfaces converge on the same Python entry points in `src/datoon/`, so behavior, errors, and token accounting remain consistent regardless of how a user invokes datoon. Source: [src/datoon/cli.py:1-40]() Source: [src/datoon/mcp_server.py:1-30]()

## Command-Line Interface

The CLI is defined in `src/datoon/cli.py` and is the primary human-facing surface. It parses arguments, dispatches to format readers, runs the converter, and prints token estimates or converted payloads to stdout.

### Argument Surface

The CLI exposes flags for input selection, format hints, sheet/table selection for tabular inputs, and token-encoding configuration. Version 1.8.0 added `--sheet` and `--table` flags for Excel and Numbers input, letting users target a specific worksheet or named table rather than always reading the first sheet. Source: [src/datoon/cli.py:80-140]()

### Error Handling

The CLI wraps reader and converter exceptions in a broad tuple to keep the user-facing exit path stable:

```python
try:
    ...
except (OSError, ValueError, DatoonError) as exc:
    ...
```

This wide catch is a direct consequence of the inconsistent reader error taxonomy documented in community issue #41, where `jsonl` raises `DatoonError` while `xml`/`yaml` raise `ValueError`. Source: [src/datoon/cli.py:160-210]() Source: [https://github.com/andrii-su/datoon/issues/41]()

### Token Encoding Configuration

The CLI accepts the token-encoding flag introduced in v1.9.0, defaulting to `o200k_base` (the GPT-4o / o-series tokenizer) instead of the legacy `cl100k_base`. The default lives in `converter._load_token_encoder`, but the CLI option lets users override per invocation. Source: [src/datoon/cli.py:60-110]()

### Known Quality Issues

Issue #44 notes that `cli.py` imports `Sequence` from the deprecated `typing` alias; it should use `collections.abc.Sequence` (ruff UP035). The fix is mechanical and has no behavioral impact. Source: [src/datoon/cli.py:1-30]() Source: [https://github.com/andrii-su/datoon/issues/44]()

## MCP Server Surface

`src/datoon/mcp_server.py` runs an MCP server that advertises datoon's operations as tools consumable by MCP-aware agents and IDEs.

### Initialization Handshake

Version 1.7.1 fixed a bug where the server's `initialize` response omitted the datoon version. After the fix, the handshake reports the running version so hosts can negotiate capabilities correctly. Source: [src/datoon/mcp_server.py:20-60]() Source: [https://github.com/andrii-su/datoon/releases/tag/v1.7.1]()

### Marketplace Distribution

Version 1.7.0 added metadata so datoon can be listed on the MCP Registry, Smithery, and Glama marketplaces. Users install datoon through these catalogs and run it as a local MCP process that agents connect to over stdio. Source: [src/datoon/mcp_server.py:60-120]() Source: [https://github.com/andrii-su/datoon/releases/tag/v1.7.0]()

### Tool Surface

The MCP server mirrors the CLI's argument surface — input path, format hint, sheet/table selector, target encoding — but exposes them as tool parameters. This keeps the mental model consistent between CLI and MCP users.

## Agent Plugin Surfaces

Agent plugin manifests describe how datoon integrates with specific agent runtimes. Each manifest follows the host's expected schema but points at the same underlying CLI/MCP entry points.

### Claude Plugin

The Claude plugin manifest at `.claude-plugin/plugin.json` declares plugin metadata, while `.claude-plugin/marketplace.json` publishes the plugin to Claude marketplaces. Source: [.claude-plugin/plugin.json:1-25]() Source: [.claude-plugin/marketplace.json:1-40]()

### Generic Agent Marketplace

`.agents/plugins/marketplace.json` provides a neutral marketplace listing so non-Claude agent hosts can discover and install datoon through a common entry point. Source: [.agents/plugins/marketplace.json:1-40]()

### Codex Plugin

The Codex integration lives at `plugins/datoon/.codex-plugin/plugin.json`, packaged inside the `plugins/datoon/` distribution so the hardened plugin distribution workflow (v1.5.0) can sign and ship it alongside the Python wheel. Source: [plugins/datoon/.codex-plugin/plugin.json:1-25]() Source: [https://github.com/andrii-su/datoon/releases/tag/v1.5.0]()

## Surface Relationships

| Surface | Entry file | Primary user | Transport |
|---|---|---|---|
| CLI | `src/datoon/cli.py` | Humans in a terminal | argv / stdout |
| MCP Server | `src/datoon/mcp_server.py` | MCP-aware agents and IDEs | stdio JSON-RPC |
| Claude plugin | `.claude-plugin/plugin.json` | Claude Code/Desktop users | marketplace install |
| Generic agent | `.agents/plugins/marketplace.json` | Other agent runtimes | marketplace install |
| Codex plugin | `plugins/datoon/.codex-plugin/plugin.json` | Codex users | bundled plugin |

All three runtime surfaces (CLI, MCP server) call the same readers and converter, so fixes like the v1.9.0 token-encoding change, the v1.7.4 XML DTD hardening, and the v1.7.2 scalar coercion fix apply uniformly. Source: [src/datoon/cli.py:1-40]() Source: [src/datoon/mcp_server.py:1-30]()

---

<!-- evidence_pipeline_checked: true -->
<!-- evidence_injected: true -->

---

## Pitfall Log

Project: andrii-su/datoon

Summary: Found 25 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Configuration risk - Configuration risk requires verification.

## 1. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.host_targets | https://github.com/andrii-su/datoon

## 2. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: Token estimate uses cl100k_base, not the tokenizer of target models
- User impact: Developers may misconfigure credentials, environment, or host setup: Token estimate uses cl100k_base, not the tokenizer of target models
- Evidence: failure_mode_cluster:github_issue | https://github.com/andrii-su/datoon/issues/42

## 3. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this configuration risk before relying on the project: v1.9.0
- User impact: Upgrade or migration may change expected behavior: v1.9.0
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.9.0

## 4. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/andrii-su/datoon/issues/41

## 5. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/andrii-su/datoon/issues/43

## 6. Capability evidence risk - Capability evidence risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://github.com/andrii-su/datoon

## 7. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Developers should check this migration risk before relying on the project: cli.py imports Sequence from typing (deprecated alias)
- User impact: Developers may hit a documented source-backed failure mode: cli.py imports Sequence from typing (deprecated alias)
- Evidence: failure_mode_cluster:github_issue | https://github.com/andrii-su/datoon/issues/44

## 8. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/andrii-su/datoon/issues/44

## 9. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/andrii-su/datoon

## 10. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://github.com/andrii-su/datoon

## 11. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://github.com/andrii-su/datoon

## 12. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/andrii-su/datoon/issues/42

## 13. Capability evidence risk - Capability evidence risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this capability risk before relying on the project: YAML reader does not validate that list items are objects
- User impact: Developers may hit a documented source-backed failure mode: YAML reader does not validate that list items are objects
- Evidence: failure_mode_cluster:github_issue | https://github.com/andrii-su/datoon/issues/43

## 14. Capability evidence risk - Capability evidence risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this conceptual risk before relying on the project: Unify reader error taxonomy (ValueError vs DatoonError)
- User impact: Developers may hit a documented source-backed failure mode: Unify reader error taxonomy (ValueError vs DatoonError)
- Evidence: failure_mode_cluster:github_issue | https://github.com/andrii-su/datoon/issues/41

## 15. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/andrii-su/datoon

## 16. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/andrii-su/datoon

## 17. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.5.0
- User impact: Upgrade or migration may change expected behavior: v1.5.0
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.5.0

## 18. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.6.0
- User impact: Upgrade or migration may change expected behavior: v1.6.0
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.6.0

## 19. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.7.0
- User impact: Upgrade or migration may change expected behavior: v1.7.0
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.7.0

## 20. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.7.1
- User impact: Upgrade or migration may change expected behavior: v1.7.1
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.7.1

## 21. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.7.2
- User impact: Upgrade or migration may change expected behavior: v1.7.2
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.7.2

## 22. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.7.3
- User impact: Upgrade or migration may change expected behavior: v1.7.3
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.7.3

## 23. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.7.4
- User impact: Upgrade or migration may change expected behavior: v1.7.4
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.7.4

## 24. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.8.0
- User impact: Upgrade or migration may change expected behavior: v1.8.0
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.8.0

## 25. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: Developers should check this maintenance risk before relying on the project: v1.9.1
- User impact: Upgrade or migration may change expected behavior: v1.9.1
- Evidence: failure_mode_cluster:github_release | https://github.com/andrii-su/datoon/releases/tag/v1.9.1

<!-- canonical_name: andrii-su/datoon; human_manual_source: deepwiki_human_wiki -->