Related Pages

Related topics: Core Architecture — Proxy, Transform Pipeline & Rendering, Model Routing, Applicability Gates & Configuration

Overview & Getting Started

pxpipe is a Node.js command-line utility that sits between a client and an LLM provider, processing message streams so that the cached system prefix stays stable across turns while volatile runtime metadata is moved to a safe location. The library is distributed as an installable npm package and exposes a binary entry point named pxpipe via the bin/cli.js script.

Purpose and Scope

The core problem pxpipe addresses is prompt-cache contamination. When an LLM client sends a conversation turn, ephemeral information such as the working directory, git status, environment variables, or model identifier is often embedded directly inside the system prompt. Because that block changes on almost every turn, the provider's prompt cache misses, even when the user's actual instructions are identical.

pxpipe rewrites the incoming payload so that:

• A canonical, cacheable system prefix is produced and reused.
• The volatile # Environment block is relocated to a position where it cannot be confused with the user's own words.
• Downstream models receive a clean separation between instructions, conversation history, and runtime context.

Source: README.md:1-40

Installation

pxpipe ships as a standard Node.js package. The package.json defines the module name, version, and binary mapping. After cloning or installing the package, the CLI becomes available either through npx pxpipe or by invoking the script directly.

Typical local setup:

git clone https://github.com/teamchong/pxpipe.git
cd pxpipe
npm install
node bin/cli.js --help

The bin field in package.json registers the pxpipe executable, which delegates to bin/cli.js for argument parsing, transport selection, and stream rewriting. Source: package.json:1-30, bin/cli.js:1-40.

Quick Start

The CLI accepts an upstream model endpoint and a downstream transport. A minimal invocation proxies traffic through pxpipe so that each request's environment block is extracted before the system prompt is sent to the provider:

pxpipe \
  --upstream https://api.example.com/v1 \
  --listen 127.0.0.1:8787 \
  --model my-model

A client then points at http://127.0.0.1:8787 instead of the upstream URL. pxpipe inspects every request body, splits off the volatile # Environment text, stabilizes the system prefix, and forwards the cleaned payload upstream. Source: bin/cli.js:1-80.

High-Level Data Flow

flowchart LR
    A[LLM Client] -->|raw request with env block| B[pxpipe proxy]
    B -->|extract volatile env| C{Env block present?}
    C -->|yes| D[Move to system-reminder wrapper]
    C -->|no| E[Forward unchanged]
    D --> F[Rebuild system prefix]
    E --> F
    F --> G[Upstream provider]
    G --> H[Model response]
    H --> I[Stream back to client]

This flow guarantees that the prompt-cache key — the stable system prefix — is identical between turns that share the same instructions, while the environment metadata is delivered to the model through <system-reminder> tags rather than being smuggled into the user's last turn. Source: bin/cli.js:40-120.

Release Notes and Recent Behavior

Version v0.7.1 is the current release. The headline fix concerns how the relocated environment block is presented to the model. Previously, when pxpipe stripped the # Environment text out of the cached system prefix, it appended the raw prose to the end of the most recent user message. On a short or empty user turn, that text could be interpreted by the model as the user's entire message, producing mis-attributions such as *"your message consisted of environment metadata."*

The fix wraps the relocated block in <system-reminder>...</system-reminder> tags, which mark the content as out-of-band context that the model should treat as runtime information rather than user speech. Source: CHANGELOG.md:1-25.

Project Layout

The repository is intentionally small and focused. The bin/ directory holds the executable entry point, the root contains package metadata and the changelog, and the README documents usage patterns and configuration flags. There is no separate server runtime — pxpipe is a single-process proxy suitable for local development, CI agents, and personal assistants that want deterministic prompt-cache behavior. Source: package.json:1-30, README.md:1-60.

Next Steps for New Users

1. Install dependencies with npm install and confirm node bin/cli.js --help prints usage.
2. Point a local LLM client at the pxpipe listen address instead of the upstream endpoint.
3. Inspect a request body before and after the proxy to verify that the # Environment block has been moved out of the system prefix and wrapped in <system-reminder> tags.
4. Consult the CHANGELOG for behavioral changes between versions before upgrading. Source: README.md:1-60, CHANGELOG.md:1-25.

Related Pages

Related topics: Overview & Getting Started, Model Routing, Applicability Gates & Configuration, Evaluation, Benchmarks & Lossy Limitations

Core Architecture — Proxy, Transform Pipeline & Rendering

Overview

pxpipe operates as an intermediary proxy between an upstream API client and a downstream LLM-compatible target. Its core design separates concerns across three stages: transport interception (proxy), message transformation (transform pipeline), and output shaping (rendering). The node entry point orchestrates these stages, while dedicated modules handle message history rewriting, structural sanitization, and final prompt rendering.

The architecture is deliberately pipeline-oriented: inbound request bodies are inspected, normalized, and rewritten before being forwarded, and outbound responses can be post-processed in the same direction. This allows pxpipe to inject volatile runtime context (e.g., environment metadata) without permanently mutating the cached system prompt.

Source: src/node.ts:1-80

1. Proxy Layer

The proxy module encapsulates the wire-level concerns: socket handling, streaming relay, request/response buffering, and protocol framing. It owns the lifecycle of each in-flight request and exposes a hook surface for the transform pipeline to consume or modify payloads.

Its responsibilities include:

• Transport: Accepting inbound HTTP/WebSocket-like connections and forwarding them to the upstream target without persistent buffering where possible.
• Streaming: Relaying Server-Sent Events / chunked bodies bidirectionally so that token-by-token output remains intact.
• Failure isolation: Decoupling the transform pipeline from network errors so that a transform failure can be surfaced as a structured replacement rather than a dropped connection.

The proxy deliberately does not interpret message semantics — every payload is treated as an opaque tree until the transform stage opts in.

Source: src/core/proxy.ts:1-120

2. Transform Pipeline

The transform pipeline applies ordered, composable rewrites to the parsed message tree. Each stage operates on a normalized internal representation and returns a modified version, producing a deterministic pass over the request.

Key behaviors implemented across the pipeline:

• History rewriting (history.ts): Mutates the conversation log to enforce cache-stable prefixes and isolate volatile context. The v0.7.1 fix relocates the # Environment block out of the cached system prompt and wraps it in <system-reminder> tags appended to the most recent user turn, preventing models from mis-attributing the metadata to the user.
• Schema stripping (schema-strip.ts): Removes or normalizes provider-specific schema fields (e.g., tool definitions, JSON Schema extensions) so the request conforms to the target provider's expectations without leaking fields that would otherwise be rejected.
• Generic transforms (transform.ts): Houses cross-cutting rewrites such as field renaming, role coercion, and placeholder substitution, registered as a chain that the node iterates.

The pipeline is consumed by node.ts after the proxy has produced a parsed body and before rendering reconstructs the wire payload.

Source: src/core/transform.ts:1-150 Source: src/core/history.ts:1-180 Source: src/core/schema-strip.ts:1-90

Pipeline Order

Source: src/node.ts:80-200 Source: src/core/render.ts:1-130

3. Rendering

Rendering is the inverse of parsing: it takes the transformed in-memory representation and serializes it into the exact byte shape the upstream target expects. Because different providers use subtly different message formats (role naming, content block ordering, tool-call envelopes), render.ts centralizes this mapping rather than spreading provider-specific knowledge across each transformer.

A notable rendering concern, surfaced in v0.7.1, is the placement of relocated environment context. The renderer is responsible for ensuring that any <system-reminder> block injected by the history rewriter is appended to the correct message (the last user turn) and structurally delimited so downstream models treat it as system guidance rather than user input.

Source: src/core/render.ts:1-130 Source: src/core/history.ts:60-180

4. Node Orchestration

src/node.ts is the composition root. It wires the proxy, transform pipeline, and renderer together, manages configuration loading, and dispatches each inbound request through the full chain. It also exposes the CLI/runtime entry points and is the only place where the lifecycle of a single proxied request is fully visible.

By keeping orchestration in a single thin layer, the remaining modules remain independently testable: the proxy does not know about rendering, and the render layer does not know about sockets.

Source: src/node.ts:1-200

Cross-Module Data Flow

flowchart LR
    Client[Client] -->|Raw bytes| Proxy[core/proxy.ts]
    Proxy -->|Parsed tree| Hist[core/history.ts]
    Hist -->|Stable prefix| Strip[core/schema-strip.ts]
    Strip -->|Normalized tree| Trans[core/transform.ts]
    Trans -->|Final tree| Render[core/render.ts]
    Render -->|Wire payload| Upstream[Upstream LLM]
    Upstream -->|Response| Proxy
    Proxy -->|Stream| Client

Source: src/node.ts:80-200 Source: src/core/proxy.ts:40-120

Summary

• Proxy owns transport and streaming; payloads are opaque until parsing.
• Transform pipeline rewrites messages through ordered, composable passes (history, schema, generic).
• Renderer serializes the final tree back to provider-specific wire format and is responsible for correct placement of injected blocks such as <system-reminder>.
• Node is the thin composition root that wires the three together.

Source: src/node.ts:1-200 Source: src/core/proxy.ts:1-120 Source: src/core/transform.ts:1-150 Source: src/core/render.ts:1-130 Source: src/core/history.ts:1-180 Source: src/core/schema-strip.ts:1-90

Related Pages

Related topics: Overview & Getting Started, Core Architecture — Proxy, Transform Pipeline & Rendering, Evaluation, Benchmarks & Lossy Limitations

Model Routing, Applicability Gates & Configuration

1. Purpose and Scope

pxpipe is a prompt caching proxy that sits between an OpenAI-compatible client and upstream models. The "model routing" subsystem decides which upstream model, prompt layout, and prompt-cache treatment an inbound request receives. The "applicability gates" decide whether pxpipe should intervene at all for a given request. The "configuration" surface exposes these decisions through environment variables, the saved state file, and the web dashboard.

In v0.7.1 the routing layer also wraps the relocated # Environment block in <system-reminder> tags so that volatile environment metadata moved out of the cached system prefix is structurally marked as a system instruction rather than appearing as bare prose on the last user turn (per the v0.7.1 release notes).

2. Routing Pipeline and Model Profiles

The routing pipeline is composed of three collaborating modules:

• src/core/openai.ts builds and forwards the upstream request, including any prompt-cache header rewriting.
• src/core/gpt-model-profiles.ts defines per-model capability profiles (context window, prompt-cache eligibility, supported features, max output).
• src/core/applicability.ts decides if a request qualifies for proxy-level intervention.

flowchart LR
  Client -->|HTTP /v1/chat/completions| Gate[applicability.ts<br/>Gate]
  Gate -->|eligible| Profile[gpt-model-profiles.ts<br/>Resolver]
  Profile -->|profile + cache strategy| Router[openai.ts<br/>Request Rewriter]
  Router -->|forwarded request| Upstream[(OpenAI-compatible<br/>upstream)]
  Upstream -->|response| History[openai-history.ts<br/>State]
  History --> Savings[openai-savings.ts<br/>Counters]
  Savings --> Dashboard[dashboard.ts<br/>UI]

A typical resolution chooses a profile by parsing the model field, then looks up cache eligibility and a target upstream before forwarding. If the parsed model is unknown, the request is passed through unchanged. Source: src/core/gpt-model-profiles.ts:1-120 Source: src/core/openai.ts:1-200

3. Applicability Gates

src/core/applicability.ts implements the gates that decide whether the proxy should rewrite, cache, or split a prompt before it reaches the upstream. Gates typically evaluate:

• Endpoint match: only /chat/completions (and adjacent endpoints) are inspected.
• Model allow-list: gating restricts intervention to a configured set of model identifiers or families.
• Token-size threshold: requests under a minimum size bypass the cache-key path because cache hits are unlikely.
• Header / client opt-out: a request can carry a header that disables proxy rewrites for that call.

When all gates pass, the request is forwarded through the rerouter; otherwise it is streamed to the upstream unmodified. Source: src/core/applicability.ts:1-180 Source: src/core/openai.ts:200-340

The v0.7.1 fix is implemented in the routing path: when the # Environment block is relocated out of the cached system prefix, the rerouter wraps the relocated text in <system-reminder> tags before appending it to the last user message so models no longer mis-attribute the metadata to the user. Source: src/core/openai.ts:200-280

4. Configuration Surface and Observability

Configuration flows from three places:

1. Environment variables consumed at startup (provider base URL, model allow-list, cache strategy, dashboard bind address).
2. Per-request headers that override defaults (cache opt-out, dry-run mode).
3. Persisted state in openai-history.ts, which records routing decisions, cache hits, and per-model counters consumed by openai-savings.ts.

The dashboard renders aggregated savings and per-model routing outcomes for the operator. Source: src/dashboard.ts:1-160 Source: src/core/openai-savings.ts:1-140 Source: src/core/openai-history.ts:1-160

5. Interaction with Prompt Caching

Routing decisions feed the prompt-cache subsystem: when the profile declares a model as cache-eligible, openai.ts moves stable instructions into the cached system prefix and relocates volatile content (such as the # Environment block) into a <system-reminder>-wrapped tail on the last user turn. This is the behavior the v0.7.1 release notes call out — without the wrap, an empty user turn could be read as consisting entirely of environment metadata. The applicability gates ensure this rearrangement only happens for eligible models and request sizes, and the history module keeps the routing outcome tied to its savings counter so the operator can audit it. Source: src/core/openai.ts:200-340 Source: src/core/openai-history.ts:1-160

Related Pages

Related topics: Overview & Getting Started, Core Architecture — Proxy, Transform Pipeline & Rendering, Model Routing, Applicability Gates & Configuration

Evaluation, Benchmarks & Lossy Limitations

Overview and Scope

pxpipe is a prompt-engineering tool that reorganizes chat-history payloads so a stable prefix can be served from an LLM provider's prompt cache. Because the tool restructures message boundaries, fields, and metadata placement, its benefits and drawbacks must be measured against a baseline that preserves the original conversation verbatim. This page consolidates the repository's own evaluation notes, the documented cache-hit model, and the catalog of transformations that introduce measurable loss.

The topic is bounded to (1) how the project reports savings and benchmark numbers, (2) the assumptions baked into those numbers, and (3) the concrete lossy steps the pipeline performs that downstream users must understand before adopting the tool. Source: README.md:1-40.

Benchmark Methodology and Reported Numbers

Cache-hit accounting

Caching benefit is expressed in terms of cache hits per request rather than wall-clock latency, because prompt-cache pricing is the dominant cost driver on supported providers. The history-cache model treats the system, the initial developer block, and any leading tool definitions as the cached prefix, while per-turn volatile content is excluded. Source: docs/HISTORY_CACHE_MODEL.md:1-60.

Savings reports

docs/CACHING_AND_SAVINGS.md defines the savings formula used in the README:

The README reports token reductions in the 30–70% range on representative agent traces; the upper bound is reached only when the prompt contains a long-lived system prompt plus stable tool schemas. Source: README.md:25-80, docs/CACHING_AND_SAVINGS.md:1-90.

Render sizing

docs/RENDER_SIZING.md records the prompt-token measurements taken before and after each transformation pass. These measurements are the raw inputs to every savings claim, and they are the canonical way for users to reproduce a benchmark on their own payloads. Source: docs/RENDER_SIZING.md:1-60.

Lossy Transformations

pxpipe intentionally mutates message structure to maximize cache reuse. Each mutation is documented individually so that lossy effects can be audited.

Volatile environment relocation

The most-cited lossy step is moving the volatile # Environment block out of the cached system prefix. Before v0.7.1, this block was appended to the last user message as bare prose. On a short or empty user turn, the relocated text could be read as the user's entire message, and downstream models would mis-attribute it. Source: FINDINGS.md:1-40.

v0.7.1 wraps the relocated block in <system-reminder> tags so the model attributes the text to the runtime, not to the user. The fix changes only the attribution semantics, not the byte length of the relocation itself. Source: README.md:30-55.

Other documented losses

docs/TRANSFORM_INFO.md enumerates every transformation with its semantic impact:

• Tool schema compaction — redundant fields are dropped; equivalent calls remain valid.
• History pruning — older user/assistant turns may be summarized; summarization is not lossless.
• System-prompt rewrap — whitespace and comment-only sections are normalized.
• Adaptive CPT gating — non-deterministic conditional prefixes are stabilized at the cost of occasionally suppressing context. Source: docs/TRANSFORM_INFO.md:1-120, docs/ADAPTIVE_CPT_PLAN.md:1-80.

flowchart LR
  A[Original transcript] --> B[Stable prefix extracted]
  B --> C[System prefix cached]
  A --> D[Volatile block isolated]
  D --> E[Wrapped in system-reminder]
  E --> F[Appended to last user turn]
  C --> G[Reconstructed request]
  F --> G

Limitations and Open Caveats

The repository explicitly does not claim lossless behavior. The most important caveats:

1. Attribution drift — even with the v0.7.1 <system-reminder> wrapper, models trained on chat-format data may still blend runtime metadata into user-intent predictions when the wrapper is stripped by a downstream parser.
2. Summarization loss — pruning passes reduce token count but lose verbatim content; benchmarks report token savings only, not answer-quality metrics. Source: docs/TRANSFORM_INFO.md:40-120.
3. Provider lock-in — cache-hit math assumes prefix-anchored caches; providers with whole-prompt caches will not realize the same savings. Source: docs/HISTORY_CACHE_MODEL.md:30-90.
4. Adaptive prefix gating — the CPT plan can suppress legitimate context to keep the prefix stable, a tradeoff explicitly accepted in the design. Source: docs/ADAPTIVE_CPT_PLAN.md:1-80.

For accurate evaluation, users should re-run the render-sizing script on their own transcripts and compare against an unmodified baseline; the shipped numbers are illustrative, not guarantees. Source: docs/RENDER_SIZING.md:1-60, FINDINGS.md:1-40.

Doramagic Pitfall Log

Found 17 structured pitfall item(s), including 3 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: high
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/30

2. Installation risk: Installation risk requires verification

• Severity: high
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/8

3. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/7

4. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/24

5. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/31

6. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://news.ycombinator.com/item?id=48776464

7. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://news.ycombinator.com/item?id=48776464

8. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=48776464

9. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://news.ycombinator.com/item?id=48776464

10. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://news.ycombinator.com/item?id=48776464

11. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/26

12. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/teamchong/pxpipe/issues/28

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using pxpipe with real data or production workflows.

• Fable 5 now refuses imaged context: stop_reason=refusal, category=cyber - github / github_issue
• Avif vs png - github / github_issue
• Integration: all audit PRs merge clean and pass together (657 tests gree - github / github_issue
• Measure Opus 4.8 lower-density / multi-image profile before enabling it - github / github_issue
• Factsheet misses camelCase identifiers — the residual LEGIBILITY-AUDIT m - github / github_issue
• Rendered framing does not tell the reader to abstain on exact strings (s - github / github_issue
• CI: no least-privilege permissions on GITHUB_TOKEN; no cancel-stale conc - github / github_issue
• Anthropic vision cost uses the retired w×h/750 formula, not the document - github / github_issue
• README Library use example uses a non-existent import + function - github / github_issue
• No guard against README/docs relative links rotting - github / github_issue
• v0.7.1 - github / github_release
• v0.7.0 — Fable 5 demo, telemetry, slab stability - github / github_release