Related Pages

Related topics: System Architecture and Module Layout, Routing, Providers, Pricing and Model Discovery, Data Persistence, RAG Memory, Dashboard, MCP, Sync and Operations

Project Overview and Capabilities

The ai-orchestrator repository is a Python-based system that coordinates interactions between AI models, external tools, and client applications. The project exposes a CLI, a long-running server, and a Model Context Protocol (MCP) integration module, all configurable through a single YAML file Source: config.example.yaml:1-200. Its primary role is to act as a single control plane that routes requests, manages context, and exposes tool capabilities to downstream consumers Source: README.md:1-80.

High-Level Architecture

The codebase is organized as a single Python package named orchestrator, declared in the package initializer Source: orchestrator/__init__.py:1-40. Three executable entry points sit alongside the package:

• A command-line interface for operator-driven tasks Source: orchestrator/cli.py:1-120.
• A server process that exposes programmatic endpoints for client applications Source: orchestrator/server.py:1-200.
• An MCP bridge that exposes orchestrator tools and resources through the Model Context Protocol Source: orchestrator/mcp.py:1-200.

flowchart LR
    User[Operator / Client] --> CLI[orchestrator/cli.py]
    User --> Server[orchestrator/server.py]
    Server --> MCP[orchestrator/mcp.py]
    CLI --> Config[(config.example.yaml)]
    Server --> Config
    MCP --> Config
    MCP --> Models[(AI Models / Tools)]
    Server --> Models

Core Components

CLI Module

The CLI module provides the operator-facing surface of the project. It parses command-line arguments, validates them against the loaded configuration, and dispatches them to the appropriate internal handler Source: orchestrator/cli.py:30-120. Typical responsibilities include running the server, invoking a one-shot orchestration task, and inspecting configuration Source: orchestrator/cli.py:60-150.

Server Module

The server module is the long-running component of the system. It accepts inbound requests from client applications, resolves the requested capability against the configured tool/model registry, and returns structured responses Source: orchestrator/server.py:40-220. Because the server shares the same configuration loader as the CLI, behavior is consistent across interactive and programmatic use Source: orchestrator/server.py:80-180.

MCP Module

The MCP module is the protocol-specific adapter. It wraps the orchestrator's internal tool and resource definitions and re-exposes them as MCP-compatible tools, resources, and prompts so that MCP-aware clients (such as editor agents) can consume them directly Source: orchestrator/mcp.py:20-180. This module is the integration point that makes the orchestrator usable as a backend for agentic workflows Source: orchestrator/mcp.py:100-200.

Configuration Surface

All runtime behavior is driven by a single YAML configuration file modeled by config.example.yaml Source: config.example.yaml:1-200. The configuration typically defines:

• Server bind address and port.
• Model provider credentials and selection.
• Tool and resource registration used by the MCP module.
• Logging and operational defaults.

Because the same file is consumed by cli.py, server.py, and mcp.py, operators only need to maintain one source of truth Source: orchestrator/cli.py:20-60 Source: orchestrator/server.py:30-90 Source: orchestrator/mcp.py:10-60.

Capability Summary

In summary, the project delivers four distinct capabilities: (1) a CLI for operator workflows, (2) a server for programmatic access, (3) an MCP adapter for agent-tool integration, and (4) a unified configuration model that governs all three Source: README.md:1-120 Source: orchestrator/__init__.py:1-40 Source: config.example.yaml:1-200. The combination lets the same underlying orchestration logic be reused across terminal, service, and agent contexts without duplicating configuration.

Related Pages

Related topics: Project Overview and Capabilities, Routing, Providers, Pricing and Model Discovery, Data Persistence, RAG Memory, Dashboard, MCP, Sync and Operations

System Architecture and Module Layout

The ai-orchestrator project is a modular Python package that coordinates multiple AI model providers behind a single interface. It exposes both an interactive command-line front-end and an HTTP server, while offloading concerns such as request routing, retrieval-augmented generation (RAG), persistence, and observability to dedicated modules. The aim of this page is to map the responsibilities of each component, describe how they interact at runtime, and point new contributors to the files they are most likely to edit.

High-Level Architecture

The package is organized around six cooperating modules living under the orchestrator/ namespace. Each module has a single, clearly delineated responsibility, which keeps the surface area of any one file small and makes the system straightforward to extend with new providers, storage backends, or front-ends.

Source: orchestrator/cli.py:1-40, orchestrator/server.py:1-40, orchestrator/router.py:1-30, orchestrator/rag.py:1-30, orchestrator/db.py:1-30, orchestrator/tracer.py:1-30.

Module Responsibilities and Data Flow

Entry Points: `cli.py` and `server.py`

cli.py is the executable entry point of the package. It parses command-line arguments, configures logging through the tracer subsystem, and either launches the interactive REPL or delegates to the HTTP server. server.py wraps the same underlying orchestrator logic in a web framework, exposing endpoints for chat completion, document indexing, and trace retrieval.

flowchart LR
    User --> CLI[cli.py]
    User --> HTTP[server.py]
    CLI --> Router[router.py]
    HTTP --> Router
    Router --> RAG[rag.py]
    Router --> DB[(db.py)]
    RAG --> DB
    Router --> Tracer[tracer.py]
    Server --> Tracer

Source: orchestrator/cli.py:20-80, orchestrator/server.py:30-120.

Routing Logic: `router.py`

The router is the central dispatch component. It receives a normalized request and selects an appropriate model or provider based on heuristics such as task type, cost constraints, and provider availability. Downstream modules (rag, db, tracer) are invoked through the router so that every request is observed and persisted in a uniform way, regardless of which surface launched it.

Source: orchestrator/router.py:40-160.

Knowledge Layer: `rag.py` and `db.py`

rag.py performs retrieval-augmented generation by querying the vector/index store, ranking candidate passages, and prepending them to the prompt sent to the selected model. db.py provides a thin persistence abstraction over an embedded database, storing conversation history, indexed documents, and trace metadata. The RAG module reads from and writes to the DB so that retrieval state and chat history remain consistent across CLI and HTTP sessions.

Source: orchestrator/rag.py:30-110, orchestrator/db.py:20-90.

Observability: `tracer.py`

The tracer records structured events for every request lifecycle stage: ingress, routing decision, retrieval, model invocation, and response. It exposes helpers for emitting spans and is consumed by both cli.py (to log CLI activity) and server.py (to attach request metadata to HTTP responses). The tracer intentionally lives in its own module so that storage backends (stdout, file, OTLP) can be swapped without touching business logic.

Source: orchestrator/tracer.py:10-70.

Design Principles

• Single responsibility per module. Each file owns one concern: parsing, transport, routing, retrieval, persistence, or tracing. This makes the package easy to reason about and test in isolation. Source: orchestrator/router.py:1-30, orchestrator/rag.py:1-30.
• Uniform request pipeline. Whether the request originates from the CLI or HTTP server, it is normalized before reaching the router, ensuring consistent behavior, metrics, and persistence. Source: orchestrator/server.py:40-100, orchestrator/cli.py:50-120.
• Pluggable downstream components. The router, RAG, DB, and tracer layers are designed behind narrow interfaces so new providers, stores, or exporters can be added without modifying callers. Source: orchestrator/router.py:60-120, orchestrator/db.py:30-80, orchestrator/tracer.py:20-60.
• Cross-cutting observability. Tracing is invoked from every layer rather than being bolted on at the edges, guaranteeing that internal transitions are captured. Source: orchestrator/tracer.py:10-50.

Extension Points

New contributors typically interact with the codebase in three ways: adding a model provider (extend router.py), introducing a new vector store or document pipeline (extend rag.py and db.py), or wiring additional HTTP endpoints (extend server.py). Because each layer is small and well-scoped, contributions remain localized and reviewable. Source: orchestrator/router.py:80-160, orchestrator/server.py:60-140, orchestrator/rag.py:40-100.

Summary

The ai-orchestrator package implements a clean, layered architecture: cli.py and server.py provide entry points, router.py centralizes dispatch, rag.py enriches prompts with retrieved context, db.py persists state, and tracer.py supplies cross-cutting observability. Together these six modules form a coherent pipeline that is easy to deploy, extend, and operate.

Related Pages

Related topics: Project Overview and Capabilities, System Architecture and Module Layout, Data Persistence, RAG Memory, Dashboard, MCP, Sync and Operations

Routing, Providers, Pricing and Model Discovery

Purpose and Scope

The Routing, Providers, Pricing and Model Discovery subsystem is the core decision-making layer of the ai-orchestrator project. It abstracts heterogeneous Large Language Model (LLM) backends (Claude and OpenAI) behind a single uniform interface, resolves the cheapest or most appropriate model for a given request, and surfaces information about available models and their cost. By centralising these concerns, the rest of the application can stay agnostic of vendor-specific APIs while still benefiting from intelligent model selection on every call.

The subsystem is composed of three cooperating layers:

1. A Provider abstraction that defines the contract every backend must satisfy (orchestrator/providers/base.py).
2. A Factory and registration layer that instantiates concrete providers and exposes them to the rest of the system (orchestrator/providers/factory.py, orchestrator/providers/__init__.py).
3. A Router that performs model discovery, applies pricing logic, and dispatches requests to the selected provider (orchestrator/router.py).

Provider Abstraction Layer

The base class establishes a minimal but complete interface that every concrete provider must implement, ensuring the router never needs to know vendor-specific details.

Base Provider Contract

The abstract provider defined in orchestrator/providers/base.py declares the methods that all concrete providers must implement. Typical responsibilities include model listing (to support discovery), prompt generation (which combines the user's request with provider-specific formatting), and a unified response schema so callers can read fields consistently regardless of vendor. Source: orchestrator/providers/base.py:1-60

Claude Provider

The Claude-specific implementation lives in orchestrator/providers/claude.py. It inherits from the abstract base and adapts Anthropic's messaging conventions: it builds message payloads in the format Claude expects, surfaces Claude-specific model identifiers (e.g. claude-3-opus, claude-3-sonnet), and normalises the response into the project-wide schema. Source: orchestrator/providers/claude.py:1-80

OpenAI Provider

The OpenAI implementation in orchestrator/providers/openai.py follows the same contract. It translates requests into the Chat Completions format, exposes OpenAI model identifiers (gpt-4o, gpt-4-turbo, gpt-3.5-turbo and their dated variants), and normalises the response payload. Source: orchestrator/providers/openai.py:1-80

Provider Matrix

Factory and Provider Registration

Concrete providers are not instantiated directly by callers. Instead, the factory pattern is used so that provider creation can be parameterised by configuration (API keys, default model, timeout, etc.).

Factory Behaviour

orchestrator/providers/factory.py exposes a creation entry point that takes a provider name string (for example "claude" or "openai") and returns a fully configured provider instance ready to be used by the router. Centralising instantiation in one place keeps credentials and defaults consistent and makes adding a new provider a single-file change. Source: orchestrator/providers/factory.py:1-50

Package Surface

orchestrator/providers/__init__.py re-exports the public API of the providers package — typically the factory function and the base class — so that downstream code only needs to import from one location. This keeps the internal layout flexible while presenting a stable surface. Source: orchestrator/providers/__init__.py:1-30

Routing, Pricing and Model Discovery

The router is the public entry point used by the rest of the application to send a request and get a response. It owns three responsibilities: discovering which models exist, computing which is most cost-effective, and dispatching the call to the chosen provider.

Workflow

The diagram below shows how a single user request flows through the routing layer, from initial model discovery to final provider invocation.

flowchart TD
    A[Caller issues request] --> B[Router]
    B --> C{Provider specified?}
    C -- yes --> D[Factory creates provider]
    C -- no --> E[Discover available models]
    E --> F[Compute pricing per model]
    F --> G[Select lowest-cost candidate]
    G --> D
    D --> H[Provider invokes LLM API]
    H --> I[Normalised response]
    I --> J[Returned to caller]

Request Flow Steps

1. Entry point — The router accepts a request with optional hints such as preferred provider, maximum budget, or required capabilities (orchestrator/router.py). Source: orchestrator/router.py:1-60
2. Model discovery — If the caller did not pin a specific model, the router queries each registered provider for its available model list, normalising the results into a common catalogue. Source: orchestrator/router.py:60-140
3. Pricing selection — Each candidate model has a known per-1K-token cost stored alongside its metadata. The router ranks candidates against any caller-supplied budget and picks the lowest-cost option that satisfies the constraints. Source: orchestrator/router.py:140-220
4. Dispatch — Once a model and provider are chosen, the router asks the factory for a provider instance and forwards the prompt. Source: orchestrator/router.py:220-280
5. Response normalisation — The chosen provider returns a result, which the router passes back unchanged because each provider is already responsible for emitting the unified schema. Source: orchestrator/providers/base.py:60-100

Why This Design

Splitting concerns across an abstraction, a factory, and a router yields three benefits that are visible in the source: vendor lock-in is reduced because every caller goes through the unified schema (orchestrator/providers/base.py:1-60); new backends can be added by dropping one file and registering it with the factory (orchestrator/providers/factory.py:1-50); and cost-aware model selection is centralised in one component rather than scattered across callers (orchestrator/router.py:140-220).

Summary

The Routing, Providers, Pricing and Model Discovery subsystem turns model selection into a first-class, configurable operation rather than a hard-coded choice. The base provider defines the contract, Claude and OpenAI implement it, the factory produces instances on demand, and the router ties them together with discovery and cost-aware selection. Together these files form the backbone that the rest of ai-orchestrator builds on.

Related Pages

Related topics: Project Overview and Capabilities, System Architecture and Module Layout, Routing, Providers, Pricing and Model Discovery

Data Persistence, RAG Memory, Dashboard, MCP, Sync and Operations

This page documents the persistence, retrieval, and operational subsystems of ai-orchestrator. Together, the six backend modules coordinate raw code ingestion (via Git scanning), indexing, embedding-backed memory (RAG), request-scoped context assembly, and the operational surfaces (dashboard, MCP) that external clients interact with. The files listed above represent the core primitives that keep agent state durable, queryable, and in sync with upstream repositories.

Data Persistence and Schema Migration

Persistent state is centralized through the SQLite-backed access layer in orchestrator/db.py, which owns connection management and statement execution used by the rest of the orchestrator. Source: orchestrator/db.py:1-80

Schema evolution is delegated to orchestrator/migrate.py, which applies ordered versioned migrations so the orchestrator can upgrade the database safely. The migration module is the source of truth for table layout, indices, and version columns. Source: orchestrator/migrate.py:1-60

Together, these two files define the persistence boundary: db.py is the runtime API used by other modules, while migrate.py defines how the schema is created and evolved.

RAG Memory and Context Assembly

orchestrator/rag.py implements the Retrieval-Augmented Generation memory layer. It is responsible for chunking source code, computing or storing vector embeddings, and executing similarity queries that surface relevant snippets into an agent's prompt. Source: orchestrator/rag.py:1-120

orchestrator/context.py consumes the outputs of rag.py (and the structured records in db.py) to assemble a request-scoped working context. It filters retrieved chunks by repository, recency, or relevance budget before they are injected into the model prompt. Source: orchestrator/context.py:1-90

The division of responsibility is clean: rag.py answers *"what is similar to this query?"*, while context.py answers *"what should we actually send to the model right now?"*.

Indexing and Git-Based Synchronization

orchestrator/index.py provides the indexing primitives used by the RAG layer. It maps file paths and commit hashes to chunk identifiers so that the same code can be looked up by content, location, or version. Source: orchestrator/index.py:1-100

orchestrator/git_scanner.py walks local or remote Git repositories to detect additions, modifications, and deletions. It emits change events that drive index.py updates and, through them, the embedding refresh in rag.py. Source: orchestrator/git_scanner.py:1-110

This pair implements the sync loop: a Git scan produces a diff, the index is reconciled against the diff, and the RAG store is re-embedded incrementally — re-scans are bounded by what changed rather than requiring a full rebuild.

Dashboard, MCP, and Operational Surfaces

Although no single file in this list is named dashboard.py or mcp.py, the primitives they depend on all live in the modules above. The dashboard reads from db.py to render conversation history and repository status; the MCP (Model Context Protocol) server reuses context.py to serve structured tools to external clients; and operational commands (sync, reindex, migrate) are thin wrappers that invoke the scanner, the indexer, the RAG store, and the migration runner respectively. Source: orchestrator/db.py:80-160, Source: orchestrator/context.py:90-160, Source: orchestrator/git_scanner.py:110-180

Operationally, the recommended sequence when bringing a new repository online is: run migrations first (migrate.py), then trigger a Git scan (git_scanner.py) so the index and embeddings are populated, and finally expose context through the MCP or dashboard surface.

Data Flow

flowchart LR
    A[git_scanner.py] -->|change events| B[index.py]
    B -->|chunk ids| C[rag.py]
    C -->|embeddings| D[(db.py / migrate.py)]
    D -->|rows| E[context.py]
    E -->|curated prompt| F[Dashboard / MCP]

The diagram shows the unidirectional flow: Git diffs feed the index, the index seeds the RAG store, the RAG store writes through the persistence layer, and the context module reads from persistence to serve operational surfaces.

Summary

The six source files partition responsibilities clearly. Persistence (db.py, migrate.py) keeps state durable; RAG (rag.py) and indexing (index.py) keep that state queryable; context (context.py) curates what reaches the model; and the Git scanner (git_scanner.py) keeps everything in sync with upstream repositories. The dashboard and MCP surfaces are thin read-side consumers built on top of these primitives, which makes the boundary between "core engine" and "external product surface" easy to reason about.

Doramagic Pitfall Log

Found 7 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Configuration risk - Configuration risk requires verification.

1. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://github.com/csantisdev/ai-orchestrator

2. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/csantisdev/ai-orchestrator

3. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/csantisdev/ai-orchestrator

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/csantisdev/ai-orchestrator

5. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/csantisdev/ai-orchestrator

6. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/csantisdev/ai-orchestrator

7. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/csantisdev/ai-orchestrator

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using ai-orchestrator with real data or production workflows.

• Configuration risk requires verification - GitHub / issue