Related Pages

Related topics: LM Clients and Adapters, Built-in Modules and Programmatic Building Blocks, Optimizers and Prompt Compilation

Overview and Core Architecture

Introduction to DSPy

DSPy is a framework for programming—rather than prompting—language models. Instead of writing brittle string prompts, developers compose modular Python code; DSPy then compiles that code into optimized instructions, demonstrations, or fine-tuned weights for the underlying model. Source: README.md:1-30.

The acronym stands for Declarative Self-improving Python. The framework is positioned for building simple classifiers, sophisticated RAG pipelines, and agent loops, all backed by algorithms that optimize prompts and weights automatically. Source: README.md:32-40.

The community has driven several major expansions that have shaped the current architecture: support for chat-mode models (Issue #662), streaming LMs (Issue #338), structured outputs (Issue #1365), and function/tool calling (Issue #192). The most recent beta, 3.3.0b1, introduces a ReActV2 module and a new BaseLM system, alongside a major internal refactor discussed in Issue #390.

Core Architectural Components

The framework is organized around a small number of powerful abstractions that grew organically over time. Source: README.md:38-44.

Language Model Clients

BaseLM defines the contract every provider adapter must satisfy. DummyLM is a built-in subclass for unit tests, supporting three modes: a list of dictionaries, a dictionary keyed by prompt, or randomized answers. Source: dspy/utils/dummies.py:1-60. DummyLM defaults to the ChatAdapter so that mocked completions mimic real structured output formats. Source: dspy/utils/dummies.py:62-95.

Adapters and Structured Output

Adapters translate between DSPy's Signature objects and provider-specific request/response shapes. Community request Issue #1365 asked for OpenAI structured-output guarantees, and adapters such as ChatAdapter (imported by DummyLM) handle that translation. The constants module exposes internal markers like IS_TYPE_UNDEFINED used when a signature field lacks an explicit type annotation. Source: dspy/utils/constants.py:1-4.

Tools and Function Calling

Tool use—central to Issue #192—is implemented through a unified Tool abstraction. DSPy ships first-class converters from external ecosystems:

Both converters route through convert_input_schema_to_tool_args, ensuring consistent arg, type, and description metadata across ecosystems.

Optimizers (Teleprompters)

The README and 3.3.0b1 release notes cite several optimizer families including GEPA, BootstrapFewShot, MIPRO, and BootstrapFinetune. These compilers take a developer-defined program and a metric, then search the space of instructions and demonstrations. The optimization layer is also why deterministic execution traces matter: developers must be able to inspect every LM call during and after compilation.

Error Handling Framework

DSPy exposes a structured error hierarchy anchored at DSPyError, which carries metadata such as code, model, provider, status, request_id, and retry_after. Source: dspy/utils/exceptions.py:1-50.

Language-model failures fall under LMError, with specialized subclasses:

• LMProviderError — wraps provider response errors and surfaces status/request-id metadata.
• LMTimeoutError, LMRateLimitError, LMServerError, LMTransportError — all retryable.
• LMContextWindowExceededError — raised when the prompt exceeds the model's window.
• LMUnsupportedFeatureError — used when a provider cannot honor a requested capability such as finetuning, reinforce, or structured_outputs. Source: dspy/utils/exceptions.py:52-150.

The helper is_retryable_lm_error lets callers classify wrapped exceptions after provider retries are exhausted; it is explicitly advisory, and callers should still honor retry_after. Source: dspy/utils/exceptions.py:120-140. AdapterParseError lives alongside LM errors to signal that an adapter could not coerce an LM response into the expected signature.

Developer Utilities and Observability

History Inspection

pretty_print_history formats the last *n* prompt/completion pairs to stdout or a user-supplied file, supporting text, image, and audio content blocks. Source: dspy/utils/inspect_history.py:1-80. This is the primary on-ramp for debugging compiled programs and tracing agent loops.

Callback System

The BaseCallback protocol defines hooks such as on_module_start, on_module_end, on_lm_start, and on_lm_end, each carrying a call_id so start/end handlers can be correlated. Source: dspy/utils/callback.py:1-60. This is the integration point for telemetry, evaluation harnesses, and the streaming research discussed in Issue #338.

Lazy Imports

_LazyModule defers expensive third-party dependencies (e.g., NumPy, Pandas) until first attribute access, then materializes the real module under a per-module lock and restores the proxy if import fails. Source: dspy/utils/lazy_import.py:1-40. This is what makes the 3.3.0b1 release notes' "fewer dependencies framework wide" claim possible without breaking existing programs.

General Utilities

dspy/dsp/utils/utils.py provides foundational helpers such as print_message (timestamped logging) and timestamp (used for directory naming during optimizer runs). Source: dspy/dsp/utils/utils.py:1-30.

Request Flow at a Glance

The following diagram summarizes how a single call flows through the architecture, tying together the abstractions described above.

flowchart LR
    A[User Module] --> B[Adapter e.g. ChatAdapter]
    B --> C[BaseLM provider]
    C --> D{Retryable error?}
    D -- yes --> C
    D -- no --> E[AdapterParseError or LMError]
    B --> F[Signature parse]
    F --> G[Callback: on_lm_end]
    G --> H[pretty_print_history]

See Also

• Signatures and Fields
• Adapters and Chat Formatting
• Optimizers and Compilation
• Error Handling Reference
• Tool Ecosystem Integration

Related Pages

Related topics: Overview and Core Architecture, Built-in Modules and Programmatic Building Blocks

LM Clients and Adapters

DSPy separates the concerns of *talking to a language model* (LM clients) from *shaping prompts and parsing outputs* (adapters). The LM client layer issues provider calls, caches results, and raises structured errors; the adapter layer turns a Signature plus demos into a chat-style prompt and back into a typed prediction. This page describes the architecture of that division, the public error vocabulary, the integration points with LangChain and MCP tools, and the supporting infrastructure for testing, history inspection, and lifecycle callbacks.

LM Client Architecture

The client layer is built around an abstract BaseLM defined in dspy/clients/base_lm.py. Concrete implementations include the LiteLLM-backed LM in dspy/clients/lm.py, the internal adapter in dspy/clients/_litellm.py, and the OpenAI-specific helpers in dspy/clients/openai.py and dspy/clients/openai_format.py. Caching is provided by dspy/clients/cache.py. Programmatic configuration is set globally through dspy.configure(lm=...) so that every module reuses the configured client.

The new BaseLM system (introduced in the 3.x line) addresses long-standing community requests such as native chat support (issue #662) and a stable substrate for streaming (issue #338), while still leaving room for provider-specific structured outputs (issue #1365). The 3.3.0b1 release notes confirm a "new BaseLM System" alongside ReActV2 and GEPA updates.

flowchart LR
    Module[Module / Predict] --> Adapter
    Adapter -->|formatted messages| LM[BaseLM / LM]
    LM -->|provider call| Provider[(OpenAI / LiteLLM / etc.)]
    LM --> Cache[(Cache)]
    Provider --> LM
    LM -->|raw text| Adapter
    Adapter -->|parsed prediction| Module

Adapters and Format/Parsing

Adapters live alongside clients and own the bidirectional translation between DSPy objects and chat messages. ChatAdapter is the default; it formats fields with [[ ## field_name ## ]] headers, which is precisely the marker pattern consumed by DummyLM in dspy/utils/dummies.py. The dummy implementation uses field_header_pattern to locate output headers in messages, demonstrating the on-the-wire contract that real adapters must also respect (Source: dspy/utils/dummies.py:42-62).

When a provider returns text that does not contain the expected fields, adapters raise AdapterParseError from dspy/utils/exceptions.py. The exception message names the adapter, includes the raw LM response, and lists the expected output fields — the same shape used by higher-level fallback strategies (Source: dspy/utils/exceptions.py:165-180).

Error Hierarchy

DSPy exposes a structured exception hierarchy so that callers can distinguish recoverable failures from user errors. DSPyError is the root and stores a stable code, model, provider, provider_code, status, request_id, and retry_after (Source: dspy/utils/exceptions.py:11-44). LMError extends it for the language-model boundary, and a family of subclasses classifies provider behavior:

The helper is_retryable_lm_error groups LMRateLimitError, LMTimeoutError, LMServerError, and LMTransportError so that orchestration code can decide when to back off (Source: dspy/utils/exceptions.py:128-137). LMUnexpectedError is intentionally distinct from parse errors so unknown provider exceptions are not misclassified (Source: dspy/utils/exceptions.py:62-71).

Tooling Integrations: LangChain and MCP

DSPy adapters can invoke external tools either by wrapping LangChain tools or by speaking the Model Context Protocol. convert_langchain_tool in dspy/utils/langchain_tool.py inspects the tool's Pydantic args_schema, derives args, arg_types, and arg_desc, and returns an async-aware DSPy Tool. This addresses the recurring community question (issue #192) about delivering structured outputs through function calling.

For MCP, dspy/utils/mcp.py defines _convert_mcp_tool_result to flatten CallToolResult into either a single text string or a list of content parts, and convert_mcp_tool builds a DSPy Tool bound to a live ClientSession. Both helpers funnel through convert_input_schema_to_tool_args in dspy/adapters/types/tool.py, which guarantees a uniform argument shape regardless of the source framework.

Observability: History, Callbacks, and Lazy Loading

History inspection is implemented by pretty_print_history in dspy/utils/inspect_history.py. It renders the last n prompt/response pairs to stdout or any TextIO target, disables ANSI colors when a file is supplied, and supports multimodal content (text, image_url, input_audio) so that vision and audio modules are first-class in traces (Source: dspy/utils/inspect_history.py:18-79).

Lifecycle observability is provided by BaseCallback in dspy/utils/callback.py. It exposes paired hooks for LM calls (on_lm_start / on_lm_end) and adapter formatting (on_adapter_format_start / on_adapter_format_end). Each event carries a call_id for start/end correlation, and on_lm_end propagates any raised exception so observers can record failures (Source: dspy/utils/callback.py:7-72).

Finally, the dependency surface is narrowed by _LazyModule in dspy/utils/lazy_import.py, which defers heavy third-party imports until first attribute access. This is the mechanism that allowed 3.3.0b1 to ship "fewer dependencies framework wide" — the base dspy install no longer needs NumPy, and provider backends are only loaded when an LM is actually instantiated (Source: dspy/utils/lazy_import.py:42-72).

See Also

• Signatures and Modules
• Adapters (ChatAdapter, JSONAdapter, TwoStepAdapter)
• Optimizers (BootstrapFewShot, MIPRO, GEPA)
• DSPy Configuration and Global State

Related Pages

Related topics: Overview and Core Architecture, LM Clients and Adapters, Optimizers and Prompt Compilation

Built-in Modules and Programmatic Building Blocks

DSPy frames language model programming as compositional Python code rather than brittle string prompts. According to the project README, the goal is to let users "iterate fast on building modular AI systems" by writing dspy.Module subclasses whose prompts and weights are then optimized automatically (README.md). The "built-in modules" surface area is therefore broad: it spans the Predict family of modules, the adapter that formats their outputs, and a deep layer of utility building blocks — exceptions, lazy importers, test doubles, tool bridges, and observability hooks — that every module and every user program ultimately relies on.

This page covers the programmatic building blocks that sit underneath the user-facing modules: the infrastructure that modules assume is available when they execute, fail, are tested, or are observed.

Error Handling Foundation

DSPy modules call language models and parse structured outputs. Both operations can fail in nuanced ways that should not be conflated with ordinary Exception. The framework provides a structured exception hierarchy rooted at DSPyError, which carries stable code, model, provider, provider_code, status, request_id, and retry_after fields (dspy/utils/exceptions.py:24-58).

Catching LMError is the recommended pattern for module-level retry and fallback logic. The separation of LMUnexpectedError is intentional: it prevents adapter fallback behavior from mis-classifying unknown provider failures as parse failures (dspy/utils/exceptions.py:64-82).

Lazy Dependency Loading

DSPy advertises itself as having "fewer dependencies framework wide" in the 3.3.0b1 release. This is enabled by a custom lazy import system that defers optional packages — langchain, weaviate, anthropic, numpy, litellm — until first attribute access. A _LazyModule proxy stands in for the real module in sys.modules; the first attribute access swaps it out under a per-module RLock (dspy/utils/lazy_import.py:42-89).

If the package is missing, require() returns a _MissingModule stand-in that raises ImportError on any attribute access, embedding the original call site (filename, lineno, function, code_context) in the traceback so the failure is actionable (dspy/utils/lazy_import.py:21-40). Attribute assignment on a _LazyModule materializes the real module immediately so that configuration writes always apply to the real dependency.

For module authors this means a clean from dspy.utils.lazy_import import require; mcp = require("mcp") is enough to integrate an optional provider without forcing every DSPy user to install it.

Tool Integration Bridges

DSPy's ReAct-style modules consume dspy.Tool objects. To interoperate with the broader ecosystem, two bridge functions convert third-party tools into DSPy tools.

convert_langchain_tool(tool) inspects a LangChain BaseTool's args_schema (a Pydantic model), reconstructs the DSPy argument name/type/description triples via convert_input_schema_to_tool_args, and wraps tool.ainvoke into a DSPy-compatible async callable (dspy/utils/langchain_tool.py:18-46). This is the answer to long-standing community questions about how to use function calling inside DSPy (cf. issue #192).

convert_mcp_tool(session, tool) does the analogous work for Model Context Protocol tools, calling session.call_tool and normalizing the CallToolResult into either a single string, a list of strings, or the original non-text content when no textual payload is present (dspy/utils/mcp.py:9-41). MCP errors surface as RuntimeError so module authors can detect them uniformly.

Testing with `DummyLM`

Every module must be testable without a real provider. DummyLM, defined in dspy/utils/dummies.py, subclasses dspy.clients.base_lm.BaseLM and supports three modes:

1. List of dictionaries — each request consumes the next dictionary, formatted with the active adapter's format_field_with_value.
2. Dictionary of dictionaries — the final user message is matched as a key.
3. Generated answers — when answers is a callable, it is invoked with the requested field names, enabling data-driven tests.

The dummy also supports follow_examples=True, in which it inspects the rendered message stream, locates field headers via field_header_pattern, and replays the matching prior assistant turn — useful for testing multi-turn pipelines (dspy/utils/dummies.py:55-89). This pattern addresses the community's recurring need (issue #390) to test complex DSPy programs deterministically without network access.

Observability Hooks

Two complementary observability mechanisms are exposed:

• BaseCallback defines on_module_start, on_module_end, on_lm_start, and on_lm_end hooks that fire around every dspy.Module.forward and LM.__call__. Each event carries a call_id so start/end pairs can be correlated (dspy/utils/callback.py:1-79).
• pretty_print_history(history, n, file) renders the last *n* prompts and completions with ANSI colors, supports tool calls, image URLs, and base64-encoded audio, and disables colors automatically when a file argument is supplied (dspy/utils/inspect_history.py:21-82).

Together these provide both machine-consumable traces (callbacks) and human-readable diagnostics (history printing). The print_message helper in dspy/dsp/utils/utils.py further standardizes timestamped console output across the framework (dspy/dsp/utils/utils.py:5-25).

Summary

While Predict, ChainOfThought, ReAct, ProgramOfThought, and CodeAct are the user-facing modules, every one of them relies on this substrate: typed DSPyError hierarchy for failure handling, lazy imports for optional providers, DummyLM for hermetic tests, LangChain and MCP bridges for tool interoperability, and callbacks plus pretty_print_history for observability. Mastering these building blocks is what makes it possible to author robust DSPy programs that survive provider outages, missing dependencies, and evolving model capabilities.

See Also

• Signatures and Adapters — the typed I/O contracts that modules consume.
• GEPA Optimizer — the latest reflective prompt-evolution optimizer, integrated in 3.3.0b1.
• BaseLM and Provider Backends — provider configuration, structured outputs (cf. issue #1365), and streaming (cf. issue #338).

Related Pages

Related topics: Overview and Core Architecture, Built-in Modules and Programmatic Building Blocks

Optimizers and Prompt Compilation

Purpose and Scope

DSPy is a framework for programming — rather than prompting — language models. Instead of writing brittle natural-language prompts, developers compose modules in Python and then call an *optimizer* (also called a *teleprompter*) to automatically tune the prompts, few-shot demonstrations, or LM weights that those modules use at runtime. As stated in the project README, DSPy "offers algorithms for optimizing their prompts and weights, whether you're building simple classifiers, sophisticated RAG pipelines, or Agent loops." (Source: README.md)

The optimizer subsystem therefore serves two related but distinct purposes:

1. Prompt compilation — generating and refining natural-language instructions and demonstrations that are injected into the LM call template for each signature field.
2. Weight/parameter compilation — fine-tuning the underlying model so the optimized behavior is baked into the weights themselves (for providers that support it).

The community has long treated this layer as a primary differentiator for DSPy: the *Major refactor roadmap* discussion (#390) specifically singles out optimizers as one of the framework's "extremely powerful concepts" and motivates a refactor around them.

Architecture: Teleprompters

The base abstraction lives in the dspy/teleprompt package. Every concrete optimizer implements the Teleprompter protocol declared in dspy/teleprompt/teleprompt.py, which exposes a compile(student, *, trainset, ...) method that returns a *compiled* copy of the program with optimized prompts/demos attached. The compiled program behaves like the original module from the caller's perspective, but its __call__ produces higher-quality outputs because the underlying adapter renders improved instructions and few-shot examples.

flowchart LR
    A[User-defined Module<br/>Predict / CoT / ReAct] --> B[Teleprompter.compile]
    T[Trainset + Metric] --> B
    LM[BaseLM] --> B
    B --> C[Compiled Module]
    C --> D[Adapter renders<br/>optimized prompt + demos]
    D --> E[LM request]
    E --> F[Parsed output]

Concrete Optimizers

GEPA's reflection loop relies on a separate instruction-proposal component implemented at dspy/teleprompt/gepa/instruction_proposal.py, and on shared helpers in dspy/teleprompt/gepa/gepa_utils.py. The 3.3.0b1 release notes confirm that GEPA is updated to version 0.1.1 in the current line.

Compilation Workflow

A typical optimization run follows this pattern:

1. Define the program. A user writes one or more dspy.Module subclasses (e.g., ChainOfThought, ReAct, or a custom pipeline). Each module references a Signature whose fields become the LM prompt template.
2. Configure an LM and a metric. dspy.configure(lm=...) registers a BaseLM globally; the metric scores candidate outputs.
3. Call optimizer.compile(program, trainset=trainset) — the teleprompter drives the inner loop, calling the program repeatedly to score candidate prompts/demos.
4. Deploy the compiled program. The returned object can be saved, reloaded, and called like any other module.

During step 3, optimizers routinely invoke the underlying LM many times in arbitrary order, which is one of the reasons the *stream to DSPy LMs* thread (#338) notes that streaming is a non-trivial feature for compiled programs.

Supporting Infrastructure

Compilation touches several supporting utilities:

• Error classification — Optimizer loops wrap LM calls in retries; dspy/utils/exceptions.py exposes is_retryable_lm_error() and the _RETRYABLE_LM_ERRORS tuple (rate limit, timeout, server, transport) so teleprompters can decide whether to re-sample. Feature gaps (e.g., a provider lacking "finetuning" or "structured_outputs") surface as LMUnsupportedFeatureError with a features=[...] list.
• Testing — dspy/utils/dummies.py provides DummyLM, which is essential when unit-testing a teleprompter: it can replay a fixed list of completions, look up completions by prompt keyword, or generate completions from a mapping. This lets optimizer tests run deterministically without network calls.
• History inspection — dspy/utils/inspect_history.py defines pretty_print_history(), which dumps the most recent prompts and outputs (with tool calls, image URLs, and audio payloads rendered safely) so a developer can see *what* the optimizer actually sent the LM during compilation.
• Tool integration — dspy/utils/langchain_tool.py and dspy/utils/mcp.py convert external tools (LangChain, MCP) into DSPy Tool objects, which the compiled module can call inside ReAct or program-of-thought loops — the *function calling* discussion (#192) and the *Support Chat Mode* discussion (#662) both surface here.

Common Failure Modes

See Also

• Signatures and Adapters — how the compiled prompt is actually rendered.
• Language Model Clients (dspy.clients) — the BaseLM interface that all optimizers call.
• ReAct / Agent Modules — pipelines that frequently benefit from MIPROv2 or GEPA compilation.
• *GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning* (arXiv:2507.19457, Jul'25) — the paper behind the GEPA teleprompter.

Doramagic Pitfall Log

Found 9 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification.

1. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/stanfordnlp/dspy

2. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/stanfordnlp/dspy

3. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/stanfordnlp/dspy

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/stanfordnlp/dspy

5. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/stanfordnlp/dspy/issues/9933

6. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/stanfordnlp/dspy/issues/9922

7. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/stanfordnlp/dspy/issues/9911

8. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/stanfordnlp/dspy

9. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/stanfordnlp/dspy

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using dspy with real data or production workflows.

• [[Bug] (withdrawn) Module.save() JSON state + Retrieve.dump_state json_mo](https://github.com/stanfordnlp/dspy/issues/9933) - github / github_issue
• [[Feature] Add append_instructions to Signature](https://github.com/stanfordnlp/dspy/issues/9829) - github / github_issue
• [[Feature] Memory/retrieval poisoning defense via OWASP Agent Memory Guar](https://github.com/stanfordnlp/dspy/issues/9911) - github / github_issue
• [[Feature] MCP guide — security agent over remote HTTP MCP servers (re #8](https://github.com/stanfordnlp/dspy/issues/9922) - github / github_issue
• 3.3.0b1 - github / github_release
• 3.2.1 - github / github_release
• 3.2.0 - github / github_release
• 3.1.3 - github / github_release
• 3.1.2 - github / github_release
• 3.1.1 - github / github_release
• 3.1.0 - github / github_release
• 3.1.0b1 - github / github_release