Related Pages

Related topics: Installation and Setup, Core Architecture

Introduction to DSPy

DSPy (Declarative Self-Improving Language Model Programs) is a framework for building and optimizing LLM-based pipelines. It provides a systematic approach to compiling declarative module calls into self-improving pipelines that can automatically optimize prompts and weights.

Sources: README.md:1

Core Philosophy

DSPy abstracts away the complexity of prompt engineering by allowing developers to define programs as modules with signatures rather than manually crafting prompts. The framework then automatically optimizes how these modules interact to achieve better performance.

graph TD
    A[User Defined Program] --> B[DSPy Signatures]
    B --> C[DSPy Modules]
    C --> D[Optimizer Compilation]
    D --> E[Optimized Pipeline]
    E --> F[Better Results]

Sources: dspy/primitives/module.py:1-50

Key Concepts

Signatures

Signatures define the input-output schema for modules. They specify which fields are inputs and which are outputs using InputField and OutputField decorators.

class MySignature(dspy.Signature):
    input_text: str = dspy.InputField(desc="Input sentence")
    output_text: str = dspy.OutputField(desc="Translated sentence")

Sources: dspy/signatures/signature.py:1-30

Modules

All DSPy programs inherit from dspy.Module, which provides methods for managing predictors and configuring language models.

Sources: dspy/primitives/module.py:50-100

Examples

The Example class represents data records used for training and evaluation. It supports input/label separation and dictionary-like access.

example = dspy.Example(question="What is 2+2?", answer="4").with_inputs("question")

Sources: dspy/primitives/example.py:1-100

Predict

Predict is the core module that generates completions based on a signature. It can be configured with or without chain-of-thought reasoning.

predictor = dspy.Predict("question -> answer")
predictor = dspy.ChainOfThought("question -> answer")

Sources: dspy/primitives/module.py:80-120

Installation

Standard Installation

pip install dspy

Development Installation

For the latest features from the main branch:

pip install git+https://github.com/stanfordnlp/dspy.git

Sources: README.md:5-12

Development Environment Setup

Python 3.10 or later is required. The recommended setup uses uv:

uv venv --python 3.10
uv sync --extra dev

Run tests with:

uv run pytest tests/predict

Sources: CONTRIBUTING.md:40-80

Basic Usage Pattern

graph LR
    A[Define Signature] --> B[Create Module]
    B --> C[Configure LM]
    C --> D[Compile with Optimizer]
    D --> E[Run Program]

Step 1: Define a Signature

class QA(dspy.Signature):
    """Answer questions based on the context."""
    context: str = dspy.InputField(desc="Context for answering")
    question: str = dspy.InputField(desc="Question to answer")
    answer: str = dspy.OutputField(desc="Answer to the question")

Step 2: Build a Module

class RAG(dspy.Module):
    def __init__(self):
        super().__init__()
        self.retrieve = dspy.Retrieve(k=3)
        self.qa = dspy.Predict(QA)
    
    def forward(self, question):
        context = self.retrieve(question).passages
        return self.qa(context=context, question=question)

Step 3: Configure Language Model

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

Step 4: Compile and Run

compiled_rag = dspy.compile(RAG(), trainset=trainset, metric=metric)
result = compiled_rag(question="What is DSPy?")

Supported Data Types

DSPy provides specialized types for common LLM use cases:

Document

doc = Document(
    data="The Earth orbits the Sun.",
    title="Basic Astronomy Facts"
)

Sources: dspy/adapters/types/document.py:1-50

History

history = History(messages=[
    {"question": "What is the capital of France?", "answer": "Paris"},
])

Sources: dspy/adapters/types/history.py:1-50

Embedding Support

The Embedder class provides a unified interface for embedding models:

embedder = dspy.Embedder("openai/text-embedding-3-small")
embeddings = embedder(["hello", "world"], batch_size=1)

Configuration options:

Sources: dspy/clients/embedding.py:1-60

Teleprompters and Optimization

DSPy includes teleprompters that automatically optimize prompts and weights:

optimizer = BootstrapFewShotWithRandomSearch(metric=metric)
compiled_program = optimizer.compile(student=student_program, trainset=trainset)

Available optimizers include:

• BootstrapFewShotWithRandomSearch - Bootstrap demonstrations with random search
• BootstrapFinetune - Fine-tune weights on bootstrapped data
• GEPA - Reflective prompt evolution
• BetterTogether - Combined prompt + weight optimization

Sources: dspy/teleprompt/bettertogether.py:1-50

Architecture Overview

graph TD
    subgraph "User Layer"
        A[User Program]
        B[Signatures]
        C[Examples]
    end
    
    subgraph "Core Layer"
        D[Module]
        E[Predict]
        F[ChainOfThought]
        G[Retrieve]
    end
    
    subgraph "Optimization Layer"
        H[Teleprompters]
        I[Proposers]
        J[Metrics]
    end
    
    subgraph "Adapter Layer"
        K[Document]
        L[History]
        M[Image]
        N[Citations]
    end
    
    subgraph "Client Layer"
        O[LM Client]
        P[Embedding Client]
        Q[Retriever Clients]
    end
    
    A --> D
    B --> D
    C --> H
    D --> E
    E --> H
    H --> I
    K --> O
    L --> O
    M --> O
    N --> O
    O --> P
    O --> Q

Citation

If you use DSPy in your research, please cite:

[Oct'23] DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

Sources: README.md:20-30

Documentation and Resources

For comprehensive documentation, visit the DSPy Docs at dspy.ai.

Research Papers

Sources: README.md:15-35

Related Pages

Related topics: Introduction to DSPy, Language Model Clients

Installation and Setup

This page covers how to install DSPy and configure your development environment for working with the framework.

Overview

DSPy (Declarative Self-Improving Language Model Programs) is a Python framework that compiles declarative language model calls into self-improving pipelines. The installation process supports both end-user usage via pip and contributor workflows with advanced development tools. Sources: README.md

Prerequisites

System Requirements

Sources: CONTRIBUTING.md

Basic Installation

Installing from PyPI

The simplest way to install DSPy is using pip:

pip install dspy

Sources: README.md

Installing from Source

To install the latest development version from the main branch:

pip install git+https://github.com/stanfordnlp/dspy.git

This approach installs the most up-to-date code that may include features not yet released to PyPI.

Sources: README.md

Development Environment Setup

For contributors who want to modify DSPy or run tests, a development environment setup is required.

Fork and Clone

First, fork the repository on GitHub, then clone your fork locally:

git clone {url-to-your-fork}
cd dspy

Sources: CONTRIBUTING.md

Using uv (Recommended)

uv is a Rust-based Python package and project manager that provides fast dependency resolution and virtual environment management.

#### Installation of uv

Follow the uv installation guide to install uv on your system.

#### Environment Creation

Create a virtual environment using Python 3.10:

uv venv --python 3.10

This creates a .venv directory in your project root.

#### Dependency Synchronization

Sync the environment with development dependencies:

uv sync --extra dev

#### Verification

Run unit tests to verify the setup:

uv run pytest tests/predict

Sources: CONTRIBUTING.md

Using conda + pip

An alternative approach using conda and pip is available for users who prefer this workflow.

#### Environment Creation

conda create -n dspy-dev python=3.10
conda activate dspy-dev

#### Package Installation

pip install -e ".[dev]"

The -e flag installs the package in editable mode, allowing code changes to take effect without reinstallation.

#### Verification

pytest tests/predict

Sources: CONTRIBUTING.md

Environment Management Patterns

Using uv run

The uv run prefix must be used for every Python command when using the uv-managed environment:

Sources: CONTRIBUTING.md

Development Workflow

graph TD
    A[Fork Repository] --> B[Clone Your Fork]
    B --> C[Install uv]
    C --> D[Create venv with Python 3.10]
    D --> E[Sync dependencies with dev extras]
    E --> F[Run tests to verify]
    F --> G[Start development]
    G --> H[Make changes]
    H --> I[Commit changes]
    I --> J[Push to fork]
    J --> K[Create Pull Request]

Project Configuration

pyproject.toml Structure

The project uses pyproject.toml for dependency management and package configuration. Key sections include:

• build-system: Build backend configuration
• project: Core dependencies and project metadata
• project.optional-dependencies: Extra dependencies for different features (dev, etc.)

Sources: pyproject.toml

Pre-commit Hooks

DSPy uses pre-commit hooks to maintain code quality. After installing dependencies, hooks are automatically installed but need to be staged and committed along with your changes.

Running Hooks Manually

All pre-commit checks must pass before creating a pull request. You can commit changes and let hooks fix formatting issues automatically.

Sources: CONTRIBUTING.md

Verification Checklist

After completing installation, verify your setup by checking:

1. Python version: python --version (should be 3.10+)
2. DSPy import: python -c "import dspy; print(dspy.__version__)"
3. Running basic tests: uv run pytest tests/predict -v

Next Steps

After successful installation and setup:

• Review the DSPy documentation for framework concepts
• Explore the signature system for defining module inputs/outputs
• Set up your language model configuration
• Try the teleprompters for prompt optimization

Troubleshooting

Common Issues

Related Documentation

• DSPy Official Docs
• GitHub Repository
• Research Papers

Related Pages

Related topics: Signatures System, Module System

Core Architecture

The DSPy framework's core architecture provides the foundational building blocks for composing, executing, and optimizing language model programs. At its heart, DSPy separates the concerns of program structure, data representation, and model interaction into distinct but interconnected components.

Architecture Overview

DSPy's core architecture follows a layered design where higher-level abstractions build upon primitive components. The primitives module (dspy/primitives/) contains the essential classes that define how programs are structured, how data flows through them, and how outputs are represented.

graph TD
    subgraph "Core Primitives"
        Example[Example<br/>Data Container]
        Module[Module<br/>Program Base]
        Prediction[Prediction<br/>Output Container]
    end
    
    subgraph "Language Model Layer"
        LM[Language Model<br/>Client]
        Embedder[Embedder<br/>Embedding Client]
    end
    
    subgraph "Program Layer"
        Predict[Predict Module]
        CoT[ChainOfThought]
        RLM[RLM Module]
    end
    
    Example -->|Defines I/O| Module
    Module -->|Executes via| LM
    LM -->|Produces| Prediction
    Predict -->|Uses| Module
    RLM -->|Extends| Module

Data Model Layer

The `Example` Class

The Example class serves as DSPy's primary data structure for representing training examples, test cases, and evaluation samples. It wraps a dictionary-based storage (_store) and maintains metadata about which fields are inputs versus labels.

Key Characteristics:

Core Methods:

Sources: dspy/primitives/example.py:1-50

Usage Pattern:

# Create an example with question-answer pairs
example = dspy.Example(
    question="What is the capital of France?",
    answer="Paris"
).with_inputs("question")

# Access inputs and labels separately
example.inputs()    # Example({'question': '...'}) 
example.labels()    # Example({'answer': '...'})

# Convert to dictionary for serialization
example.toDict()    # {'question': '...', 'answer': '...'}

The toDict() method handles nested objects by recursively converting Example objects, Pydantic models, lists, and dicts to JSON-friendly structures.

Sources: dspy/primitives/example.py:1-30

The `Prediction` Class

The Prediction class represents the output of a DSPy program execution. It encapsulates the results produced by language model calls, typically containing fields that correspond to the output fields defined in a signature.

Program Structure Layer

The `Module` Class

The Module class is the fundamental base class from which all DSPy programs inherit. It provides the runtime infrastructure for composing language model calls, managing parameters, and executing programs.

Key Methods:

Sources: dspy/primitives/module.py:1-60

Predictor Discovery:

The named_predictors() method uses introspection to find all Predict instances within a module:

class MyProgram(dspy.Module):
    def __init__(self):
        super().__init__()
        self.qa = dspy.Predict("question -> answer")
        self.summarize = dspy.Predict("text -> summary")

program = MyProgram()
for name, predictor in program.named_predictors():
    print(name)  # 'qa', 'summarize'

Sources: dspy/primitives/module.py:1-45

Language Model Integration

The set_lm() method traverses the module's parameter tree to attach a language model to every Predict instance:

lm = dspy.LM("openai/gpt-4o-mini")
program.set_lm(lm)

This recursive propagation ensures that nested modules and chained predictors all share the same language model configuration.

Sources: dspy/primitives/module.py:45-55

Execution Flow

sequenceDiagram
    participant User
    participant Module
    participant Predict
    participant LM
    participant Prediction
    
    User->>Module: forward(**inputs)
    Module->>Predict: forward(**inputs)
    Predict->>LM: generate(input_fields, signature)
    LM-->>Predict: raw_output
    Predict->>Prediction: format(raw_output, signature)
    Prediction-->>Module: Prediction object
    Module-->>User: Program output
    
    Note over Predict: Validates and transforms<br/>LM output according<br/>to Signature fields

Signature-Based Input/Output Contract

DSPy programs use Signatures to define the contract between modules and language models. A signature specifies input fields and output fields, along with optional descriptions that guide the LM's behavior.

The Signature.with_updated_fields() method allows runtime modification of field metadata:

NewSig = MySig.with_updated_fields(
    "output_text", 
    desc="The translated French text"
)

Sources: dspy/signatures/signature.py:1-40

Extended Modules

RLM (Recursive Language Model)

The RLM class extends the base Module to provide sandboxed REPL-based code execution capabilities, enabling LLMs to programmatically explore large contexts through Python code:

rlm = RLM(
    sandbox="python",  # or "bash"
    timeout=30
)

Sources: dspy/predict/rlm.py:1-30

Embedder

The Embedder class provides a consistent interface for embedding generation with built-in batching and caching:

embedder = dspy.Embedder(
    model="openai/text-embedding-3-small",
    batch_size=200,
    caching=True
)

Sources: dspy/clients/embedding.py:1-50

Callback System

DSPy's callback system (dspy/utils/callback.py) enables observability and instrumentation of program execution:

graph LR
    A[Module Forward] -->|on_module_start| B[Callback Handler]
    B --> C[Execute]
    C -->|on_module_end| B

The CallbackHandler base class defines hooks for:

Callbacks can be attached either globally or per-component:

# Global callback
dspy.LM("gpt-3.5-turbo", callbacks=[LoggingCallback()])

# Local callback on specific component
lm(question="What is 2+2?", callbacks=[LoggingCallback()])

Sources: dspy/utils/callback.py:1-40

Component Hierarchy

graph TD
    BaseModule[BaseModule<br/>dspy.primitives]
    Module[Module<br/>extends BaseModule]
    Program[User Program<br/>extends Module]
    
    Predict[Predict<br/>Signature-driven predictor]
    ChainOfThought[ChainOfThought<br/>extends Predict]
    RLM[RLM<br/>extends Module]
    
    Example[Example<br/>Data primitive]
    Prediction[Prediction<br/>Output primitive]
    
    BaseModule --> Module
    Module --> Program
    Module --> Predict
    Predict --> ChainOfThought
    Module --> RLM
    
    Example --> Program
    Program --> Prediction

Key Design Principles

1. Separation of Concerns: Data (Example, Prediction), structure (Module), and execution (LM) are cleanly separated
2. Introspection: The module system supports dynamic discovery of contained predictors
3. Recursive Configuration: Language model and other settings propagate through nested modules
4. Serialization: All data primitives support conversion to JSON-compatible formats
5. Extensibility: Custom modules can inherit from Module and use the same infrastructure

Summary

DSPy's core architecture provides a composable framework where Example objects define inputs and labels, Module subclasses implement programs, Predict modules handle signature-driven LM interactions, and Prediction objects capture outputs. This architecture enables systematic optimization and evaluation of language model programs through a consistent, introspectable API.

Related Pages

Related topics: Core Architecture, Prediction Modules

Signatures System

Overview

The Signatures System is a core abstraction in DSPy that defines the structure of inputs and outputs for language model calls. A Signature describes what fields a module expects as input and what fields it will produce as output, effectively serving as a declarative contract between DSPy programs and language models.

Signatures enable DSPy to:

• Compile declarative LM calls into self-improving pipelines
• Separate input fields from output fields for proper data routing
• Provide rich metadata (descriptions) for each field that guides the LM's behavior
• Support dynamic field manipulation and signature transformation

Sources: dspy/signatures/signature.py:1-50

Core Concepts

InputField vs OutputField

Every field in a Signature is classified as either an InputField or an OutputField. This distinction is fundamental to DSPy's execution model:

class MySignature(dspy.Signature):
    question: str = dspy.InputField(desc="Input question")
    answer: str = dspy.OutputField(desc="Generated answer")

Sources: dspy/signatures/field.py:1-30

Field Descriptions

The desc parameter in InputField/OutputField provides natural language descriptions that are injected into prompts. Well-crafted descriptions significantly improve LM accuracy.

class TranslationSignature(dspy.Signature):
    english_text: str = dspy.InputField(desc="The text to translate, written in English")
    french_text: str = dspy.OutputField(desc="The translated text in French")

Signature Class

Creating Signatures

Signatures can be created in multiple ways:

Class-based definition:

class QASignature(dspy.Signature):
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

String-based definition:

qa_sig = make_signature("question -> answer")

Dictionary-based definition:

sig = make_signature({
    "question": (str, dspy.InputField()),
    "answer": (str, dspy.OutputField())
})

Sources: dspy/signatures/signature.py:200-280

Signature Methods

The Signature class provides several class methods for creating modified copies:

# Update instructions
NewSig = MySig.with_instructions("Translate to French.")

# Add a new field
EnhancedSig = MySig.append("confidence", dspy.OutputField(desc="Translation confidence"))

# Remove a field
SimplifiedSig = MySig.delete("source_language")

Sources: dspy/signatures/signature.py:30-120

Signature Structure and Fields

Fields Dictionary

Internally, a Signature maintains a fields dictionary that maps field names to their corresponding FieldInfo objects:

class Signature:
    fields: dict[str, FieldInfo]
    instructions: str

The fields are automatically categorized into inputs and outputs based on whether each field uses InputField or OutputField.

Type Handling

Signatures support flexible type annotations:

class CustomSig(dspy.Signature):
    # Basic string type
    text: str = dspy.InputField()
    
    # With custom type
    history: dspy.History = dspy.InputField()
    
    # List of custom types
    images: list[dspy.Image] = dspy.InputField()

When types are not explicitly provided, they default to str to maintain backward compatibility with existing programs.

Sources: dspy/signatures/utils.py:1-50

Using Signatures with Examples

The Example class works in conjunction with Signatures to manage training data and evaluation:

Marking Input Fields

# Mark which fields are inputs vs labels
example = dspy.Example(
    question="What is the capital of France?",
    answer="Paris"
).with_inputs("question")

# Access inputs and labels separately
inputs = example.inputs()      # Example({'question': '...'}) (input_keys={'question'})
labels = example.labels()     # Example({'answer': '...'}) (input_keys={'question'})

Converting Examples

# Convert to dictionary for serialization
data = example.toDict()

# Copy with overrides
new_example = example.copy(answer="Lyon")

Sources: dspy/primitives/example.py:50-150

Custom Types System

DSPy's Type system extends Signatures with custom data types. The base Type class requires implementations of the format() method.

Built-in Custom Types

Image Type

class Image(Type):
    url: str
    
    def format(self) -> list[dict[str, Any]]:
        return [{"type": "image_url", "image_url": {"url": self.url}}]

# Usage with signatures
class VQASignature(dspy.Signature):
    image: dspy.Image = dspy.InputField(desc="Image to analyze")
    question: str = dspy.InputField(desc="Question about the image")
    answer: str = dspy.OutputField(desc="Answer to the question")

Sources: dspy/adapters/types/base_type.py:1-50

History Type

class History(Type):
    messages: list[dict[str, Any]]
    
    def format(self) -> list[dict[str, Any]]:
        # Formats conversation history as a list of message dictionaries
        pass

# Usage for multi-turn conversations
class ChatSignature(dspy.Signature):
    question: str = dspy.InputField()
    history: dspy.History = dspy.InputField()
    answer: str = dspy.OutputField()

Sources: dspy/adapters/types/history.py:1-60

Architecture Diagram

graph TD
    A[Signature Definition] --> B[Field Resolution]
    B --> C{Field Type}
    C -->|InputField| D[Input Fields Dict]
    C -->|OutputField| E[Output Fields Dict]
    
    F[make_signature] -->|String format| G[_parse_signature]
    F -->|Dict format| H[Direct field assignment]
    
    I[Example with Signature] --> J[with_inputs]
    J --> K[Inputs]
    J --> L[Labels]
    
    M[Custom Types] --> N[Type.format]
    N --> O[Image.format]
    N --> P[History.format]
    
    D --> Q[Predict Module]
    E --> Q
    Q --> R[LM Call]

Field Modification Workflow

graph LR
    A[Original Signature] --> B[with_updated_fields]
    A --> C[prepend/append]
    A --> D[delete]
    
    B --> E[Updated field metadata]
    C --> F[New field added]
    D --> G[Field removed]
    
    E --> H[Fresh Signature Instance]
    F --> H
    G --> H

Advanced Usage

Composing Multiple Signatures

Signatures can be dynamically combined using the insertion methods:

class BaseSignature(dspy.Signature):
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

# Add context field
ExtendedSig = BaseSignature.prepend(
    "context", 
    dspy.InputField(desc="Additional context")
)

# Add metadata output
FinalSig = ExtendedSig.append(
    "confidence", 
    dspy.OutputField(desc="Confidence score")
)

Custom Type Extraction

The Type system automatically extracts custom types from nested annotations:

class MyType(dspy.Type):
    @classmethod
    def extract_custom_type_from_annotation(cls, annotation):
        # Detects custom types in list[dict[str, MyType]]
        pass

This enables DSPy to handle complex nested structures with custom types throughout the signature system.

Sources: dspy/adapters/types/base_type.py:40-80

Best Practices

1. Provide Clear Descriptions: Always use descriptive desc parameters for fields to improve LM performance
2. Explicit Input/Output Separation: Use .with_inputs() to clearly mark which fields are inputs
3. Leverage Type Annotations: Use custom types like Image and History for structured data
4. Immutability: Remember that signature modification methods return new instances; originals are unchanged

Related Components

Summary

The Signatures System is the foundational abstraction in DSPy that defines the interface between programs and language models. By separating inputs from outputs and providing rich metadata through field descriptions, Signatures enable DSPy's compilation and optimization capabilities. The system supports flexible creation methods, dynamic modification, and integration with custom types for complex applications.

Related Pages

Related topics: Core Architecture, Prediction Modules

Module System

The DSPy Module System provides a declarative, composable framework for building language model programs. It serves as the foundational abstraction layer that enables automatic optimization of both prompts and model weights through DSPy's teleprompter system.

Overview

DSPy's module system extends Python's native class mechanism to create reusable, optimizable building blocks for LLM pipelines. Unlike traditional Python modules, DSPy modules represent computational stages that can be automatically configured, optimized, and composed into larger programs.

graph TD
    A[dspy.Module] --> B[dspy.Predict]
    A --> C[dspy.ChainOfThought]
    A --> D[dspy.MultiChainComparison]
    A --> E[User-defined Module]
    B --> F[Signature]
    B --> G[Language Model]
    E --> H[Other DSPy Modules]

Sources: dspy/primitives/module.py:1-50

Core Components

Module Base Class

The dspy.Module class serves as the foundation for all DSPy components. It extends Python's nn.Module to integrate seamlessly with neural network training paradigms while adding DSPy-specific functionality.

Sources: dspy/primitives/module.py:60-95

Predictor Parameters

DSPy uses Predict and Parameter classes to define trainable components within modules. These parameters encapsulate the signature and configuration for LLM calls.

class Parameter(dspy.BaseModule):
    """Trainable parameter that wraps a Predict module."""
    
    def __init__(self, signature, **kwargs):
        self.signature = signature
        self.kwargs = kwargs

Sources: dspy/predict/parameter.py:1-20

Module Hierarchy

graph TD
    A[nn.Module] --> B[dspy.BaseModule]
    B --> C[dspy.Module]
    C --> D[dspy.Predict]
    C --> E[User Module]
    D --> F[ChainOfThought]
    D --> G[Retry]
    E --> H[Composite Programs]
    F --> I[MultiChainComparison]

BaseModule

BaseModule provides the essential interface that all DSPy components implement. It establishes the contract for modules that can be used within DSPy programs.

Module

The Module class adds advanced functionality including:

• Automatic predictor discovery: Scans submodules to find all Predict instances
• Language model management: Centralized LM configuration across the module tree
• Serialization support: Ability to save and load compiled programs

Sources: dspy/primitives/module.py:40-80

Predictor System

Predict Class

Predict is the primary building block for LLM interactions. It binds a Signature to a language model call.

class Predict(Module):
    def __init__(self, signature, **config):
        self.signature = dspy.Signature(signature)
        self.config = config

Sources: dspy/predict/predict.py:1-60

Predictor Discovery

Modules can automatically discover their predictor submodules:

class MyProgram(dspy.Module):
    def __init__(self):
        super().__init__()
        self.qa = dspy.Predict("question -> answer")
        self.summarize = dspy.Predict("text -> summary")

program = MyProgram()
for name, predictor in program.named_predictors():
    print(f"{name}: {predictor.signature}")
# Output:
# qa: question -> answer
# summarize: text -> summary

Sources: dspy/primitives/module.py:50-65

Language Model Integration

Setting Language Models

Language models can be set at different levels of granularity:

# Set LM for entire module tree
program.set_lm(dspy.LM("openai/gpt-4o-mini"))

# Set LM for specific predictor
program.qa.lm = dspy.LM("anthropic/claude-3")

# Retrieve configured LM
lm = program.get_lm()

Sources: dspy/primitives/module.py:80-95

Signature System

Signatures define the schema for module inputs and outputs. They are central to the module system's type safety and documentation.

class Signature:
    @classmethod
    def with_instructions(cls, instructions: str) -> type["Signature"]:
        """Create new Signature with custom instructions."""
        return Signature(cls.fields, instructions)
    
    @classmethod
    def with_updated_fields(cls, name: str, **kwargs) -> type["Signature"]:
        """Update field metadata (descriptions, types)."""
        fields_copy = deepcopy(cls.fields)
        fields_copy[name].json_schema_extra.update(kwargs)
        return Signature(fields_copy, cls.instructions)

Sources: dspy/signatures/signature.py:1-50

Creating Custom Modules

Custom modules extend dspy.Module and compose existing predictors:

import dspy

class RAG(dspy.Module):
    def __init__(self, num_passages=5):
        super().__init__()
        self.retrieve = dspy.Retrieve(k=num_passages)
        self.generate_answer = dspy.ChainOfThought("context, question -> answer")
    
    def forward(self, question):
        context = self.retrieve(query=question).passages
        return self.generate_answer(context=context, question=question)

Workflow Diagram

graph LR
    A[Define Module] --> B[Compose Predictors]
    B --> C[Set Language Model]
    C --> D[Compile with Teleprompter]
    D --> E[Optimized Program]
    E --> F[Execute on new inputs]

Module Configuration Options

Compilation and Optimization

Modules interface with DSPy's teleprompter system for automatic optimization:

from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(metric=my_metric)
compiled_program = optimizer.compile(
    student=MyProgram(),
    trainset=training_examples
)

The module system provides hooks for:

• Prompt optimization: Automatically improving instructions
• Weight optimization: Fine-tuning model weights via BootstrapFinetune
• Demonstration selection: Choosing optimal few-shot examples

Sources: dspy/primitives/module.py:1-30

See Also

• Signature System - Input/output schema definitions
• Predict Module - Core LLM interaction component
• Teleprompters - Optimization strategies
• Evaluator - Metrics and evaluation

Related Pages

Related topics: Language Model Clients

Adapters

Adapters in DSPy are core components responsible for formatting and structuring messages sent to Language Models (LMs). They act as an intermediary layer between DSPy's internal data structures and the API formats expected by various LLM providers.

Overview

When a DSPy module executes a signature (such as dspy.Predict or dspy.ChainOfThought), the adapter transforms the structured input/output fields into a format the LM can understand, and then parses the LM's response back into structured data.

graph LR
    A[Signature Fields] --> B[Adapter]
    B --> C[LM API Format]
    C --> D[LM Response]
    D --> B
    B --> E[Structured Output]

Key Responsibilities:

Sources: dspy/adapters/__init__.py:1-50

Architecture

Base Adapter

All adapters inherit from dspy.adapters.base.BaseAdapter, which defines the common interface:

classDiagram
    class BaseAdapter {
        +format(signature, demos, inputs) List[Message]
        +parse(signature, completion) dict
        +format_extractors(signature) str
    }
    
    class ChatAdapter {
        +format(signature, demos, inputs)
        +parse(signature, completion)
    }
    
    class JSONAdapter {
        +format(signature, demos, inputs)
        +parse(signature, completion)
    }
    
    class XMLAdapter {
        +format(signature, demos, inputs)
        +parse(signature, completion)
    }
    
    BaseAdapter <|-- ChatAdapter
    BaseAdapter <|-- JSONAdapter
    BaseAdapter <|-- XMLAdapter

BaseAdapter Methods:

Sources: dspy/adapters/base.py:1-100

Adapter Selection

Adapters can be selected at different levels:

1. Global Configuration: Set via dspy.configure(adapter=...)
2. Per-Module: Pass adapter parameter to module constructors
3. Default: ChatAdapter is used when no adapter is specified

# Global configuration
dspy.configure(adapter=dspy.JSONAdapter())

# Per-module configuration
predict = dspy.Predict("question -> answer", adapter=dspy.XMLAdapter())

Built-in Adapters

ChatAdapter

The default adapter that formats messages in OpenAI's chat format. It uses special delimiters to separate input fields, demonstrations, and output fields.

Characteristics:

Message Structure:

graph TD
    A[System Message] --> B[Instructions]
    B --> C[Input Fields]
    C --> D[Demos Section]
    D --> E[Output Fields Request]
    E --> F[User Message]

Sources: dspy/adapters/chat_adapter.py:1-150

JSONAdapter

Formats outputs as JSON, instructing the LM to respond with valid JSON that can be directly parsed.

Characteristics:

Parsing Logic:

# The adapter looks for JSON blocks in the completion
json_match = re.search(r"```json\s*(.*?)\s*```", completion, re.DOTALL)
if json_match:
    return json.loads(json_match.group(1))

Sources: dspy/adapters/json_adapter.py:1-100

XMLAdapter

Uses XML-style tags to delimit fields, providing a structured format similar to HTML/XML.

Characteristics:

Example Output Format:

<answer>
  The capital of France is Paris.
</answer>

Sources: dspy/adapters/xml_adapter.py:1-100

TwoStepAdapter

A specialized adapter that breaks the response into two steps:

1. Thought Step: The LM first provides reasoning or analysis
2. Response Step: The LM provides the final structured output

This adapter is particularly useful with dspy.ChainOfThought modules.

Sources: dspy/adapters/two_step_adapter.py:1-100

BamlAdapter

Adapter specifically designed for BAML (Business Application Markup Language) output formatting, used with BAML-compiled signatures.

Sources: dspy/adapters/baml_adapter.py:1-50

Custom Types

DSPy adapters support custom types that can be used in signatures:

Image Type

The Image type allows passing images to multimodal models:

import dspy
from dspy.adapters.types import Image

class VisionSignature(dspy.Signature):
    image: Image = dspy.InputField()
    description: str = dspy.OutputField()

predict = dspy.Predict(VisionSignature)
result = predict(image=Image(url="https://example.com/image.jpg"))

Image Formats Supported:

Sources: dspy/adapters/types/image.py:1-80

History Type

The History type allows passing conversation history:

import dspy
from dspy.adapters.types import History

class ConversationSignature(dspy.Signature):
    history: History = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

history = History(messages=[
    {"question": "What is 2+2?", "answer": "4"},
    {"question": "What is 3+3?", "answer": "6"},
])

predict = dspy.Predict(ConversationSignature)
result = predict(history=history, question="What is 5+5?")

Sources: dspy/adapters/types/history.py:1-100

Custom Type Identifiers

When custom types are used, adapters use special identifiers:

These identifiers help adapters extract and process custom type content from LM responses.

Sources: dspy/adapters/types/base_type.py:1-50

Workflow

sequenceDiagram
    participant User
    participant Module
    participant Adapter
    participant LM
    participant Parser

    User->>Module: Forward pass with inputs
    Module->>Adapter: format(signature, demos, inputs)
    Adapter->>Adapter: Structure messages
    Adapter->>LM: Send formatted messages
    LM-->>Adapter: Return completion
    Adapter->>Parser: parse(signature, completion)
    Parser->>Parser: Extract structured fields
    Parser-->>Module: Return dict output
    Module-->>User: Return structured output

Configuration Options

Adapter Parameters

Setting Adapters

import dspy

# Method 1: Global configuration
dspy.configure(adapter=dspy.JSONAdapter())

# Method 2: Per-module
predict = dspy.Predict(
    "question -> answer",
    adapter=dspy.XMLAdapter()
)

# Method 3: At initialization
dspy.settings.adapter = dspy.ChatAdapter()

Best Practices

1. Use JSONAdapter when you need strict structured output parsing
2. Use ChatAdapter for general-purpose applications (default)
3. Use XMLAdapter for nested or hierarchical data structures
4. Use TwoStepAdapter with ChainOfThought for reasoning-intensive tasks
5. Use custom types (Image, History) for multimodal or conversational applications

Extending Adapters

To create a custom adapter, inherit from BaseAdapter:

from dspy.adapters import BaseAdapter

class MyCustomAdapter(BaseAdapter):
    def format(self, signature, demos, inputs):
        # Custom formatting logic
        messages = []
        # ... format messages ...
        return messages
    
    def parse(self, signature, completion):
        # Custom parsing logic
        return {"output_field": parsed_value}

Summary

Adapters are a fundamental part of DSPy's architecture, providing a flexible abstraction layer between DSPy's declarative signatures and the various API formats expected by different LMs. By supporting multiple built-in adapters and allowing custom extensions, DSPy can work seamlessly with any language model while maintaining a consistent programming interface.

Related Pages

Related topics: Adapters

Language Model Clients

Overview

The Language Model (LM) client system in DSPy provides a unified interface for interacting with various LLM providers. This abstraction layer enables developers to swap between different LLM backends (OpenAI, Anthropic, Azure, local models, etc.) without modifying application code.

Architecture

The LM client architecture follows a layered design pattern with a base class defining the common interface and provider-specific implementations extending it.

graph TD
    A[Application Code] --> B[dspy.LM Interface]
    B --> C[OpenAI Client]
    B --> D[LiteLLM Client]
    B --> E[Databricks Client]
    B --> F[Local LM Client]
    C --> G[OpenAI API]
    D --> H[40+ LLM Providers]
    E --> I[Databricks Endpoint]
    F --> J[Ollama / vLLM]

Core Components

BaseLM Abstract Class

The BaseLM class defines the abstract interface that all LM clients must implement. This ensures consistency across different providers.

Sources: dspy/clients/base_lm.py

LM Factory Class

The main LM class serves as a factory that instantiates the appropriate client based on the model identifier format.

Supported model formats include:

• provider/model-name (e.g., openai/gpt-4, anthropic/claude-3)
• provider/model-name@timestamp (e.g., openai/gpt-4o-mini-2024-07-18)
• provider/model-name:variant (e.g., openai/gpt-4-turbo:n-1106)

# Example instantiation patterns
lm = dspy.LM("openai/gpt-4o-mini")
lm = dspy.LM("anthropic/claude-3-opus-20240229")
lm = dspy.LM("azure/gpt-4o", api_base="https://...", api_key="...")

Sources: dspy/clients/lm.py:1-100

LM Client Implementations

OpenAI Client

The OpenAI client provides direct integration with OpenAI's API endpoints.

Sources: dspy/clients/openai.py

LiteLLM Client

The LiteLLM client wraps the litellm library, providing access to over 40 LLM providers through a unified interface.

# Using LiteLLM-backed LM
lm = dspy.LM("anthropic/claude-3-sonnet-20240229", 
             api_key=os.getenv("ANTHROPIC_API_KEY"))

Supported providers include:

• Anthropic
• Azure OpenAI
• AWS Bedrock
• Google Vertex AI
• Cohere
• Mistral
• Together AI

Sources: dspy/clients/_litellm.py

Databricks Client

The Databricks client integrates with Databricks Model Serving endpoints for enterprise deployments.

from dspy.clients.databricks import Databricks

lm = Databricks(
    model="databricks-meta-llama-3-70b-instruct",
    Databricks_host="https://your-workspace.cloud.databricks.com",
    Databricks_token="your-token",
)

Sources: dspy/clients/databricks.py

Local LM Client

The Local LM client supports self-hosted models through Ollama or vLLM.

from dspy.clients.lm_local import LocalLM

# Using Ollama
lm = LocalLM(model="llama3", base_url="http://localhost:11434")

# Using vLLM
lm = LocalLM(model="mistralai/Mistral-7B-Instruct-v0.2", base_url="http://localhost:8000")

Sources: dspy/clients/lm_local.py

Caching System

DSPy implements a response caching mechanism to reduce API costs and improve latency for repeated requests.

graph LR
    A[Request] --> B{Cache Hit?}
    B -->|Yes| C[Return Cached Response]
    B -->|No| D[Execute LM Call]
    D --> E[Store in Cache]
    E --> F[Return Response]

Cache Configuration

The cache uses a hash-based key generation strategy:

1. Serialize the input prompt
2. Serialize adapter inputs and configuration
3. Combine and hash to create cache key

Sources: dspy/clients/cache.py

Callback System

Callbacks enable monitoring and logging of LM interactions throughout the application.

graph TD
    A[LM Call] --> B[Callback Manager]
    B --> C[on_module_start]
    B --> D[on_module_end]
    B --> E[on_lm_start]
    B --> F[on_lm_end]

Built-in Callbacks

class LoggingCallback(Handler):
    def on_lm_start(self, call_id, instance, inputs):
        print(f"LM is called with inputs: {inputs}")
    
    def on_lm_end(self, call_id, instance, outputs):
        print(f"LM is finished with outputs: {outputs}")

Sources: dspy/utils/callback.py

Configuration Options

Global Configuration

Configure LM settings globally using dspy.configure():

import dspy

dspy.configure(
    lm=dspy.LM("openai/gpt-4o-mini"),
    rm=dspy.Retrieve(k=3),
    max_tokens=512,
    temperature=0.1,
)

Per-Instance Configuration

Override settings at the component level:

cot = dspy.ChainOfThought(
    "question -> answer",
    temperature=0.5,
    max_tokens=256
)

Configuration Precedence

Usage Patterns

Basic Usage

import dspy

# Initialize LM
lm = dspy.LM("openai/gpt-4o-mini")

# Direct call
result = lm("What is the capital of France?")
print(result)

With Modules

import dspy

lm = dspy.LM("openai/gpt-4o-mini")

# Set LM for a module
program = dspy.Predict("question -> answer")
program.set_lm(lm)

# Execute
result = program(question="What is 2+2?")

Batch Processing

import dspy

lm = dspy.LM("openai/gpt-4o-mini", max_tokens=100)

# Multiple calls tracked in history
for q in questions:
    lm(q)

# Inspect all calls
for item in lm.inspect_history():
    print(item)

Error Handling

The LM client system handles various error conditions:

Best Practices

1. Use environment variables for API keys to avoid hardcoding credentials
2. Enable caching for development and testing to reduce API costs
3. Set appropriate max_tokens to prevent overly long responses
4. Use callbacks for production monitoring and debugging
5. Configure timeouts appropriately for your use case (default: 120s)

See Also

• Signature System - How DSPy defines input/output schemas
• Modules - Using LMs within DSPy modules
• Teleprompters - Optimizing prompts with LMs
• Retrievers - Combining LMs with retrieval systems

Related Pages

Related topics: Module System, Agent Modules

Prediction Modules

Overview

Prediction Modules are the core building blocks in DSPy for executing language model calls. They encapsulate the logic for invoking language models, handling inputs/outputs defined by Signatures, and providing various reasoning and optimization strategies. All predictors inherit from the base Predict class and can be composed within dspy.Module objects to build complex AI pipelines.

Key responsibilities:

• Execute language model calls with specified inputs
• Parse and validate model outputs according to Signature definitions
• Support multiple reasoning strategies (chain-of-thought, multi-chain comparison, etc.)
• Integrate with DSPy's teleprompters for automatic prompt optimization
• Track execution history for debugging and introspection

Sources: dspy/predict/predict.py

Related Pages

Related topics: Prediction Modules

Agent Modules

Overview

Agent Modules in DSPy are specialized modules that enable language models to interact with external tools, execute code, and perform multi-step reasoning tasks. Unlike standard Predict modules that generate direct outputs, Agent Modules implement agentic behaviors where the LM can plan, take actions (such as calling tools or executing code), observe results, and iteratively refine their approach until reaching a final answer.

DSPy provides several agent implementations that share a common architectural pattern: they use a signature to define input/output fields, maintain a trajectory of their reasoning and actions, and leverage code interpreters or tool execution systems to perform computations that the language model cannot do directly.

Sources: dspy/primitives/module.py:1-50

Architecture

Core Components

Agent Modules in DSPy are built on a layered architecture:

graph TD
    A[Agent Module<br/>e.g., ReAct, Program of Thought] --> B[Signature<br/>Input/Output Field Definitions]
    A --> C[Tool Registry<br/>Available Actions]
    A --> D[History/Trajectory<br/>Reasoning & Actions]
    B --> E[Code Interpreter<br/>Execution Backend]
    C --> F[Tool Execution<br/>Function Calls]
    D --> G[Observation Processing<br/>Feedback Loop]
    E --> H[Python Interpreter<br/>Sandboxed Execution]

Agent Base Pattern

All DSPy agent implementations follow a similar control flow:

1. Initialization: Set up signature, tools, and configuration parameters
2. Loop Execution: Iterate until max iterations or final output received
3. Thought Generation: LM produces reasoning and next action
4. Action Execution: Tools or code interpreters execute the proposed action
5. Observation Processing: Results are appended to trajectory
6. Termination: Return final prediction with full trajectory

Sources: dspy/predict/react.py:1-80

ReAct Agent

The ReAct agent implements the Reasoning + Acting pattern, combining deliberate reasoning with tool usage. It allows language models to use external tools to gather information and complete tasks.

Signature Configuration

class ReAct(dspy.Module):
    def __init__(
        self,
        signature,
        max_iters: int = 10,
        tools: list[Tool | Callable] = None,
    ):

Sources: dspy/predict/react.py:50-90

Tool Integration

The ReAct agent automatically generates instructions that inform the LM about available tools:

inputs = ", ".join([f"`{k}`" for k in signature.input_fields.keys()])
outputs = ", ".join([f"`{k}`" for k in signature.output_fields.keys()])
instr = [
    f"You are an Agent. In each episode, you will be given the fields {inputs} as input.",
    f"Your goal is to use one or more of the supplied tools to collect any necessary information for producing {outputs}.",
]

Tools are converted to a Tool registry and made available by name for the agent to call.

Sources: dspy/predict/react.py:70-85

Usage Example

def get_weather(city: str) -> str:
    return f"The weather in {city} is sunny."

react = dspy.ReAct(signature="question -> answer", tools=[get_weather])
pred = react(question="What is the weather in Tokyo?")

Code Interpreter Agents

DSPy provides agents that can execute Python code to perform computations. These agents use a sandboxed Python interpreter to run code safely.

Python Interpreter

The PythonInterpreter class provides the execution backend for code-based agents:

The interpreter extracts variables after each execution, making them available for subsequent code blocks or tool calls.

Sources: dspy/primitives/python_interpreter.py:1-60

Code Interpreter Primitive

The CodeInterpreter class wraps the Python interpreter with additional utilities:

class CodeInterpreter:
    def __init__(self, max_output_chars: int = 2000):
        self.max_output_chars = max_output_chars
    
    def execute(self, code: str, variables: dict) -> Any:
        # Execute code and return result

Sources: dspy/primitives/code_interpreter.py:1-50

Reflexion Language Model (RLM) Agent

The RLM agent implements a Reflexion-style approach where the agent maintains a history of reasoning, code execution, and observations to iteratively improve its output.

Execution Flow

graph TD
    A[Start with Input] --> B[Generate Reasoning]
    B --> C[Generate Code]
    C --> D[Execute Code in Interpreter]
    D --> E{Error?}
    E -->|Yes| F[Record Error in History]
    F --> G[Generate Fix Reasoning]
    G --> C
    E -->|No| H{Is Final Output?}
    H -->|No| B
    H -->|Yes| I[Return Prediction with Trajectory]

Trajectory Tracking

The RLM agent maintains a REPLHistory that tracks all iterations:

return Prediction(
    **parsed_outputs,
    trajectory=[e.model_dump() for e in final_history],
    final_reasoning=pred.reasoning,
)

Each history entry contains:

• reasoning: The agent's thought process
• code: The code that was executed
• output: The result or error message

Sources: dspy/predict/rlm.py:100-130

Result Processing

The agent processes final outputs by parsing structured results:

def _process_final_output(self, result: FinalOutput, output_field_names: list[str]):
    parsed_outputs, error = self._process_final_output(result, output_field_names)
    
    if error:
        return history.append(reasoning=pred.reasoning, code=code, output=error)

Program of Thought

The Program of Thought agent combines natural language reasoning with programmatic code execution, allowing the agent to offload complex calculations to executed code while maintaining reasoning chains.

Architecture

graph LR
    A[Input Question] --> B[Reasoning Module]
    B --> C[Code Generation]
    C --> D[Python Interpreter]
    D --> E[Output Extraction]
    E --> F{More Steps Needed?}
    F -->|Yes| B
    F -->|No| G[Final Answer]

Tool Definition

DSPy uses a Tool class to wrap functions and make them available to agents:

tool = Tool(
    name="function_name",
    func=my_function,
    desc="Description for LLM"
)

Tools can be:

• Plain Python functions
• Decorated with the @Tool decorator
• Already instantiated Tool objects

Configuration Parameters

Common Agent Parameters

History and Trajectory

Agents maintain execution history which can be inspected:

agent = dspy.ReAct(signature="question -> answer", tools=[my_tool])
result = agent(question="What is 2+2?")

# Access trajectory
for step in result.trajectory:
    print(f"Thought: {step['reasoning']}")
    print(f"Action: {step['code']}")
    print(f"Observation: {step['output']}")

Integration with DSPy Module System

All agent modules inherit from dspy.Module, giving them access to:

• named_predictors(): List all Predict instances
• set_lm(lm): Set language model for all predictors
• get_lm(): Retrieve the configured language model
• inspect_history(n): Display recent LM call history

class MyAgent(dspy.Module):
    def __init__(self):
        super().__init__()
        self.predictor = dspy.ReAct("question -> answer", tools=[...])
    
    def forward(self, question):
        return self.predictor(question=question)

Sources: dspy/primitives/module.py:50-120

Best Practices

Tool Design

1. Clear Function Names: Use descriptive names that indicate the tool's purpose
2. Type Annotations: Add type hints for better LLM understanding
3. Docstrings: Provide clear descriptions of what the tool does
4. Return Values: Return string or serializable types for consistency

Agent Configuration

1. Set Appropriate max_iters: Too few iterations may prevent task completion; too many wastes resources
2. Use Temperature Control: Lower temperatures (0.0-0.3) for deterministic tasks
3. Inspect Trajectories: Use inspect_history() for debugging agent behavior

Error Handling

Agents should be configured with error handling for:

• Code execution failures
• Tool invocation errors
• Maximum iteration limits reached
• Invalid output parsing

Related Pages

Related topics: Signatures System

Optimizers (Teleprompt)

Overview

In DSPy, Optimizers (also referred to as Teleprompters) are compilation strategies that automatically optimize the prompts and/or weights of your LM programs. They take a raw program with a signature and training data, then search for improved instructions and/or demonstrations to include in the program's prompts.

The teleprompter system enables declarative, data-driven optimization of LLM pipelines without manual prompt engineering. Instead of crafting prompts by hand, you define:

1. Your program structure (modules, signatures)
2. A metric function to evaluate quality
3. A training set of examples

The optimizer then compiles your program by finding optimal combinations of instructions, demonstrations, and (optionally) fine-tuned weights.

Architecture

The optimizer system follows a hierarchical architecture with a base Teleprompter abstract class and specialized subclasses for different optimization strategies:

graph TD
    A[Teleprompter Base] --> B[Prompt Optimizers]
    A --> C[Weight Optimizers]
    A --> D[Meta-Optimizers]
    
    B --> B1[BootstrapFewShot]
    B --> B2[MIPROv2]
    B --> B3[COPRO]
    B --> B4[GEPA]
    B --> B5[GRPO]
    B --> B6[BootstrapFewShotWithRandomSearch]
    
    C --> C1[BootstrapFinetune]
    
    D --> D1[BetterTogether]
    
    A --> E[Ensemble]
    E --> E1[Ensemble]

Base Class: Teleprompter

All optimizers inherit from the abstract Teleprompter class defined in dspy/teleprompt/teleprompt.py. The base interface requires implementing a compile() method.

Compile Interface

def compile(self, student: Module, *, trainset: list[Example], ...) -> Module:

Prompt Optimizers

Prompt optimizers focus on improving the instructions and demonstrations in your program's prompts without modifying model weights.

BootstrapFewShot

BootstrapFewShot is the foundational prompt optimizer that composes demonstrations (examples) into a predictor's prompt. These demos come from two sources:

1. Labeled examples from the training set
2. Bootstrapped demos generated by running the program with the LM

Key Features:

• Each bootstrap round copies the LM with a new rollout_id at temperature=1.0 to bypass caches and gather diverse traces
• Supports multi-round bootstrapping for complex multi-step programs

Sources: dspy/teleprompt/bootstrap.py:20-45

teleprompter = BootstrapFewShot(
    metric=your_metric,
    metric_threshold=0.5,        # Minimum score to accept bootstrapped demos
    max_bootstrapped_demos=4,      # Maximum bootstrap examples per predictor
    max_labeled_demos=16,          # Maximum labeled examples from trainset
    max_rounds=1,                  # Number of bootstrap rounds
    max_errors=4,                 # Max consecutive errors before stopping
)
compiled = teleprompter.compile(student, trainset=trainset)

BootstrapFewShotWithRandomSearch

This optimizer extends BootstrapFewShot by adding random search over different demonstration configurations. It explores multiple bootstrap configurations and selects the best performing one.

Sources: dspy/teleprompt/random_search.py

MIPROv2 (Multi-Instruction Prompt Optimization)

MIPROv2 optimizes both instructions and demonstrations using Bayesian optimization. It treats the instruction template and demonstration selection as hyperparameters to optimize.

Key Characteristics:

• Uses Bayesian optimization to efficiently search the space of prompts
• Supports fine-grained control over instruction generation
• Can use a separate prompt model for generating candidate instructions

Sources: dspy/teleprompt/mipro_optimizer_v2.py

from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(
    metric=metric,
    num_trials=100,           # Number of optimization trials
    max_bootstrapped_demos=4,
    minibend_size=50,
    auto="medium",           # Automatic complexity control
)
compiled = optimizer.compile(student, trainset=trainset, valset=valset)

COPRO (Coordinate Prompt Optimization)

COPRO generates new prompt instructions iteratively, using a prompt model to propose variations based on the history of previous prompts.

Algorithm Parameters:

Sources: dspy/teleprompt/copro_optimizer.py:20-35

from dspy.teleprompt import COPRO

teleprompter = COPRO(
    prompt_model=prompt_model,
    metric=metric,
    breadth=10,
    depth=3,
    init_temperature=1.4,
)
compiled = teleprompter.compile(program, trainset=trainset)

GEPA (Generalized Evolutionary Prompt Algorithm)

GEPA implements reflective prompt evolution using evolutionary strategies. It can outperform reinforcement learning approaches for prompt optimization.

Sources: dspy/teleprompt/gepa/gepa.py

from dspy.teleprompt import GEPA

optimizer = GEPA(
    metric=metric,
    auto="medium",            # Automatic complexity control
    num_trials=50,
)
compiled = optimizer.compile(student, trainset=trainset)

GRPO (Group Relative Policy Optimization)

GRPO applies group relative policy optimization for instruction tuning. It uses a collection of candidate prompts and selects based on relative performance rankings.

Sources: dspy/teleprompt/grpo.py

Weight Optimizers

Weight optimizers fine-tune the weights of the language model itself (or adapter weights) to improve program performance.

BootstrapFinetune

BootstrapFinetune uses bootstrapped demonstrations to fine-tune the language model weights. Unlike prompt-only optimization, this method modifies the actual model parameters.

Requirements:

• Student programs must have LMs explicitly set via set_lm() (not relying on global dspy.settings.lm)
• Requires a training set with high-quality demonstrations

Sources: dspy/teleprompt/bootstrap_finetune.py

from dspy.teleprompt import BootstrapFinetune

optimizer = BootstrapFinetune(metric=metric)
student.set_lm(lm)  # Required: explicit LM setting
compiled = optimizer.compile(student, trainset=trainset)

Meta-Optimizers

Meta-optimizers combine multiple optimization strategies into a unified compilation pipeline.

BetterTogether

BetterTogether is a meta-optimizer that orchestrates multiple optimizers in sequence. It separates optimization into distinct phases:

• Prompt Optimization Phase (p): Optimizes instructions and demonstrations
• Weight Optimization Phase (w): Fine-tunes model weights

Execution Strategy:

The optimizer uses a strategy string to define execution order:

graph LR
    A[Input Program] --> B["Strategy: 'p -> w'"]
    B --> C[Prompt Optimizer<br/>e.g., GEPA/MIPROv2]
    C --> D[Weight Optimizer<br/>BootstrapFinetune]
    D --> E[Compiled Program]

Sources: dspy/teleprompt/bettertogether.py:1-50

from dspy.teleprompt import BetterTogether, GEPA, BootstrapFinetune

# Combine GEPA for prompt optimization with BootstrapFinetune for weights
optimizer = BetterTogether(
    metric=metric,
    p=GEPA(metric=metric, auto="medium"),
    w=BootstrapFinetune(metric=metric),
)

student.set_lm(lm)  # Required for weight optimization
compiled = optimizer.compile(
    student,
    trainset=trainset,
    valset=valset,
    strategy="p -> w",  # Execute prompt optimization, then weight optimization
)

Optimizer-Specific Arguments:

You can pass custom arguments to individual optimizers via optimizer_compile_args:

compiled = optimizer.compile(
    student,
    trainset=trainset,
    valset=valset,
    strategy="p -> w",
    optimizer_compile_args={
        "p": {"num_trials": 10, "max_bootstrapped_demos": 8},
    }
)

Ensemble

The Ensemble teleprompter combines predictions from multiple compiled programs using voting or other aggregation strategies.

Sources: dspy/teleprompt/ensemble.py

from dspy.teleprompt import Ensemble

# Combine multiple compiled programs
ensemble = Ensemble(metric=metric)
combined = ensemble.compile(
    [program1, program2, program3],
    trainset=trainset,
)

Bootstrap Tracing

BootstrapFewShot and related optimizers rely on bootstrap_trace.py to collect demonstrations by executing programs and recording successful traces.

Sources: dspy/teleprompt/bootstrap_trace.py

Trace Collection Process:

1. Execute the student program on each training example
2. Record input/output pairs for each predictor
3. Accept traces that exceed the metric threshold
4. Store demonstrations for inclusion in compiled prompts

Optimization Workflow

The typical workflow for using optimizers in DSPy:

graph TD
    A[Define Program<br/>dspy.Module] --> B[Prepare Training Set<br/>list of Examples]
    B --> C[Define Metric<br/>evaluation function]
    C --> D[Select Optimizer<br/>BootstrapFewShot/MIPROv2/etc.]
    D --> E[Compile Program<br/>optimizer.compile]
    E --> F[Evaluate on Test Set<br/>dspy.Evaluate]
    
    subgraph "During Compilation"
        G[Generate Candidates<br/>instructions/demos/weights]
        H[Evaluate Candidates<br/>using metric]
        I[Select Best<br/>update program]
        G --> H --> I
    end
    
    F --> J{H满意?}
    J -->|Yes| K[Deploy Program]
    J -->|No| L[Adjust Parameters<br/>retry with different config]
    L --> D

Choosing an Optimizer

Best Practices

1. Start with BootstrapFewShot: It's the simplest and often effective baseline for getting started.

1. Use validation sets: When available, provide a valset parameter to allow the optimizer to select the best program among iterations.

1. Set LMs explicitly for weight optimization: Call student.set_lm(lm) before compiling with optimizers like BootstrapFinetune.

1. Monitor metric threshold: Adjust metric_threshold to control the quality of bootstrapped demonstrations.

1. Consider multi-phase optimization: For best results, use BetterTogether to combine prompt and weight optimization.

Module Exports

All optimizers are exported from dspy.teleprompt:

from dspy.teleprompt import (
    # Base and utilities
    Teleprompter,
    BootstrapFewShot,
    BootstrapFewShotWithRandomSearch,
    
    # Prompt optimizers
    MIPROv2,
    COPRO,
    GEPA,
    GRPO,
    
    # Weight optimizers
    BootstrapFinetune,
    
    # Meta-optimizers
    BetterTogether,
    
    # Ensemble
    Ensemble,
)

See Also

• Signatures - How to define input/output schemas for modules
• Modules - Building programs with Predict, ChainOfThought, and other modules
• Evaluation - Evaluating compiled programs
• Examples - Working with Example objects

Doramagic Pitfall Log

Doramagic extracted 16 source-linked risk signals. Review them before installing or handing real data to the project.

1. Security or permission risk: PythonInterpreter: paths containing commas are silently misparsed by Deno's --allow-read

• Severity: high
• Finding: Security or permission risk is backed by a source signal: PythonInterpreter: paths containing commas are silently misparsed by Deno's --allow-read. Treat it as a review item until the current version is checked.
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/issues/9749

2. Installation risk: 3.0.4b1

• Severity: medium
• Finding: Installation risk is backed by a source signal: 3.0.4b1. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.0.4b1

3. Installation risk: 3.1.2

• Severity: medium
• Finding: Installation risk is backed by a source signal: 3.1.2. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.1.2

4. Installation risk: 3.1.3

• Severity: medium
• Finding: Installation risk is backed by a source signal: 3.1.3. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.1.3

5. Installation risk: 3.2.0

• Severity: medium
• Finding: Installation risk is backed by a source signal: 3.2.0. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.2.0

6. Installation risk: Use Tool functions that require external libaries in CodeAct

• Severity: medium
• Finding: Installation risk is backed by a source signal: Use Tool functions that require external libaries in CodeAct. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/issues/8839

7. Capability assumption: README/documentation is current enough for a first validation pass.

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: The project should not be treated as fully validated until this signal is reviewed.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: capability.assumptions | github_repo:587050620 | https://github.com/stanfordnlp/dspy | README/documentation is current enough for a first validation pass.

8. Maintenance risk: Maintainer activity is unknown

• Severity: medium
• Finding: Maintenance risk is backed by a source signal: Maintainer activity is unknown. Treat it as a review item until the current version is checked.
• User impact: Users cannot judge support quality until recent activity, releases, and issue response are checked.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: evidence.maintainer_signals | github_repo:587050620 | https://github.com/stanfordnlp/dspy | last_activity_observed missing

9. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: downstream_validation.risk_items | github_repo:587050620 | https://github.com/stanfordnlp/dspy | no_demo; severity=medium

10. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: risks.scoring_risks | github_repo:587050620 | https://github.com/stanfordnlp/dspy | no_demo; severity=medium

11. Security or permission risk: 3.0.4

• Severity: medium
• Finding: Security or permission risk is backed by a source signal: 3.0.4. Treat it as a review item until the current version is checked.
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.0.4

12. Security or permission risk: 3.0.4b2

• Severity: medium
• Finding: Security or permission risk is backed by a source signal: 3.0.4b2. Treat it as a review item until the current version is checked.
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/stanfordnlp/dspy/releases/tag/3.0.4b2

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using dspy with real data or production workflows.

• PythonInterpreter: paths containing commas are silently misparsed by Den - github / github_issue
• [[Bug] PythonInterpreter fails with default setup due to missing Deno rea](https://github.com/stanfordnlp/dspy/issues/9501) - github / github_issue
• Use Tool functions that require external libaries in CodeAct - github / github_issue
• 3.2.1 - github / github_release
• 3.2.0 - github / github_release
• 3.1.3 - github / github_release
• 3.1.2 - github / github_release
• 3.1.1 - github / github_release
• 3.1.0 - github / github_release
• 3.1.0b1 - github / github_release
• 3.0.4 - github / github_release
• 3.0.4b2 - github / github_release