Related Pages

Related topics: Getting Started with Instructor, Installation and Setup

Introduction to Instructor

Instructor is an open-source Python library that simplifies structured output extraction from Large Language Models (LLMs). It provides a unified API across multiple LLM providers, enabling developers to define response models using Pydantic and automatically receive validated, typed outputs from AI responses.

Overview

Instructor addresses a common challenge in LLM development: parsing and validating unstructured model outputs into structured data types. Traditional approaches require manual JSON parsing, custom validation logic, and provider-specific code. Instructor streamlines this by:

• Integrating directly with existing LLM provider clients
• Using Pydantic models for response schema definition
• Automatically retrying failed validations
• Supporting multiple providers through a unified interface

Sources: README.md

Core Concepts

Response Model Pattern

At the heart of Instructor is the response_model parameter. Developers define a Pydantic BaseModel class that specifies the expected structure of the LLM response:

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

Automatic Retries

When validation fails (e.g., the LLM returns invalid data), Instructor automatically retries the request, passing the validation error back to the model for correction:

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v

# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

Supported Providers

Instructor provides a unified API that works with multiple LLM providers. The from_provider() method creates a configured client for any supported provider.

Provider Support Matrix

Supported Models by Provider

OpenAI Models:

• openai/gpt-4o
• openai/gpt-4o-mini
• openai/gpt-4-turbo

Anthropic Models:

• anthropic/claude-3-5-sonnet-20241022
• anthropic/claude-3-opus-20240229
• anthropic/claude-3-haiku-20240307

Google Models:

• google/gemini-2.0-flash-001
• google/gemini-pro
• google/gemini-pro-vision

Sources: README.md, examples/batch_api/README.md

Installation

Standard Installation

pip install instructor

Package Managers

# Using uv
uv add instructor

# Using poetry
poetry add instructor

API Key Configuration

Providers can be configured using environment variables or passed directly:

# From environment variables
client = instructor.from_provider("openai/gpt-4o")

# Direct API key
client = instructor.from_provider(
    "openai/gpt-4o", 
    api_key="sk-..."
)

client = instructor.from_provider(
    "anthropic/claude-3-5-sonnet",
    api_key="sk-ant-..."
)

Sources: README.md

Advanced Features

Caching

Instructor includes built-in caching mechanisms to reduce API calls and costs for repeated requests.

AutoCache (In-Memory LRU):

from instructor.cache import AutoCache

cache = AutoCache(maxsize=100)
client = instructor.from_openai(OpenAI(), cache=cache)

DiskCache (Persistent):

from instructor.cache import DiskCache

cache = DiskCache(directory=".instructor_cache")
client = instructor.from_openai(OpenAI(), cache=cache)

Cache TTL:

client.create(
    model="gpt-3.5-turbo",
    messages=messages,
    response_model=User,
    cache_ttl=3600,  # 1 hour
)

Performance Benefits:

• 156x faster cache hits compared to API calls
• Identical results from cache and API
• Persistent storage across client instances
• Cache invalidation based on prompts, models, schemas, and TTL

Sources: examples/caching_prototype/README.md

Hooks System

Hooks allow developers to attach event handlers for monitoring and debugging LLM interactions.

Available Hook Events:

• Request details (model, prompt)
• Input token count
• Token usage statistics
• Successful responses
• Parse errors
• Completion errors
• Retry attempt notifications

Hook Types:

1. On Request Hook: Triggered before/after API requests
2. On Response Hook: Triggered when responses are received
3. Parse Error Hook: Handles JSON parsing failures
4. Completion Error Hook: Handles API errors
5. Retry Hook: Notifies on retry attempts

Sources: examples/hooks/README.md

Batch Processing

The BatchProcessor provides a unified interface for creating batch jobs across providers.

Supported Operations:

# Create batch from file
instructor batch create \
  --messages-file messages.jsonl \
  --model "openai/gpt-4o-mini" \
  --response-model "examples.User" \
  --output-file batch_requests.jsonl

# Submit batch
instructor batch create-from-file \
  --file-path batch_requests.jsonl \
  --model "openai/gpt-4o-mini"

Sources: examples/batch_api/README.md

Validation with LLM Validators

Beyond standard Pydantic validators, Instructor supports llm_validator for complex validation scenarios:

from typing_extensions import Annotated
from pydantic import BeforeValidator
from instructor import llm_validator

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Sources: examples/validators/readme.md

Architecture

Provider Organization

The library organizes provider implementations in instructor/providers/ with consistent patterns:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory
│   └── utils.py       # Provider-specific utilities

Provider Implementation Categories:

Sources: instructor/providers/README.md

Workflow Diagram

graph TD
    A[Define Pydantic Model] --> B[Create Instructor Client]
    B --> C[Call create with response_model]
    C --> D{Provider Selection}
    D -->|OpenAI| E[OpenAI Handler]
    D -->|Anthropic| F[Anthropic Handler]
    D -->|Google| G[Google Handler]
    E --> H[Parse Response]
    F --> H
    G --> H
    H --> I{Validation Passed?}
    I -->|Yes| J[Return Typed Object]
    I -->|No| K[Retry with Error]
    K --> C

Example Applications

Citation Extraction

Instructor powers applications that extract structured citations from documents:

curl -X 'POST' \
  'https://jxnl--rag-citation-fastapi-app.modal.run/extract' \
  -H 'Authorization: Bearer <OPENAI_API_KEY>' \
  -d '{
  "context": "My name is Jason Liu...",
  "query": "What did the author do in school?"
}'

Response Format:

{
  "body": "In school, the author went to an arts high school.",
  "spans": [[91, 106]],
  "citation": ["arts highschool"]
}

Sources: examples/citation_with_extraction/README.md

Comparison: Without vs With Instructor

Sources: README.md

Next Steps

• Review the official documentation for detailed guides
• Explore example implementations in the examples/ directory
• Contribute via GitHub issues and pull requests
• Join the Instructor community for support and updates

License

Instructor is released under the MIT License. See LICENSE for details.

Related Pages

Related topics: Installation and Setup, Response Models and Type Safety

Getting Started with Instructor

Instructor is a Python library that simplifies structured data extraction from Large Language Model (LLM) responses. It integrates with multiple LLM providers and uses Pydantic for automatic validation, retry logic, and type safety. With Instructor, developers can define response models and receive fully validated, typed data directly from LLM outputs.

Sources: README.md:1-50

Installation

Install Instructor using pip or your preferred package manager:

pip install instructor

Or with alternative package managers:

uv add instructor
poetry add instructor

Sources: README.md:51-60

Quick Start

The simplest way to use Instructor is with the from_provider() factory function:

from pydantic import BaseModel
import instructor

client = instructor.from_provider("openai/gpt-4o-mini")


class Product(BaseModel):
    name: str
    price: float
    in_stock: bool


product = client.chat.completions.create(
    response_model=Product,
    messages=[{"role": "user", "content": "iPhone 15 Pro, $999, available now"}],
)

print(product)
# Product(name='iPhone 15 Pro', price=999.0, in_stock=True)

Sources: README.md:125-150

Supported Providers

Instructor works with multiple LLM providers using a unified API. The provider is automatically detected from the model identifier.

Provider Configuration Table

Usage Examples by Provider

# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

# With explicit API keys
client = instructor.from_provider("openai/gpt-4o", api_key="sk-...")
client = instructor.from_provider("anthropic/claude-3-5-sonnet", api_key="sk-ant-...")

Sources: README.md:61-90

Response Models

Response models define the expected output structure using Pydantic's BaseModel. Instructor automatically parses and validates LLM responses against these models.

Basic Model Definition

from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int

Without Instructor vs With Instructor

graph TD
    A[User Request] --> B[LLM API Call]
    B --> C{Using Instructor?}
    C -->|No| D[Raw JSON Response]
    D --> E[Manual Parsing]
    E --> F[Manual Validation]
    F --> G[Handle Errors]
    C -->|Yes| H[Response Model Defined]
    H --> I[Automatic Parsing]
    I --> J[Automatic Validation]
    J --> K[Validated Output]

Sources: README.md:1-40

Validation

Instructor provides automatic retry logic when validation fails. Define custom validators using Pydantic's @field_validator decorator.

Automatic Retries

from pydantic import BaseModel, field_validator


class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v


# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md:35-50

Custom LLM Validation

Use llm_validator for content-based validation:

from typing_extensions import Annotated
from pydantic import BeforeValidator
from instructor import llm_validator


class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Sources: examples/validators/readme.md:1-50

Hooks System

Hooks allow you to intercept and process requests and responses. The hooks system provides visibility into API calls.

Available Hook Events

1. Request Hook: Triggered before sending a request to the LLM
2. Parse Error Hook: Triggered when response parsing fails
3. Multiple Hooks: Shows how to attach multiple handlers to the same event

Hook Usage Example

# Hooks provide detailed information about each request:
# - Request details (model, prompt)
# - Approximate input token count
# - Token usage statistics
# - Successful responses
# - Parse errors
# - Completion errors
# - Retry attempt notifications

Sources: examples/hooks/README.md:1-40

Advanced Features

Citation with Extraction

Extract structured data with precise citations from source text:

# FastAPI endpoint example
response = client.chat.completions.create(
    response_model=Facts,
    messages=[...],
)
# Returns extracted facts with exact span positions and citations

Sources: examples/citation_with_extraction/README.md:1-80

Batch Processing

Process multiple requests efficiently using batch APIs:

# Test OpenAI
python run_batch_test.py create --model "openai/gpt-4o-mini"

# Test Anthropic
python run_batch_test.py create --model "anthropic/claude-3-5-sonnet-20241022"

Supported Batch Models Table

Sources: examples/batch_api/README.md:1-70

Fine-Tuning and Distillation

Generate fine-tuning datasets from Instructor outputs:

# Run the script to generate training data
python three_digit_mul.py

# Create fine-tuning job
instructor jobs create-from-file math_finetunes.jsonl

# With validation data
instructor jobs create-from-file math_finetunes.jsonl --n-epochs 4 --validation-file math_finetunes_val.jsonl

Sources: examples/distilations/readme.md:1-60

Provider Architecture

Each provider is organized in its own subdirectory under providers/:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory
│   └── utils.py       # Provider-specific utilities

Provider Implementation Patterns

Sources: instructor/providers/README.md:1-40

Utility Scripts

The repository includes maintenance scripts in the scripts/ directory:

Script Usage

# Clean markdown files
python scripts/make_clean.py --dry-run

# Check blog excerpts
python scripts/check_blog_excerpts.py

# Generate sitemap
python scripts/make_sitemap.py

# Audit documentation
python scripts/audit_patterns.py --summary

Sources: scripts/README.md:1-100

Next Steps

• Explore the examples directory for complete working examples
• Review the validators documentation for advanced validation patterns
• Learn about hooks for request introspection
• Set up batch processing for large-scale extractions
• Consider fine-tuning for domain-specific tasks

Statistics and Community

Instructor is trusted by over 100,000 developers and used in production by teams at OpenAI, Google, Microsoft, and AWS.

Sources: README.md:95-110

Related Pages

Related topics: Getting Started with Instructor, LLM Provider Support

Installation and Setup

Overview

Instructor is a Python library that enables structured outputs from Large Language Models (LLMs) using type validation through Pydantic. It provides a unified API across multiple LLM providers, allowing developers to define response models and receive type-safe, validated responses automatically.

Sources: README.md:1-50

System Requirements

Sources: examples/distilations/readme.md:10-12

Installation Methods

Using pip

The simplest installation method uses pip:

pip install instructor

Sources: README.md:60

Using uv

For projects using uv as the package manager:

uv add instructor

Sources: README.md:64

Using poetry

For projects using Poetry for dependency management:

poetry add instructor

Sources: README.md:65

Provider Setup

Instructor supports multiple LLM providers with a unified interface. The following diagram illustrates the provider architecture:

graph TD
    A[Application Code] --> B[Instructor Client]
    B --> C{Provider Type}
    C -->|OpenAI| D[OpenAI Provider]
    C -->|Anthropic| E[Anthropic Provider]
    C -->|Google| F[Gemini Provider]
    C -->|Ollama| G[Ollama Provider]
    C -->|Groq| H[Groq Provider]
    C -->|VertexAI| I[VertexAI Provider]
    D --> J[Provider API]
    E --> J
    F --> J
    G --> J
    H --> J
    I --> J

Sources: README.md:70-90

Basic Provider Initialization

Initialize clients using the from_provider() factory function:

# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

Sources: README.md:70-85

API Key Configuration

API keys can be configured in two ways:

#### Direct API Key Configuration

# OpenAI
client = instructor.from_provider("openai/gpt-4o", api_key="sk-...")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet", api_key="sk-ant-...")

# Groq
client = instructor.from_provider("groq/llama-3.1-8b-instant", api_key="gsk_...")

Sources: README.md:85-92

Basic Usage Pattern

The fundamental usage pattern involves:

1. Creating an instructor client
2. Defining a Pydantic response model
3. Calling the client with the response model

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_provider("openai/gpt-4o")

user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md:45-55

Provider-Specific Initialization

Some providers may require additional setup:

OpenAI Provider

The OpenAI provider is the reference implementation and uses from_openai() or from_provider():

import instructor
import openai

client = instructor.from_provider("openai/gpt-4o-mini")

Sources: README.md:70-72

Anthropic Provider

Requires the ANTHROPIC_API_KEY environment variable:

export ANTHROPIC_API_KEY="your-anthropic-api-key"

Sources: examples/batch_api/README.md:45-50

Google Provider

For Gemini models, ensure the Google SDK is installed and GOOGLE_API_KEY is set:

pip install google-generativeai

Sources: examples/batch_api/README.md:55-60

Optional Dependencies

Depending on your use case, you may need additional dependencies:

For Caching Features

from instructor.cache import AutoCache, DiskCache

Sources: examples/caching_prototype/README.md:12-15

For Validation

from instructor import llm_validator, patch

Sources: examples/validators/readme.md:15-17

For Sitemap Generation (Documentation Scripts)

uv add openai typer rich tenacity pyyaml

Sources: scripts/README.md:85-87

Supported Models by Provider

Sources: examples/batch_api/README.md:25-35

Verification Installation

To verify your installation, create a simple test:

import instructor
from pydantic import BaseModel

class TestModel(BaseModel):
    result: str

client = instructor.from_provider("openai/gpt-4o-mini")
result = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=TestModel,
    messages=[{"role": "user", "content": "Say 'Hello, World!'"}],
)
print(result.result)  # Should print: Hello, World!

Common Setup Issues

Sources: examples/batch_api/README.md:65-75

Next Steps

After installation, consider exploring:

• Response Models: Define Pydantic models for structured outputs
• Validation: Add custom validators to response models
• Retries: Instructor automatically retries failed validations
• Hooks: Add logging and monitoring to requests

Sources: README.md:95-115

Provider Architecture Details

The providers directory follows a consistent structure:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory
│   └── utils.py       # Provider-specific utilities

Not all providers require both files. Simpler providers like genai, groq, and vertexai use only client.py, while more complex ones like anthropic, bedrock, and gemini include both files.

Sources: providers/README.md:1-25

Adding New Providers

To add support for a new LLM provider:

1. Create a new subdirectory under providers/
2. Add an __init__.py file
3. Implement the from_<provider>() factory function
4. Add provider-specific utilities as needed

Sources: providers/README.md:27-32

Related Pages

Related topics: Core Components Architecture, LLM Provider Support

Project Structure

Overview

Instructor is a Python library that enables structured outputs from Large Language Models (LLMs) using Pydantic models for type validation. The project is organized into a modular architecture that separates core functionality, provider implementations, and example applications.

The primary goal of Instructor's project structure is to provide a unified API that works across multiple LLM providers while maintaining clean separation of concerns between client logic, provider-specific implementations, and response processing.

Sources: README.md

High-Level Architecture

graph TD
    A[User Code] --> B[Core Client API]
    B --> C[Provider Detection]
    C --> D{Provider Type}
    D -->|OpenAI| E[OpenAI Utils]
    D -->|Anthropic| F[Anthropic Client]
    D -->|Google| G[Gemini Utils]
    D -->|Other| H[Provider-specific]
    E --> I[Response Processing]
    F --> I
    G --> I
    H --> I
    I --> J[Pydantic Validation]
    J --> K[Typed Response Model]

Sources: instructor/core/client.py

Directory Structure

instructor/
├── core/                    # Core client and base functionality
│   ├── client.py           # Main client implementation
│   └── ...
├── providers/              # Provider-specific implementations
│   ├── anthropic/          # Anthropic Claude integration
│   ├── bedrock/            # AWS Bedrock integration
│   ├── cerebras/           # Cerebras integration
│   ├── cohere/             # Cohere integration
│   ├── fireworks/          # Fireworks AI integration
│   ├── gemini/             # Google Gemini integration
│   ├── genai/              # Google Generative AI
│   ├── groq/               # Groq integration
│   ├── mistral/            # Mistral AI integration
│   ├── perplexity/         # Perplexity AI integration
│   ├── vertexai/           # Google Vertex AI integration
│   ├── writer/             # Writer AI integration
│   ├── xai/                 # xAI integration
│   └── README.md           # Provider documentation
├── processing/             # Response processing and validation
│   ├── schema.py           # Schema handling
│   └── ...
├── cli/                    # Command-line interface
│   └── ...
└── ...

examples/
├── batch_api/              # Batch processing examples
├── citation_with_extraction/ # Extraction examples
├── distilations/           # Fine-tuning examples
├── hooks/                  # Hook system examples
└── validators/             # Validation examples

Sources: instructor/providers/README.md

Core Components

Core Client Module

The core client module (instructor/core/client.py) provides the unified API for all LLM providers. It handles:

Sources: instructor/core/client.py

Basic Usage Pattern

# Create client from any supported provider
client = instructor.from_provider("openai/gpt-4o")

# Use unified API for structured outputs
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

Provider System

Provider Organization

Each provider follows a consistent directory structure:

providers/
├── provider_name/
│   ├── __init__.py    # Module exports
│   ├── client.py      # Provider-specific client factory (optional)
│   └── utils.py       # Provider-specific utilities (optional)

Sources: instructor/providers/README.md

Provider Categories

Sources: instructor/providers/README.md

Supported Providers

graph LR
    A[Instructor] --> B[OpenAI]
    A --> C[Anthropic]
    A --> D[Google]
    A --> E[Groq]
    A --> F[Vertex AI]
    A --> G[Mistral]
    A --> H[ Cohere]
    A --> I[Fireworks]
    A --> J[Cerebras]
    A --> K[Perplexity]
    A --> L[Writer]
    A --> M[xAI]

Sources: README.md

Processing Module

The processing module (instructor/processing/) handles response parsing and validation:

Sources: instructor/processing/schema.py

Example Applications

The examples/ directory contains functional demonstrations:

Sources: examples/batch_api/README.md

Batch Processing Architecture

graph TD
    A[User Messages] --> B[BatchProcessor]
    B --> C[Provider Detection]
    C --> D{Provider}
    D -->|OpenAI| E[batch.jsonl]
    D -->|Anthropic| F[beta.messages.batches]
    D -->|Google| G[GCS Simulation]
    E --> H[Batch Submission]
    F --> H
    G --> H
    H --> I[Batch ID Storage]

Sources: examples/batch_api/README.md

Hooks System

The hooks system enables monitoring and logging of API operations:

# Hook event types
- on_request: Triggered before API call
- on_response: Triggered after successful response
- on_retry: Triggered during retry attempts
- on_error: Triggered on validation/parse errors

Sources: examples/hooks/README.md

Validation Examples

Custom validation is achieved using Pydantic validators:

from instructor import llm_validator, patch

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Sources: examples/validators/readme.md

CLI Tools

The project includes a command-line interface for common operations:

# Batch processing
instructor batch create --messages-file messages.jsonl --model "openai/gpt-4o-mini"

# Fine-tuning job creation
instructor jobs create-from-file math_finetunes.jsonl

Sources: examples/distilations/readme.md

Installation and Dependencies

The project supports multiple package managers:

# pip
pip install instructor

# uv
uv add instructor

# poetry
poetry add instructor

Sources: README.md

Scripts and Maintenance

The scripts/ directory contains maintenance utilities:

Sources: scripts/README.md

Key Design Patterns

Unified API Pattern

Instructor provides a consistent interface across all providers:

# Same code works for any provider
client = instructor.from_provider("provider/model-name")
result = client.chat.completions.create(
    response_model=YourModel,
    messages=[...]
)

Factory Pattern

The from_provider() factory function creates appropriate clients based on the provider string:

client = instructor.from_provider("openai/gpt-4o")
client = instructor.from_provider("anthropic/claude-3-5-sonnet")
client = instructor.from_provider("google/gemini-pro")

Validation Pipeline

graph LR
    A[LLM Response] --> B[Parse Tool Call]
    B --> C[Extract Arguments]
    C --> D[JSON Deserialization]
    D --> E[Pydantic Validation]
    E -->|Success| F[Typed Response]
    E -->|Failure| G[Retry with Error]
    G --> A

Version Compatibility Notes

• OpenAI uses a specialized utils.py without a client.py because from_openai() is defined in the core client
• This is because OpenAI serves as the reference implementation for the library
• OpenAI utilities are still required by core processing logic for standard handling

Sources: instructor/providers/README.md

Related Pages

Related topics: Project Structure, Validation and Retry Mechanisms

Core Components Architecture

Overview

The Instructor library is built on a modular architecture that enables structured LLM outputs with automatic validation and retry capabilities. The core components work together to provide a unified API across multiple LLM providers while maintaining extensibility through hooks and a robust retry mechanism.

graph TD
    A[User Code] --> B[Instructor Client]
    B --> C[Provider Adapter]
    C --> D[LLM API]
    D --> E[Response]
    E --> F[Response Processor]
    F --> G[Validation]
    G -->|Success| H[Typed Response]
    G -->|Failure| I[Retry Logic]
    I -->|Retry| D
    I -->|Max Retries| J[Error]
    F --> K[Hooks System]
    K -->|On Response| B
    K -->|On Validation Error| B

Architecture Components

1. Client Module (`instructor/core/client.py`)

The client module serves as the primary entry point for the Instructor library. It provides factory functions for creating provider-specific clients and implements the unified API interface.

#### Key Responsibilities

#### Supported Providers

The client architecture supports multiple providers through a consistent interface:

# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

# Groq
client = instructor.from_provider("groq/llama-3.1-8b-instant")

#### Core Client Interface

The unified client provides these primary methods:

Sources: examples/batch_api/README.md

2. Hooks System (`instructor/core/hooks.py`)

The hooks system provides event-driven extensibility, allowing developers to intercept and respond to various stages of the LLM interaction lifecycle.

#### Available Hook Events

#### Hook Handler Structure

graph LR
    A[LLM Request] --> B[on_request Hooks]
    B --> C[API Call]
    C --> D[on_response Hooks]
    D --> E{Validation}
    E -->|Pass| F[on_success Hooks]
    E -->|Fail| G[on_validation_error Hooks]
    F --> H[Return Response]
    G --> I[Retry Logic]
    I --> J[on_retry Hooks]

#### Hook Implementation Pattern

from instructor.hooks import Hooks, HookEvent

class LoggingHooks(Hooks):
    def on_request(self, request, **kwargs):
        print(f"🔍 Request: {request}")
        
    def on_response(self, response, **kwargs):
        print(f"✅ Response received")

# Attach hooks to client
client = instructor.from_provider("openai/gpt-4o")
client.attach_hooks(LoggingHooks())

Sources: examples/hooks/README.md

3. Retry Mechanism (`instructor/core/retry.py`)

The retry module handles automatic retries when validation fails, implementing exponential backoff and maximum retry limits.

#### Retry Configuration

#### Retry Flow

graph TD
    A[Initial Call] --> B{Validation Pass?}
    B -->|Yes| C[Return Response]
    B -->|No| D{Retry Count < Max?}
    D -->|Yes| E[Apply Backoff]
    E --> F[Log Error]
    F --> A
    D -->|No| G[Raise Exception]

#### Custom Retry with Field Validators

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v

# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

4. Response Processing (`instructor/process_response.py`)

The response processing module handles parsing LLM outputs into structured Pydantic models, including text parsing, JSON extraction, and model validation.

#### Processing Pipeline

#### Validation Error Handling

from typing_extensions import Annotated
from pydantic import BaseModel, BeforeValidator

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

try:
    qa: QuestionAnswerNoEvil = client.chat.completions.create(
        model="gpt-3.5-turbo",
        response_model=QuestionAnswerNoEvil,
        messages=[...],
    )
except Exception as e:
    print(e)  # Handles validation errors

#### LLM-Based Validation

The response processor supports llm_validator for semantic validation beyond Pydantic type checking:

from instructor import llm_validator

class OutputModel(BaseModel):
    reasoning: Annotated[str, BeforeValidator(llm_validator("be logical"))]
    answer: Annotated[str, BeforeValidator(llm_validator("be concise"))]

Sources: examples/validators/readme.md

Provider Architecture

Provider Organization

The providers/ directory follows a consistent structure across all supported providers:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory
│   └── utils.py       # Provider-specific utilities

#### Provider Categories

Sources: instructor/providers/README.md

Provider Detection

The unified client automatically detects the provider from the model string format:

# Format: provider/model-name
client = instructor.from_provider("openai/gpt-4o")
#   - Provider: openai
#   - Model: gpt-4o

client = instructor.from_provider("anthropic/claude-3-5-sonnet")
#   - Provider: anthropic
#   - Model: claude-3-5-sonnet

Data Flow Diagram

sequenceDiagram
    participant User
    participant Client
    participant Provider
    participant Processor
    participant Validator
    participant Hooks

    User->>Client: create(response_model, messages)
    Client->>Hooks: on_request
    Client->>Provider: API Call
    Provider->>Client: Raw Response
    Client->>Processor: Parse Response
    Processor->>Validator: Validate Model
    Validator-->>Processor: Valid/Invalid
    alt Valid
        Processor->>Client: Parsed Response
        Client->>Hooks: on_response
        Client-->>User: Typed Response
    else Invalid
        Processor->>Hooks: on_validation_error
        Client->>Hooks: on_retry
        Client->>Provider: Retry API Call
        Note over Client: Loop until max_retries
    end

Configuration Options

Client Configuration

Provider-Specific Configuration

# With API keys directly
client = instructor.from_provider(
    "openai/gpt-4o", 
    api_key="sk-..."
)

client = instructor.from_provider(
    "anthropic/claude-3-5-sonnet", 
    api_key="sk-ant-..."
)

# With custom settings
client = instructor.from_provider(
    "openai/gpt-4o",
    api_key="sk-...",
    max_retries=5,
    timeout=30.0
)

Key Design Patterns

1. Factory Pattern

The from_provider() factory method creates appropriate client instances based on the provider string:

def from_provider(provider: str, **kwargs) -> Instructor:
    """Factory function to create provider-specific client."""

2. Adapter Pattern

Each provider implements a consistent interface while handling provider-specific nuances:

graph TD
    A[Instructor API] --> B[Provider Adapter]
    B --> C[OpenAI Adapter]
    B --> D[Anthropic Adapter]
    B --> E[Google Adapter]
    B --> F[Other Providers]

3. Decorator Pattern

Hooks provide a way to extend functionality without modifying core classes:

@hooks.on_response
def log_response(response):
    # Logging functionality
    pass

Error Handling

Error Types

Exception Flow

graph TD
    A[API Call] --> B{Success?}
    B -->|No| C[API Error]
    C --> D{Retry < Max?}
    D -->|Yes| E[Backoff]
    E --> A
    D -->|No| F[Raise APIError]
    B -->|Yes| G[Parse Response]
    G --> H{Parse Success?}
    H -->|No| I[Parse Error]
    I --> D
    H -->|Yes| J[Validate Model]
    J --> K{Valid?}
    K -->|Yes| L[Return Result]
    K -->|No| M[Validation Error]
    M --> D

Summary

The Core Components Architecture of Instructor provides:

1. Unified Client Interface - Single API across all LLM providers
2. Automatic Validation - Pydantic-based response validation
3. Smart Retries - Automatic retry with exponential backoff
4. Extensible Hooks - Event-driven customization
5. Provider Abstraction - Consistent patterns across providers

This architecture enables developers to focus on application logic while Instructor handles the complexities of structured LLM outputs, validation, and error recovery.

Related Pages

Related topics: Validation and Retry Mechanisms

Response Models and Type Safety

Overview

Instructor provides robust Response Models and Type Safety mechanisms that enable structured, validated outputs from Large Language Models. By leveraging Pydantic's validation system, Instructor ensures that LLM responses conform to expected schemas while automatically retrying failed validations.

What are Response Models?

Response Models are Pydantic BaseModel classes that define the expected structure and types of LLM outputs. They serve as a contract between your application and the LLM, ensuring type-safe, validated responses.

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

Sources: README.md

Core Architecture

graph TD
    A[LLM API Call] --> B[Instructor Client]
    B --> C[Response Model Definition]
    C --> D[Pydantic Schema Validation]
    D --> E{Validation Pass?}
    E -->|Yes| F[Return Validated Object]
    E -->|No| G[Extract Error Message]
    G --> H[Retry with Error Context]
    H --> B

Using Response Models

Basic Usage

from instructor import from_provider
from pydantic import BaseModel

client = instructor.from_provider("openai/gpt-4o")

class User(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Jason is 28 years old"}],
)

Sources: README.md

Nested Response Models

Response Models support nested structures for complex data:

class Address(BaseModel):
    street: str
    city: str
    country: str

class UserProfile(BaseModel):
    name: str
    age: int
    address: Address

Field Validators

Pydantic's @field_validator decorator enables custom validation logic for model fields.

Basic Validation

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v

Sources: README.md

Validation with Regex

from pydantic import field_validator

class EmailModel(BaseModel):
    email: str

    @field_validator('email')
    def validate_email_format(cls, v):
        if '@' not in v:
            raise ValueError('Invalid email format')
        return v.lower()

LLM Validators

Instructor provides llm_validator for content-based validation using the LLM itself.

Setup

from typing_extensions import Annotated
from pydantic import BaseModel, BeforeValidator
from instructor import llm_validator

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Sources: examples/validators/readme.md

Validation Workflow

graph TD
    A[LLM Generates Response] --> B[Apply BeforeValidator]
    B --> C[LLM Validator Check]
    C --> D{Validation Pass?}
    D -->|Yes| E[Accept Response]
    D -->|No| F[Raise Assertion Error]
    F --> G[Retry with Context]

Automatic Retries

When validation fails, Instructor automatically retries the request with the error message included.

Retry Configuration

qa: QuestionAnswerNoEvil = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    response_model=QuestionAnswerNoEvil,
    max_retries=2,  # Number of retry attempts
    messages=[
        {
            "role": "system",
            "content": "You are a system that answers questions based on the context.",
        },
        {
            "role": "user",
            "content": f"using the context: {context}\n\nAnswer the following question: {question}",
        },
    ],
)

Sources: examples/validators/readme.md

Retry Behavior

Error Handling

Catching Validation Errors

try:
    qa: QuestionAnswerNoEvil = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        response_model=QuestionAnswerNoEvil,
        messages=[
            {"role": "system", "content": "Answer questions based on context."},
            {"role": "user", "content": f"context: {context}\nQuestion: {question}"},
        ],
    )
except Exception as e:
    print(f"Validation failed: {e}")

Error Output Example

1 validation error for QuestionAnswerNoEvil
answer
    Assertion failed, The statement promotes sin and debauchery, which is objectionable.

Sources: examples/validators/readme.md

Supported Data Types

Complete Example: User Extraction

from instructor import from_provider
from pydantic import BaseModel, field_validator

client = instructor.from_provider("openai/gpt-4o")

class User(BaseModel):
    name: str
    age: int
    email: str

    @field_validator('email')
    def validate_email(cls, v):
        if '@' not in v:
            raise ValueError('Invalid email address')
        return v.lower()

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0 or v > 150:
            raise ValueError('Age must be between 0 and 150')
        return v

# Automatic validation and retries
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "John is 35 years old, email: [email protected]"}],
)

print(f"Name: {user.name}")      # John
print(f"Age: {user.age}")        # 35
print(f"Email: {user.email}")    # [email protected]

Best Practices

1. Define clear schemas: Use descriptive field names and type hints
2. Add validators early: Catch invalid data at the source
3. Set appropriate retry limits: Balance between reliability and cost
4. Use allow_override wisely: Only when LLM feedback is acceptable
5. Handle exceptions: Always wrap calls in try/except for production code

Related Concepts

• Fields Documentation - Advanced field configuration
• Hooks System - Request/response lifecycle hooks
• Validations Examples - Custom validation patterns

Related Pages

Related topics: Response Models and Type Safety, Streaming and Partial Responses

Validation and Retry Mechanisms

Overview

Instructor provides robust validation and automatic retry mechanisms that work seamlessly with Pydantic models to ensure LLM outputs conform to expected schemas. When an LLM generates a response that fails validation, Instructor automatically retries the request with the validation error message included, allowing the model to self-correct its output.

The validation system extends beyond traditional Pydantic validators to include LLM-based validators that can check semantic content, tone, safety, and other qualitative aspects of generated text. Combined with automatic retries, this creates a feedback loop that significantly improves extraction reliability without requiring manual intervention.

Sources: README.md:1-50

Related Pages

Related topics: Validation and Retry Mechanisms

Streaming and Partial Responses

Overview

Streaming and partial responses are core features in Instructor that enable real-time processing of LLM outputs. Instead of waiting for the complete response to be generated, these features allow developers to receive and process data incrementally as it's being generated by the model.

The primary purpose of streaming support is to:

• Provide faster perceived latency by displaying partial results immediately
• Enable real-time UI updates for interactive applications
• Support progressive data extraction for long-form content
• Allow applications to start processing data before the entire response is complete

Sources: README.md:streaming support

Architecture

Instructor implements streaming through two complementary abstractions:

1. Partial - Represents a partially validated response model
2. Iterable - Represents a stream of validated response models

Data Flow Architecture

graph TD
    A[LLM API Streaming Response] --> B[Instructor Response Handler]
    B --> C{Response Mode}
    C -->|Partial Mode| D[Partial Validator]
    C -->|Iterable Mode| E[Iterable Validator]
    D --> F[Partial Response Model]
    E --> G[Validated Response Objects]
    F --> H[Application Consumer]
    G --> H

Component Overview

Sources: instructor/dsl/partial.py Sources: instructor/dsl/iterable.py

Partial Responses

What Are Partial Responses?

Partial responses enable receiving incrementally validated data as the LLM generates content. When using Partial[T], each streamed chunk contains a validated response model where some fields may be None until the model finishes generating them.

Usage Pattern

from instructor import Partial
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

# Stream partial responses as they're generated
for partial_user in client.chat.completions.create(
    response_model=Partial[User],
    messages=[{"role": "user", "content": "..."}],
    stream=True,
):
    print(partial_user)

Evolution of Partial Response

During streaming, the response model evolves as follows:

# Initial state - no fields populated
# User(name=None, age=None)

# After name is generated
# User(name="John", age=None)

# After full generation complete
# User(name="John", age=25)

Sources: README.md:Streaming support Sources: docs/concepts/partial.md

Iterable Responses

What Are Iterable Responses?

Iterable responses handle streams of complete, validated response models. Unlike Partial[T] which shows evolution of a single response, Iterable[T] represents multiple discrete responses that can be streamed from the LLM.

Usage Pattern

from instructor import Iterable

class TaskResult(BaseModel):
    task_id: str
    result: str
    status: str

# Stream multiple validated responses
for task in client.chat.completions.create(
    response_model=Iterable[TaskResult],
    messages=[{"role": "user", "content": "Process these items..."}],
    stream=True,
):
    print(f"Task {task.task_id}: {task.status}")

Sources: instructor/dsl/iterable.py Sources: docs/concepts/iterable.md

API Reference

Client Method Signature

Both partial and iterable streaming use the standard client creation method with additional parameters:

Response Type Resolution

Sources: examples/partial_streaming/run.py

Provider Compatibility

Universal Streaming Support

Streaming is supported across all major LLM providers through a unified interface:

# OpenAI
client = instructor.from_provider("openai/gpt-4o")
for partial in client.chat.completions.create(
    response_model=Partial[User],
    messages=[...],
    stream=True,
):
    print(partial)

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")
for partial in client.chat.completions.create(
    response_model=Partial[User],
    messages=[...],
    stream=True,
):
    print(partial)

# Google
client = instructor.from_provider("google/gemini-pro")
for partial in client.chat.completions.create(
    response_model=Partial[User],
    messages=[...],
    stream=True,
):
    print(partial)

Provider Response Handling

Each provider implements streaming through provider-specific response handlers:

graph LR
    A[Provider API] --> B[Provider Utils]
    B --> C[Standardized Chunk Format]
    C --> D[Instructor DSL]
    D --> E[Partial/Iterable Validator]
    E --> F[Validated Response]

Sources: instructor/providers/README.md

Validation Behavior

Validation During Streaming

Partial and iterable responses undergo the same validation as non-streaming responses. The key difference is that validation occurs on each streamed chunk rather than on a complete response.

Validation Pipeline Flow

graph TD
    A[Stream Chunk Received] --> B[Parse to Response Model]
    B --> C{Validation Pass?}
    C -->|Yes| D[Yield Validated Partial]
    C -->|No| E{Retry Available?}
    E -->|Yes| F[Retry with Error Context]
    E -->|No| G[Raise Validation Error]
    D --> H[Next Chunk]
    H --> A

Error Handling

When validation fails during streaming, Instructor can retry automatically if max_retries is configured:

for partial_user in client.chat.completions.create(
    response_model=Partial[User],
    messages=[{"role": "user", "content": "..."}],
    stream=True,
    max_retries=3,  # Automatic retry on validation failure
):
    print(partial_user)

Sources: examples/validators/readme.md

Use Cases

Real-Time UI Updates

Partial responses are ideal for applications requiring immediate visual feedback:

from instructor import Partial

class SearchResult(BaseModel):
    title: str
    url: str
    snippet: str

# Update UI as results arrive
for partial_result in client.chat.completions.create(
    response_model=Partial[SearchResult],
    messages=[{"role": "user", "content": f"Search for: {query}"}],
    stream=True,
):
    update_ui(partial_result)  # Progressive rendering

Long-Running Task Processing

Iterable responses suit scenarios where the LLM generates multiple distinct outputs:

class AnalysisReport(BaseModel):
    section: str
    findings: List[str]
    confidence: float

for report_section in client.chat.completions.create(
    response_model=Iterable[AnalysisReport],
    messages=[{"role": "user", "content": "Analyze this document..."}],
    stream=True,
):
    save_section(report_section)

Progressive Data Extraction

Extract complex nested structures incrementally:

class Document(BaseModel):
    title: str
    author: str
    sections: List["Section"]

class Section(BaseModel):
    heading: str
    content: str

for partial_doc in client.chat.completions.create(
    response_model=Partial[Document],
    messages=[{"role": "user", "content": "Extract document structure..."}],
    stream=True,
):
    display_preview(partial_doc)

Best Practices

Performance Considerations

Error Handling Strategy

try:
    for partial in client.chat.completions.create(
        response_model=Partial[User],
        messages=[...],
        stream=True,
        max_retries=3,
    ):
        process(partial)
except ValidationError as e:
    handle_validation_failure(e)
except Exception as e:
    handle_stream_error(e)

Memory Efficiency

When processing large streams, consume the generator incrementally rather than collecting all results:

# Memory efficient - processes one at a time
for item in client.chat.completions.create(
    response_model=Iterable[LargeModel],
    messages=[...],
    stream=True,
):
    write_to_disk(item)

# Memory intensive - collects all results
all_items = list(client.chat.completions.create(
    response_model=Iterable[LargeModel],
    messages=[...],
    stream=True,
))

Related Concepts

• Retry Logic: Streaming integrates with Instructor's automatic retry mechanism for validation failures
• Hooks System: Hooks can be attached to streaming operations for monitoring and logging
• Provider Architecture: Each LLM provider implements streaming through provider-specific utilities

Summary

Streaming and partial responses in Instructor provide powerful mechanisms for real-time data processing with LLM outputs. The Partial[T] wrapper enables progressive validation of single responses, while Iterable[T] supports streaming multiple discrete responses. Both features maintain full integration with Pydantic validation, retry logic, and all supported LLM providers through a unified API.

Related Pages

Related topics: Unified Provider Interface, Installation and Setup

LLM Provider Support

Overview

Instructor provides a unified API for working with multiple Large Language Model (LLM) providers. The library abstracts provider-specific implementations behind a common interface, allowing developers to switch between providers without modifying their application logic.

The from_provider() method serves as the primary entry point for initializing clients across all supported providers Sources: README.md.

Supported Providers

Instructor supports the following LLM providers through a consistent API:

Sources: README.md

Provider Architecture

Directory Structure

Each provider is organized in its own subdirectory within instructor/providers/:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory (optional)
│   └── utils.py       # Provider-specific utilities (optional)

Sources: instructor/providers/README.md

Provider Classification

Providers are categorized based on their implementation complexity:

#### Providers with Both client.py and utils.py

These providers require custom response handling logic and utility functions:

• anthropic
• bedrock
• cerebras
• cohere
• fireworks
• gemini
• mistral
• perplexity
• writer
• xai

#### Providers with Only client.py

These are simpler providers using standard response handling from the core:

• genai
• groq
• vertexai

#### Special Case: OpenAI

OpenAI doesn't have a client.py because from_openai() is defined in core/client.py. This is because OpenAI is the reference implementation that other providers are based on.

Sources: instructor/providers/README.md

Unified Client API

Basic Initialization

All providers use the same initialization pattern:

client = instructor.from_provider("provider/model-name")

Usage Example

from instructor import from_provider

# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

Sources: README.md

API Key Configuration

API keys can be passed directly to the from_provider() method:

# OpenAI with explicit API key
client = instructor.from_provider("openai/gpt-4o", api_key="sk-...")

# Anthropic with explicit API key
client = instructor.from_provider("anthropic/claude-3-5-sonnet", api_key="sk-ant-...")

# Groq with explicit API key
client = instructor.from_provider("groq/llama-3.1-8b-instant", api_key="gsk_...")

Alternatively, providers respect environment variables for API key configuration.

Sources: README.md

Response Model Integration

All providers support Pydantic response models for structured outputs:

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_provider("openai/gpt-4o")
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

Validator Support

Providers work seamlessly with Pydantic validators:

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v

# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

LLM Validators

Custom validation using llm_validator is supported across all providers:

from typing_extensions import Annotated
from pydantic import BeforeValidator
from instructor import llm_validator

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Sources: examples/validators/readme.md

Automatic Retries

Failed validations are automatically retried with the error message:

user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
    max_retries=2,
)

Sources: README.md

Batch Processing Support

Providers support batch API operations through the unified BatchProcessor interface:

Supported Batch Operations

# Create batch using CLI
instructor batch create \
  --messages-file messages.jsonl \
  --model "openai/gpt-4o-mini" \
  --response-model "examples.User" \
  --output-file batch_requests.jsonl

# Submit the batch
instructor batch create-from-file \
  --file-path batch_requests.jsonl \
  --model "openai/gpt-4o-mini"

Provider-Specific Batch Features

Sources: examples/batch_api/README.md

Batch Test Script

A test script is available to validate batch processing across providers:

# Test OpenAI
export OPENAI_API_KEY="your-api-key"
python run_batch_test.py create --model "openai/gpt-4o-mini"

# Test Anthropic
export ANTHROPIC_API_KEY="your-api-key"
python run_batch_test.py create --model "anthropic/claude-3-5-sonnet-20241022"

# Test Google (simulation mode)
python run_batch_test.py create --model "google/gemini-2.0-flash-001"

Sources: examples/batch_api/README.md

Hooks System

Providers integrate with Instructor's hooks system for monitoring and debugging:

Available Hook Events

• Request hooks: Execute before API calls
• Response hooks: Execute after successful responses
• Error hooks: Execute on completion errors
• Retry hooks: Execute on retry attempts

Hook Implementation Example

def on_request(detail):
    print(f"🔍 Request: {detail}")

def on_response(detail):
    print(f"✅ Response: {detail}")

def on_error(detail):
    print(f"❌ Error: {detail}")

client = instructor.from_provider("openai/gpt-4o")
client.attach_hook("request", on_request)
client.attach_hook("response", on_response)
client.attach_hook("error", on_error)

Sources: examples/hooks/README.md

Fine-tuning Integration

All providers support Instructor's fine-tuning capabilities:

# Generate fine-tuning data
python three_digit_mul.py

# Create fine-tuning job
instructor jobs create-from-file math_finetunes.jsonl

# With validation data
instructor jobs create-from-file math_finetunes.jsonl \
  --n-epochs 4 \
  --validation-file math_finetunes_val.jsonl

Sources: examples/distilations/readme.md

Provider Selection Guidelines

Flowchart: Provider Selection

graph TD
    A[Select Provider] --> B{Need Local Model?}
    B -->|Yes| C[Ollama]
    B -->|No| D{Need Specific Capabilities?}
    D -->|Anthropic Claude| E[Anthropic]
    D -->|Google Gemini| F[Google]
    D -->|OpenAI GPT| G[OpenAI]
    D -->|Low Cost| H[Groq]
    D -->|Custom| I[Other Providers]
    G --> J[Create Client]
    E --> J
    F --> J
    C --> J
    H --> J
    I --> J
    J --> K[Use with Response Model]

Selection Criteria

Adding New Providers

When adding a new provider to Instructor:

1. Create a new subdirectory under instructor/providers/
2. Add an __init__.py file
3. Implement client.py for the provider factory function
4. Add utils.py for provider-specific response handling
5. Update the unified from_provider() dispatcher

Sources: instructor/providers/README.md

Error Handling

Common Provider Errors

Error Recovery

Providers support automatic retry with exponential backoff for transient failures:

user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
    max_retries=3,
)

Sources: examples/batch_api/README.md

See Also

• Core Client Implementation
• Response Models and Validation
• Hooks System
• Batch Processing
• Fine-tuning

Related Pages

Related topics: LLM Provider Support, Core Components Architecture

Unified Provider Interface

Overview

The Unified Provider Interface is a core architectural feature of the Instructor library that provides a single, consistent API for interacting with multiple Large Language Model (LLM) providers. This abstraction eliminates provider-specific code patterns and allows developers to switch between different LLM backends without modifying their application logic.

The unified interface is accessed through the from_provider() factory function, which accepts provider/model strings in the format "provider/model-name" and returns a configured client instance that follows a standardized interface. Sources: README.md

Architecture

Core Design Principles

1. Provider Agnosticism: All supported providers implement the same method signatures
2. Automatic Model Detection: Provider is inferred from the model string prefix
3. Transparent Retries: Failed validations are automatically retried with error context
4. Type Safety: Full Pydantic integration for response validation

Supported Providers

The unified interface supports multiple LLM providers through a consistent abstraction layer:

Sources: README.md, providers/README.md

Usage

Basic Initialization

import instructor

# Create unified client for any provider
client = instructor.from_provider("openai/gpt-4o")

# Use the same API regardless of provider
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Provider Comparison

# OpenAI
client = instructor.from_provider("openai/gpt-4o")

# Anthropic
client = instructor.from_provider("anthropic/claude-3-5-sonnet")

# Google
client = instructor.from_provider("google/gemini-pro")

# Ollama (local)
client = instructor.from_provider("ollama/llama3.2")

Sources: README.md

API Key Configuration

The unified interface supports multiple ways to provide API credentials:

# With API keys directly (no environment variables needed)
client = instructor.from_provider("openai/gpt-4o", api_key="sk-...")
client = instructor.from_provider("anthropic/claude-3-5-sonnet", api_key="sk-ant-...")
client = instructor.from_provider("groq/llama-3.1-8b-instant", api_key="gsk_...")

Sources: README.md

Provider Directory Structure

Each provider implementation follows a standardized directory structure:

providers/
├── provider_name/
│   ├── __init__.py
│   ├── client.py      # Provider-specific client factory (optional)
│   └── utils.py       # Provider-specific utilities (optional)

Provider Categories

Providers with both client.py and utils.py:

• anthropic, bedrock, cerebras, cohere, fireworks, gemini, mistral, perplexity, writer, xai

Providers with only client.py:

• genai, groq, vertexai

Special Case - OpenAI:

• OpenAI doesn't have a client.py because from_openai() is defined in core/client.py
• OpenAI is the reference implementation that other providers are based on

Sources: providers/README.md

API Methods

Create Method

The primary method for generating structured responses:

response = client.chat.completions.create(
    response_model=User,  # Pydantic model
    messages=[...],
    model="...",
    max_retries=3,  # Automatic retry on validation failure
)

Alternative API Patterns

For simplified usage, these patterns are also supported:

# All equivalent to client.chat.completions.create
client.create()
client.create_partial()
client.create_iterable()
client.create_with_completion()

Sources: scripts/README.md

Response Model Integration

The unified interface seamlessly integrates with Pydantic for type validation:

Basic Model Definition

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

Model with Validation

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int

    @field_validator('age')
    def validate_age(cls, v):
        if v < 0:
            raise ValueError('Age must be positive')
        return v

When validation fails, Instructor automatically retries the request with the error message:

# Instructor automatically retries when validation fails
user = client.chat.completions.create(
    response_model=User,
    messages=[{"role": "user", "content": "..."}],
)

Sources: README.md

Batch Processing

The unified interface supports batch processing across all providers through the BatchProcessor class:

from instructor.batch import BatchProcessor

processor = BatchProcessor()

# Create batch with provider detection
processor.create_batch(
    messages=test_messages,
    model="openai/gpt-4o-mini",
    response_model=User,
)

Provider-Specific Batch Support

Sources: examples/batch_api/README.md

Workflow Diagram

graph TD
    A[User Code] --> B[from_provider factory]
    B --> C{Parse provider/model}
    C -->|openai| D[OpenAI Client]
    C -->|anthropic| E[Anthropic Client]
    C -->|google| F[Google Client]
    C -->|other| G[Other Provider Client]
    
    D --> H[Unified API Layer]
    E --> H
    F --> H
    G --> H
    
    H --> I[Provider-Specific Transport]
    I --> J{Response Validation}
    J -->|Pass| K[Return Typed Response]
    J -->|Fail| L[Retry with Error Context]
    L --> H

Advanced Configuration

Custom Validators

You can add custom LLM-based validation:

from typing_extensions import Annotated
from pydantic import BeforeValidator
from instructor import llm_validator

class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator("don't say objectionable things", allow_override=True)
        ),
    ]

Retry Configuration

# Allow retries on validation failure
qa = client.chat.completions.create(
    response_model=QuestionAnswer,
    max_retries=2,
    messages=[...],
)

Sources: examples/validators/readme.md

Migration from Legacy Patterns

Old Patterns to Unified Interface

Automated Migration Scripts

The repository includes scripts to automate migration:

# Replace old patterns automatically
python scripts/fix_old_patterns.py --dry-run
python scripts/fix_old_patterns.py

# Audit documentation for old patterns
python scripts/audit_patterns.py

Sources: scripts/README.md

Summary

The Unified Provider Interface is a fundamental architectural feature that enables Instructor to work seamlessly across multiple LLM providers. By abstracting provider-specific details behind a common API:

• Developers can switch providers without code changes
• Response validation works consistently across all providers
• Automatic retries provide resilience against transient failures
• The Pydantic integration ensures type safety throughout

This design philosophy makes Instructor highly adaptable and future-proof, as new providers can be added without affecting existing code.

Doramagic Pitfall Log

Doramagic extracted 16 source-linked risk signals. Review them before installing or handing real data to the project.

1. Installation risk: Documentation (at least Google-related) is an outdated mess.

• Severity: high
• Finding: Installation risk is backed by a source signal: Documentation (at least Google-related) is an outdated mess.. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/issues/2289

2. Installation risk: v1.13.0

• Severity: medium
• Finding: Installation risk is backed by a source signal: v1.13.0. Treat it as a review item until the current version is checked.
• User impact: First-time setup may fail or require extra isolation and rollback planning.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.13.0

3. Configuration risk: v1.12.0

• Severity: medium
• Finding: Configuration risk is backed by a source signal: v1.12.0. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.12.0

4. Configuration risk: v1.14.0

• Severity: medium
• Finding: Configuration risk is backed by a source signal: v1.14.0. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.14.0

5. Configuration risk: v1.14.3

• Severity: medium
• Finding: Configuration risk is backed by a source signal: v1.14.3. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.14.3

6. Configuration risk: v1.14.4

• Severity: medium
• Finding: Configuration risk is backed by a source signal: v1.14.4. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.14.4

7. Configuration risk: v1.15.0

• Severity: medium
• Finding: Configuration risk is backed by a source signal: v1.15.0. Treat it as a review item until the current version is checked.
• User impact: Users may get misleading failures or incomplete behavior unless configuration is checked carefully.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.15.0

8. Capability assumption: README/documentation is current enough for a first validation pass.

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: The project should not be treated as fully validated until this signal is reviewed.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: capability.assumptions | github_repo:653589102 | https://github.com/567-labs/instructor | README/documentation is current enough for a first validation pass.

9. Project risk: v1.14.2

• Severity: medium
• Finding: Project risk is backed by a source signal: v1.14.2. Treat it as a review item until the current version is checked.
• User impact: The project should not be treated as fully validated until this signal is reviewed.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: Source-linked evidence: https://github.com/567-labs/instructor/releases/tag/v1.14.2

10. Maintenance risk: Maintainer activity is unknown

• Severity: medium
• Finding: Maintenance risk is backed by a source signal: Maintainer activity is unknown. Treat it as a review item until the current version is checked.
• User impact: Users cannot judge support quality until recent activity, releases, and issue response are checked.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: evidence.maintainer_signals | github_repo:653589102 | https://github.com/567-labs/instructor | last_activity_observed missing

11. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: downstream_validation.risk_items | github_repo:653589102 | https://github.com/567-labs/instructor | no_demo; severity=medium

12. Security or permission risk: no_demo

• Severity: medium
• Finding: no_demo
• User impact: The project may affect permissions, credentials, data exposure, or host boundaries.
• Recommended check: Open the linked source, confirm whether it still applies to the current version, and keep the first run isolated.
• Evidence: risks.scoring_risks | github_repo:653589102 | https://github.com/567-labs/instructor | no_demo; severity=medium

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using instructor with real data or production workflows.

• Documentation (at least Google-related) is an outdated mess. - github / github_issue
• Tool: NEXUS structured financial data - github / github_issue
• Catching IncompleteOutputException : not possible as presently documente - github / github_issue
• bump lightllm upper bound for recent vulnerabililties - github / github_issue
• reask_anthropic_tools retry fails with HTTP 400 on AWS Bedrock — ToolUse - github / github_issue
• logger.debug in response.py leaks api_key verbatim via new_kwargs - github / github_issue
• RESPONSES_TOOLS streaming drops reasoning summary events (summary: auto) - github / github_issue
• v1.15.1 - github / github_release
• v1.15.0 - github / github_release
• v1.14.5 - github / github_release
• v1.14.4 - github / github_release
• v1.14.3 - github / github_release