# https://github.com/Upsonic/Upsonic Project Manual

Generated at: 2026-06-26 10:07:46 UTC

## Table of Contents

- [Overview & Getting Started](#page-overview)
- [Agent Core Architecture](#page-agent-core)
- [Models, Providers & External Integrations](#page-models-integrations)
- [Knowledge, Storage & Document Processing](#page-data-knowledge)

<a id='page-overview'></a>

## Overview & Getting Started

### Related Pages

Related topics: [Agent Core Architecture](#page-agent-core)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md)
- [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)
- [src/upsonic/cli/commands/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)
- [src/upsonic/cli/commands/init/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/__init__.py)
- [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)
- [src/upsonic/cli/commands/install/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/install/__init__.py)
- [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py)
- [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)
- [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)
- [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)
</details>

# Overview & Getting Started

Upsonic is an open-source framework for building, packaging, and serving AI agents. It ships with a small but opinionated developer experience: a single `upsonic` CLI that scaffolds a project, a configuration file (`upsonic_configs.json`) that drives both runtime behavior and generated OpenAPI schema, and an `agent.run` / `agent.arun` execution model that streams fine-grained lifecycle events. This page gives a source-backed tour of those building blocks so a new contributor can get productive in minutes.

## 1. What Upsonic Is

Upsonic centers on a **minimalist agent + MCP** approach to building autonomous AI systems. As described in the project README, the framework exposes a Python API for defining agents and tasks, plus a CLI for the surrounding lifecycle (scaffolding, dependency management, serving).

The codebase is organized around a few clear subsystems:

- **Agent runtime** — `src/upsonic/run/agent/` contains the input/output pipelines that adapt user prompts, documents, and structured outputs to the model layer. Source: [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)
- **Event system** — `src/upsonic/utils/agent/events.py` enumerates the async and sync yield-event helpers that fire during an agent run (text deltas, tool calls, cache checks, model selection, policy checks, reflection, etc.). Source: [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)
- **CLI** — `src/upsonic/cli/commands/` packages the developer-facing commands into a flat, import-friendly surface. Source: [src/upsonic/cli/commands/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)

```mermaid
flowchart LR
    A[Developer] --> B[upsonic CLI]
    B --> C[upsonic_configs.json]
    C --> D[FastAPI Server<br/>upsonic run]
    C --> E[Streamlit UI<br/>optional]
    B --> F[main.py<br/>agent async main]
    F --> G[Agent Runtime]
    G --> H[Event Stream]
    H --> I[Consumer / UI]
```

## 2. Installing & Scaffolding a Project

The CLI is the recommended entry point. It exposes six top-level commands: `init`, `add`, `remove`, `install`, `run`, and `zip`. These are re-exported from the `upsonic.cli.commands` namespace. Source: [src/upsonic/cli/commands/__init__.py:7-21](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)

The `upsonic init` command bootstraps a new project. It prompts for an agent name, writes `main.py` and `upsonic_configs.json`, and registers default dependencies across three sections: `api`, `streamlit`, and `development`. Source: [src/upsonic/cli/commands/init/command.py:42-70](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)

Typical first-time flow:

```bash
# 1. Scaffold a project in the current directory
upsonic init

# 2. Install the default API dependencies (uses uv when available, else pip)
upsonic install            # installs the 'api' section by default
upsonic install all       # installs api + streamlit + development

# 3. Run the agent as a FastAPI server with auto-generated OpenAPI docs
upsonic run --host 0.0.0.0 --port 8000
```

Each of these commands has a dedicated help panel rendered through the Rich-based printer. The CLI help text documents every flag and example inline. Source: [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)

The generated `upsonic_configs.json` is the single source of truth. It declares the agent's `agent_name`, `description`, machine spec, dependency sections, entrypoints, and — most importantly — the input/output schemas that get materialized into HTTP request/response models. Source: [src/upsonic/cli/commands/init/command.py:42-70](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)

## 3. Managing Dependencies

Dependencies are tracked in three sections inside `upsonic_configs.json`. The default `api` section includes `fastapi`, `uvicorn`, `aiofiles`, `celery`, `sqlalchemy`, `psycopg2-binary`, `redis`, `pytz`, `psutil`, `fire`, `ruamel.yaml`, and `upsonic` itself. Source: [src/upsonic/cli/commands/init/command.py:55-67](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)

| Command | Purpose |
| --- | --- |
| `upsonic add <library> <section>` | Append a pinned or unpinned requirement to a section (e.g., `upsonic add requests==2.31.0 api`). |
| `upsonic remove <library> <section>` | Remove a library from a section. |
| `upsonic install [section]` | Install dependencies; `section` is one of `api` (default), `streamlit`, `development`, or `all`. Prefers `uv` and falls back to `pip`. |

Source: [src/upsonic/cli/commands/install/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/install/__init__.py); help text in [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)

## 4. Running & Observing Agents

Once the project is installed, `upsonic run` boots a FastAPI server that dynamically synthesizes the OpenAPI schema from `upsonic_configs.json`. The `map_inputs_props` and related helpers in `src/upsonic/cli/commands/shared/openapi.py` translate declarative input types (`string`, `number`, `integer`, `boolean`, `file`, `files`) into JSON Schema and `multipart/form-data` request bodies, and `map_output_props` produces the response schema. Source: [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py)

The runtime supports both `multipart/form-data` and `application/json` content types, and exposes interactive docs at `/docs`. Source: [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)

Programmatic runs surface a rich event stream. The exports list in `src/upsonic/utils/agent/events.py` includes both async (`ayield_*`) and sync (`yield_*`) variants for text deltas, tool calls, tool results, cache lookups (check/hit/miss/stored), model selection and response, messages built, run started, memory updates, system prompt built, context built, user input built, chat history loaded, storage connection, LLM prepared, policy check/feedback, reflection, reliability, and execution complete / final output. This means a consumer can subscribe to `agent.arun(...)` and react to every stage without polling. Source: [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)

Inputs and outputs are normalized by dedicated adapters. `run/agent/input.py` converts URL-based `ImageUrl` / `DocumentUrl` content to `BinaryContent` (sync via `httpx.get`, async via `httpx.AsyncClient`), and `run/agent/output.py` rehydrates `__builtin_type__`, `__pydantic_type__`, and `__type__` markers back into their original Python classes using `importlib`. Sources: [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py); [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)

## 5. Common Pitfalls & Community Notes

A few recurring themes from the issue tracker worth knowing on day one:

- **Configuration gaps** — Issue #333 notes there is no `temperature` parameter on the agent configuration class. Treat the agent config surface as declarative and check release notes before assuming a model parameter is exposed. Source: [Issue #333](https://github.com/Upsonic/Upsonic/issues/333)
- **MCP integration** — Issue #244 asks how to register MCP servers with environment variables. The `client.add_mcp_server(name, command, args)` pattern shown in the issue is the supported hook point. Source: [Issue #244](https://github.com/Upsonic/Upsonic/issues/244)
- **Release cadence** — v0.77.x lines focus on centralized usage tracking and tool-args tolerance (v0.77.3, v0.77.2). Pin a known-good version when reproducing issues. Sources: [v0.77.3 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.3); [v0.77.2 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.2)
- **Newest capabilities** — v0.75.0 introduced a `KBState` enum (`UNINITIALIZED → CONNECTED → INDEXED → CLOSED`) for `KnowledgeBase` lifecycle safety, and v0.77.0 added an agent `cost` property plus `asqav` governance. Sources: [v0.75.0 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.75.0); [v0.77.0 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.0)

## See Also

- [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md) — High-level project overview and links to the docs site at `docs.upsonic.ai`.
- [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py) — Authoritative help text for every CLI subcommand.
- [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py) — Full list of agent lifecycle events for stream consumers.
- [Discord community](https://discord.gg/pmYDMSQHqY) — Real-time help from the Upsonic team and other developers.

---

<a id='page-agent-core'></a>

## Agent Core Architecture

### Related Pages

Related topics: [Overview & Getting Started](#page-overview), [Models, Providers & External Integrations](#page-models-integrations)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/upsonic/agent/agent.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/agent/agent.py)
- [src/upsonic/agent/base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/agent/base.py)
- [src/upsonic/agent/autonomous_agent/autonomous_agent.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/agent/autonomous_agent/autonomous_agent.py)
- [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)
- [src/upsonic/utils/agent/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/__init__.py)
- [src/upsonic/run/agent/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/__init__.py)
- [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)
- [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)
- [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)
- [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)
- [src/upsonic/cli/commands/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)
- [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md)
</details>

# Agent Core Architecture

## Purpose and Scope

The Agent Core is the central orchestration layer of Upsonic. It exposes a minimal, opinionated Python API — `Agent` and `Task` — that lets developers build autonomous AI agents with a few lines of code, while exposing advanced lifecycle hooks for tooling, telemetry, MCP integration, and policy enforcement.

According to the [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md), Upsonic's tagline is **"Build Autonomous AI Agents in Python"**, and the library positions itself as a minimalist agent + MCP stack. The Agent Core is the runtime that satisfies that promise: it takes a `Task` description, binds it to a configured `Agent`, executes it through the model and tool layer, and yields a stream of structured events.

The Agent Core is consumed in two ways:

1. **Programmatic** — directly importing `Agent` and `Task` from the `upsonic` package.
2. **Via the CLI** — `upsonic init` scaffolds a project that imports and runs `Agent.print_do_async(task)` inside an async `main()` ([src/upsonic/cli/commands/init/command.py:1-50](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)).

A community thread ([#244 "How to add MCP?"](https://github.com/Upsonic/Upsonic/issues/244)) and an open collab request ([#609 OpenxAI x Upsonic](https://github.com/Upsonic/Upsonic/issues/609)) both reference this minimalist API surface, indicating that the Agent Core is the surface most users encounter first.

## High-Level Architecture

The Agent Core sits between user code and the model/tool/storage backends. It accepts a `Task`, configures the LLM and tool surface, builds the message stream, dispatches the model request, and emits structured lifecycle events that callers can consume synchronously or asynchronously.

```mermaid
flowchart TD
    A[User Code<br/>Agent + Task] --> B[Agent Core<br/>agent.py / base.py]
    B --> C[Event Layer<br/>utils/agent/events.py]
    B --> D[Run I/O Layer<br/>run/agent/input.py + output.py]
    B --> E[Model Provider<br/>LLM backend]
    B --> F[Tools / MCP<br/>autonomous_agent, deepagent]
    C --> G[Consumers<br/>print_do_async, telemetry, UIs]
    D --> H[Persistence<br/>SQL/Storage]
    F --> I[Shell, FS, Mail,<br/>KnowledgeBase]
```

The `Agent` class is the single entry point; `Task` is the unit of work. The CLI `init` template confirms this two-object pattern ([src/upsonic/cli/commands/init/command.py:80-95](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)):

```python
from upsonic import Task, Agent

async def main(inputs):
    user_query = inputs.get("user_query")
    answering_task = Task(f"Answer the user question {user_query}")
    agent = Agent()
    result = await agent.print_do_async(answering_task)
    return {"bot_response": result}
```

## Public API Surface

The Agent Core exposes two top-level types and a thin set of execution helpers.

| Symbol | Location | Role |
| --- | --- | --- |
| `Agent` | `src/upsonic/agent/agent.py` | Orchestrates a single task: model selection, tool binding, message assembly, event emission, final output. |
| `Agent` (base) | `src/upsonic/agent/base.py` | Holds shared lifecycle, configuration, and event subscription primitives. |
| `Task` | `src/upsonic` package surface | Unit of work passed to `Agent`; carries description, context, attachments, and constraints. |
| `AutonomousAgent` | `src/upsonic/agent/autonomous_agent/autonomous_agent.py` | Long-running variant that plans, executes, and iterates without a pre-shaped prompt. |
| `DeepAgent` | `src/upsonic/agent/deepagent/deepagent.py` | Hierarchical sub-agent orchestration. |

A frequent community question ([#333 "How to set the Agent temperature?"](https://github.com/Upsonic/Upsonic/issues/333)) notes that not every sampling parameter lives on the `Agent` configuration class itself — model-level parameters are typically applied through the configured model provider profile, which is then attached to the agent run output ([src/upsonic/run/agent/output.py:1-30](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)).

## Event System

All meaningful lifecycle moments in an agent run are exposed as named, lazy-loaded event-yielding functions. The Agent Core emits events for text deltas, tool calls, cache hits/misses, model selection, policy checks, reflection, reliability scoring, and execution completion.

The catalogue is defined in [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py) and re-exported via the lazy-loading shim at [src/upsonic/utils/agent/__init__.py:1-40](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/__init__.py). The shim defers the heavy event imports until the caller actually references a yield function:

```python
def __getattr__(name: str) -> Any:
    try:
        events_module = _lazy_import("upsonic.utils.agent.events")()
        return getattr(events_module, name)
    except (ImportError, AttributeError):
        pass
    raise AttributeError(f"module '{__name__}' has no attribute '{name}'.")
```

Source: [src/upsonic/utils/agent/__init__.py:24-35](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/__init__.py)

### Event Categories

| Category | Examples | Trigger |
| --- | --- | --- |
| Text streaming | `yield_text_delta_event`, `yield_text_complete_event` | Model token output |
| Tool dispatch | `yield_tool_call_event`, `yield_tool_result_event`, `yield_external_tool_pause_event` | Tool invocation and result |
| Cache | `yield_cache_check_event`, `yield_cache_hit_event`, `yield_cache_miss_event`, `yield_cache_stored_event` | Cache layer activity |
| Model | `yield_model_selected_event`, `yield_model_request_start_event`, `yield_model_response_event` | Provider interaction |
| Policy & reliability | `yield_policy_check_event`, `yield_reflection_event`, `yield_reliability_event` | Governance hooks |
| Run lifecycle | `yield_run_started_event`, `yield_execution_complete_event`, `yield_final_output_event` | Run boundaries |
| Memory & context | `yield_memory_update_event`, `yield_messages_built_event`, `yield_system_prompt_built_event` | Context preparation |

Source: [src/upsonic/utils/agent/events.py:1-90](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)

The v0.77.2 release notes ([v0.77.2](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.2)) describe how the chat and messages layer tolerates trailing junk in tool arguments and resets the task on retry, which is implemented by emitting the appropriate tool and run lifecycle events through this system.

## Run I/O and Serialization

The Agent Core wraps every run in a pair of typed containers — `AgentRunInput` and `AgentRunOutput` — exposed via the lazy shim at [src/upsonic/run/agent/__init__.py:1-50](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/__init__.py). These objects are what the CLI's FastAPI server serializes to and from the HTTP client.

`AgentRunInput` handles prompt construction, multimodal attachments, and URL → binary conversion ([src/upsonic/run/agent/input.py:1-60](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)):

```python
@staticmethod
def _convert_url_to_binary(url_content):
    import httpx
    from upsonic.messages.messages import BinaryContent
    response = httpx.get(url_content.url)
    response.raise_for_status()
    return BinaryContent(
        data=response.content,
        media_type=url_content.media_type,
        identifier=url_content.identifier,
    )
```

Source: [src/upsonic/run/agent/input.py:24-40](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)

`AgentRunOutput` reconstructs the output schema, thinking parts, and provider profile from serialized data ([src/upsonic/run/agent/output.py:1-40](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)). It supports both `__pydantic_type__` and `__builtin_type__` discriminators, so a persisted run can round-trip `str`, `BaseModel`, or arbitrary annotated classes.

## CLI Integration

The CLI is a thin wrapper around the Agent Core. `upsonic init` writes a project that depends on `upsonic` and a FastAPI/uvicorn stack ([src/upsonic/cli/commands/init/command.py:40-70](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)). `upsonic run` then boots a FastAPI server that builds its OpenAPI schema dynamically from the configured input and output schemas ([src/upsonic/cli/printer.py:1-40](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)).

Command registration is centralized in [src/upsonic/cli/commands/__init__.py:1-20](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py):

```python
from upsonic.cli.commands.init.command import init_command
from upsonic.cli.commands.add.command import add_command
from upsonic.cli.commands.remove.command import remove_command
from upsonic.cli.commands.install.command import install_command
from upsonic.cli.commands.run.command import run_command
from upsonic.cli.commands.zip.command import zip_command
```

Source: [src/upsonic/cli/commands/__init__.py:8-18](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)

This means the Agent Core has exactly one stable Python surface, but it can be invoked from a CLI script, a FastAPI endpoint, a Streamlit UI, or an MCP server with no code changes.

## Configuration Notes and Common Pitfalls

- **Model-level parameters.** Sampling controls like `temperature` are provider-specific and are not always surfaced on `Agent`'s configuration class. Configure them through the model provider profile that the agent resolves ([src/upsonic/run/agent/output.py:20-40](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)). See community discussion in [#333](https://github.com/Upsonic/Upsonic/issues/333).
- **MCP servers.** MCP integration is added through the `instance.client.add_mcp_server(...)` call, which lives on the underlying client used by `Agent`. The community thread [#244](https://github.com/Upsonic/Upsonic/issues/244) clarifies that environment variables for MCP servers must be passed alongside the command.
- **Shell command safety.** A community-reported best-practice item ([#596](https://github.com/Upsonic/Upsonic/issues/596)) highlights that `shell=True` style invocations in autonomous toolkits should be reviewed; the `shell_toolkit.py` module under `autonomous_agent` is the relevant integration point.
- **Cost tracking.** v0.77.0 added a `cost` property on agents ([#579](https://github.com/Upsonic/Upsonic/issues/579)), and v0.77.3 centralized the usage registry ([#602](https://github.com/Upsonic/Upsonic/issues/602)). Callers that aggregate usage should read from the agent's `cost` property rather than re-parsing model responses.
- **KnowledgeBase state.** v0.75.0 introduced the `KBState` enum (`UNINITIALIZED → CONNECTED → INDEXED → CLOSED`) replacing ad-hoc booleans. The Agent Core checks the state before dispatching retrieval.

## See Also

- [MCP Integration Guide](#)
- [Vector Database Providers](#)
- [KnowledgeBase Lifecycle](#)
- [CLI Reference: `upsonic init`, `run`, `install`](#)
- Release notes: [v0.77.3](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.3), [v0.77.0](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.0), [v0.75.0](https://github.com/Upsonic/Upsonic/releases/tag/v0.75.0)

---

<a id='page-models-integrations'></a>

## Models, Providers & External Integrations

### Related Pages

Related topics: [Overview & Getting Started](#page-overview), [Knowledge, Storage & Document Processing](#page-data-knowledge)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/upsonic/cli/printer.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/printer.py)
- [src/upsonic/cli/commands/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)
- [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)
- [src/upsonic/cli/commands/init/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/__init__.py)
- [src/upsonic/cli/commands/install/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/install/__init__.py)
- [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py)
- [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)
- [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)
- [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)
- [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md)
</details>

# Models, Providers & External Integrations

## Overview

Upsonic's "Models, Providers & External Integrations" surface area unifies three concerns: the LLM abstraction used by the agent runtime, the pluggable data/storage backends (vector databases, SQL stores, queues), and the suite of external services that agents can reach (MCP servers, OCR engines, mail). The project is built around an "agent + MCP" minimalism philosophy, with an emphasis on local-first execution and a thin plugin surface — explicitly referenced in the OpenxAI x Upsonic collaboration discussion ([issue #609](https://github.com/Upsonic/Upsonic/issues/609)).

In the source tree, two layers are visible: the runtime layer under `src/upsonic/run/agent/` (handling I/O, schemas, and event emission) and the CLI layer under `src/upsonic/cli/commands/` (scaffolding projects, dependency management, and exposing agents as FastAPI services).

## LLM Models & Runtime I/O Pipeline

The `Agent` runtime consumes model configuration and dispatches it through a model wrapper. As of v0.77.0, the `Agent` exposes a `cost` property for tracking model usage ([v0.77.0 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.0)). A v0.77.3 patch centralised the usage registry under TECH-1625 so that cost tokens are aggregated consistently across the call lifecycle ([v0.77.3 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.77.3)).

A persistent question in the community is whether the `Agent` class exposes a `temperature` parameter. [Issue #333](https://github.com/Upsonic/Upsonic/issues/333) documents that no `temperature` field is currently present on the agent configuration class — sampling parameters must be set through the model provider or wrapper layer.

### Input Pipeline

The `Input` class accepts URL-typed content (images and documents) and transparently downloads them. It detects `ImageUrl` and `DocumentUrl` types and converts them to `BinaryContent` via `httpx`, both synchronously and asynchronously:

```python
async with httpx.AsyncClient() as client:
    response = await client.get(url_content.url)
    response.raise_for_status()
    return BinaryContent(...)
```
Source: [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)

### Output Pipeline

The `Output` class preserves the original Pydantic model class when serialising structured responses. It round-trips `__pydantic_type__` and `__type__` markers so the originating module/class can be rehydrated on deserialisation:

```python
module = importlib.import_module(module_name)
output_schema = getattr(module, class_name)
```
Source: [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)

### Agent Event Surface

The runtime emits a rich set of events that downstream integrations can subscribe to — model-selected, tool-call, cache-check, reflection, reliability, and final-output events are all exported as `yield_*` and `ayield_*` helpers:

```python
__all__ = [
    "yield_model_selected_event", "yield_model_request_start_event",
    "yield_model_response_event", "yield_tool_call_event",
    "yield_tool_result_event", "yield_cache_hit_event",
    "yield_reflection_event", "yield_reliability_event", ...
]
```
Source: [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)

## Pluggable Providers & External Services

### Vector Databases & Storage Backends

Upsonic ships a pluggable vector-database layer with eight providers today. A representative proposal — Valkey Search, leveraging the `valkey-search` module (v1.2) on Valkey 9.1+ — is described in [issue #603](https://github.com/Upsonic/Upsonic/issues/603) as an "ultra-low-latency in-memory vector" option.

KnowledgeBase lifecycle was hardened in v0.75.0 with a `KBState` enum state machine (`UNINITIALIZED → CONNECTED → INDEXED → CLOSED`) replacing boolean flags, alongside new sync and async content-management APIs (`aadd_source`, `aadd_text`, `aremove_document`, etc.) ([v0.75.0 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.75.0)).

The default project template also bundles relational and queueing backends — `sqlalchemy`, `psycopg2-binary`, `redis`, and `celery` — and supports streamlit UI mode.
Source: [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)

| Backend | Library | Role |
|---|---|---|
| Vector store | (provider plug-in) | Embedding index |
| Relational | `sqlalchemy>=2.0.40` | ORM / SQL persistence |
| Postgres driver | `psycopg2-binary>=2.9.9` | DB connectivity |
| Queue / cache | `redis>=5.0.0` | Cache + Celery broker |
| Task queue | `celery>=5.5.2` | Distributed work |

### Model Context Protocol (MCP)

MCP servers are attached to a client instance and invoked through the agent. Community guidance ([issue #244](https://github.com/Upsonic/Upsonic/issues/244)) demonstrates the canonical pattern:

```python
instance.client.add_mcp_server("websearch", "npx", ["-y", "@mzxrai/mcp-webresearch"])
```

Servers are stdio-spawned processes; environment variables and per-server configuration are passed at the same call site — this is the surface that makes Upsonic's "minimalist agent + MCP" claim concrete.

### OCR Engines

The `OCR` façade supports multiple backends out of the box: `EasyOCR`, `RapidOCR`, `Tesseract`, `PaddleOCR`, `DeepSeek OCR`, and `DeepSeek via Ollama`. The engine is selected by passing `_ocr_engine=engine` to the constructor and content is read through `ocr.get_text(path)`.
Source: [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md)

### Mail Interface

Introduced in v0.74.4, the mail interface is a full SMTP/IMAP client with attachment handling, whitelist-based access control, and a heartbeat auto-poll loop. It works with any standard provider (Gmail, Outlook, Yahoo, Zoho, self-hosted) ([v0.74.4 release](https://github.com/Upsonic/Upsonic/releases/tag/v0.74.4)).

## CLI-Driven Project Scaffolding

The CLI is the public integration surface for spinning up an agent project. The available commands are exported from a single module:

```python
from upsonic.cli.commands import (
    init_command, add_command, remove_command,
    install_command, run_command, zip_command,
)
```
Source: [src/upsonic/cli/commands/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/__init__.py)

The `init` command produces `main.py` and `upsonic_configs.json`, the latter holding input/output schemas plus a multi-section dependency map (`api`, `streamlit`, `development`). The `install` command resolves those sections with `uv` (preferred) or `pip` as a fallback. The `run` command boots a FastAPI server and dynamically generates an OpenAPI schema from the declared input/output types, supporting both `multipart/form-data` and `application/json` request shapes:

```python
def map_inputs_props(inputs_schema):
    # Maps schema items to json_props, multipart_props, required fields
```
Source: [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py)

```mermaid
flowchart LR
    A[upsonic init] --> B[upsonic_configs.json + main.py]
    B --> C[upsonic add lib section]
    B --> D[upsonic install api|streamlit|development|all]
    D --> E[upsonic run --host --port]
    E --> F[FastAPI + Swagger UI]
    B --> G[upsonic zip]
```

### Common Pitfalls & Failure Modes

- **Missing `temperature` on Agent** — Pass temperature through the model provider configuration rather than the `Agent` constructor (see [issue #333](https://github.com/Upsonic/Upsonic/issues/333)).
- **MCP environment variables** — When adding an MCP server with environment variables, pass them at the `add_mcp_server` call site; the project README and CLI do not provide a config-file location for them (see [issue #244](https://github.com/Upsonic/Upsonic/issues/244)).
- **URL inputs require network access** — `ImageUrl` / `DocumentUrl` inputs are fetched with `httpx`; offline runs will fail at the `BinaryContent` conversion step.
- **KnowledgeBase state transitions** — A `KBState` must advance through `UNINITIALIZED → CONNECTED → INDEXED` before queries are accepted; invalid transitions raise `RuntimeError` (v0.75.0).

## See Also

- Agent Run Lifecycle (input/output, event emission)
- KnowledgeBase & Vector Store (KBState, provider plug-ins)
- CLI Reference (`upsonic init`, `run`, `install`, `add`, `remove`, `zip`)
- MCP Integration Guide
- OCR & Mail Interface Guides

---

<a id='page-data-knowledge'></a>

## Knowledge, Storage & Document Processing

### Related Pages

Related topics: [Agent Core Architecture](#page-agent-core), [Models, Providers & External Integrations](#page-models-integrations)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [src/upsonic/knowledge_base/knowledge_base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/knowledge_base/knowledge_base.py)
- [src/upsonic/knowledge_base/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/knowledge_base/__init__.py)
- [src/upsonic/vectordb/base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/base.py)
- [src/upsonic/vectordb/config.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/config.py)
- [src/upsonic/vectordb/factory.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/factory.py)
- [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py)
- [src/upsonic/run/agent/output.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/output.py)
- [src/upsonic/messages/messages.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/messages/messages.py)
- [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py)
- [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py)
- [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py)
- [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md)
</details>

# Knowledge, Storage & Document Processing

The Knowledge, Storage & Document Processing layer is the persistence and ingestion backbone of Upsonic. It combines a lifecycle-managed `KnowledgeBase`, a pluggable vector database abstraction, rich multimodal content types, and a serialization pipeline that lets agents carry documents, images, and binary payloads into every run. This page maps out the modules that participate, the states they pass through, and the APIs users interact with when wiring knowledge into agents.

## KnowledgeBase Subsystem

### Purpose and State Machine

The `KnowledgeBase` is the user-facing object that owns the lifecycle of a knowledge corpus. Starting with the v0.75.0 release, the subsystem was reworked to expose a typed state machine that replaces ad-hoc boolean flags, making invalid transitions raise `RuntimeError` ([release notes for v0.75.0](https://github.com/Upsonic/Upsonic/releases/tag/v0.75.0)).

```mermaid
stateDiagram-v2
    [*] --> UNINITIALIZED
    UNINITIALIZED --> CONNECTED: connect()
    CONNECTED --> INDEXED: add_source / add_text
    INDEXED --> CONNECTED: remove_document
    CONNECTED --> CLOSED: close()
    INDEXED --> CLOSED: close()
    CLOSED --> [*]
```

The canonical transitions are encoded in the `KBState` enum and surfaced through lifecycle guards on the `KnowledgeBase` class. Source: [src/upsonic/knowledge_base/knowledge_base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/knowledge_base/knowledge_base.py).

### Content Management APIs

KnowledgeBase ships both synchronous and asynchronous ingestion APIs, all declared in its module-level `__init__`. The v0.75.0 release introduced these new entry points: `aadd_source`, `aadd_text`, `aremove_document`, alongside their sync siblings `add_source`, `add_text`, and `remove_document` ([release notes for v0.75.0](https://github.com/Upsonic/Upsonic/releases/tag/v0.75.0)). Each ingestion call advances the state machine from `CONNECTED` to `INDEXED`, while removal rolls the state back, providing an auditable trail for downstream cache, policy, and reliability events. Source: [src/upsonic/knowledge_base/__init__.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/knowledge_base/__init__.py).

## Vector Database Layer

### Pluggable Provider Architecture

Upsonic decouples knowledge storage from any single vector engine via a provider abstraction. The base contract is defined in `vectordb/base.py`, with concrete providers instantiated through a factory in `vectordb/factory.py` configured by `vectordb/config.py`. Source: [src/upsonic/vectordb/base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/base.py), [src/upsonic/vectordb/factory.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/factory.py).

A community proposal ([#603](https://github.com/Upsonic/Upsonic/issues/603)) to add Valkey Search brings the total to nine providers, each chosen for a specific operating niche — from low-latency in-memory search to long-term disk-backed stores. The factory accepts a `VectorDBConfig` (defined in [src/upsonic/vectordb/config.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/config.py)) and returns an object that satisfies the common interface, so swapping providers never requires changes to the `KnowledgeBase` itself.

### Query and Storage Events

The agent runtime emits dedicated events whenever a vector store is contacted. The `ayield_storage_connection_event` and `yield_storage_connection_event` helpers, exported through [src/upsonic/utils/agent/events.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/utils/agent/events.py), make it possible to observe connection lifecycle hooks from custom event handlers and to integrate observability tools.

## Document and Binary Content Pipeline

### Multimodal Input Types

The agent input layer normalizes the wide variety of artifacts an Upsonic agent can consume. `ImageUrl` and `DocumentUrl` instances are detected by `_is_url_type` and downloaded through `_convert_url_to_binary` (sync) or `_aconvert_url_to_binary` (async) using `httpx`, then wrapped as `BinaryContent` with the original `media_type` and `identifier` preserved. Source: [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py).

### Serialization with Type Adapters

Once normalized, payloads are serialized using Pydantic `TypeAdapter`s so the same structures survive a round trip to a FastAPI endpoint or to disk. `ModelMessagesTypeAdapter` and `BinaryContentTypeAdapter` are used in `to_dict`, with Pydantic `BaseModel` payloads tagged via the `__pydantic__` discriminator. Source: [src/upsonic/run/agent/input.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/run/agent/input.py), [src/upsonic/messages/messages.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/messages/messages.py).

### OCR Engine Catalog

For document understanding, Upsonic exposes a pluggable OCR surface. The README documents five engines that can be swapped at construction time: `EasyOCR`, `RapidOCR`, `Tesseract`, `PaddleOCR`, `DeepSeek OCR`, and `DeepSeek via Ollama`. Source: [README.md](https://github.com/Upsonic/Upsonic/blob/main/README.md).

## CLI Scaffolding and API Surface

### Project Initialization

`upsonic init` lays down `main.py` and `upsonic_configs.json`, including default dependency sections for `api`, `streamlit`, and `development`. The scaffold wires up FastAPI, Celery, SQLAlchemy, Redis, and a default `upsonic` reference, defining entry points for the API and Streamlit layers. Source: [src/upsonic/cli/commands/init/command.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/init/command.py).

### OpenAPI Schema Mapping

`upsonic run` produces a FastAPI server with an OpenAPI document built dynamically from the input/output schemas declared in `upsonic_configs.json`. The `map_inputs_props` helper in `cli/commands/shared/openapi.py` translates each declared input into both a JSON-Schema and a `multipart/form-data` definition, while `map_output_props` does the same for the response model. This is what allows document uploads to reach the same `RequestModel` regardless of transport. Source: [src/upsonic/cli/commands/shared/openapi.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/cli/commands/shared/openapi.py).

## Common Failure Modes and Community Notes

- **State machine errors**: Calling `add_source` on a closed `KnowledgeBase` raises `RuntimeError`. Always re-`connect()` or instantiate a fresh base before reuse. Source: [src/upsonic/knowledge_base/knowledge_base.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/knowledge_base/knowledge_base.py).
- **MCP environment variables**: Issue [#244](https://github.com/Upsonic/Upsonic/issues/244) shows users struggling to inject `env` blocks into MCP servers; the `add_mcp_server` helper in the agent client accepts the same argv list as the underlying npx/uvx process and any required environment must be set in the parent process.
- **Vector DB provider selection**: A best-practice review ([#596](https://github.com/Upsonic/Upsonic/issues/596)) flagged `pickle` persistence patterns and `shell=True` in agent tooling. When persisting vector indexes, prefer the provider-native serializers surfaced by `VectorDBConfig` instead of generic `pickle`. Source: [src/upsonic/vectordb/config.py](https://github.com/Upsonic/Upsonic/blob/main/src/upsonic/vectordb/config.py).
- **Agent temperature**: Issue [#333](https://github.com/Upsonic/Upsonic/issues/333) notes the absence of a top-level `temperature` parameter on the agent configuration class; the parameter lives on the underlying model configuration.

## See Also

- Agent Run Lifecycle and Events
- MCP (Model Context Protocol) Integration
- Vector Database Providers Reference
- CLI Command Reference
- OpenAPI Generation and FastAPI Surface

---

<!-- evidence_pipeline_checked: true -->
<!-- evidence_injected: true -->

---

## Pitfall Log

Project: Upsonic/Upsonic

Summary: Found 12 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

## 1. Installation risk - Installation risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/Upsonic/Upsonic/issues/603

## 2. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.host_targets | https://github.com/Upsonic/Upsonic

## 3. Capability evidence risk - Capability evidence risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://github.com/Upsonic/Upsonic

## 4. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/Upsonic/Upsonic

## 5. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://github.com/Upsonic/Upsonic

## 6. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://github.com/Upsonic/Upsonic

## 7. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/Upsonic/Upsonic/issues/596

## 8. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/Upsonic/Upsonic/issues/619

## 9. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/Upsonic/Upsonic/issues/609

## 10. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/Upsonic/Upsonic/issues/609

## 11. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/Upsonic/Upsonic

## 12. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/Upsonic/Upsonic

<!-- canonical_name: Upsonic/Upsonic; human_manual_source: deepwiki_human_wiki -->