Related Pages

Related topics: Providers

Overview

Purpose & Scope

Argus is a local-first research and content retrieval assistant designed to be invoked either as an MCP (Model Context Protocol) server by coding agents such as Claude Code, Codex CLI, and OpenCode, or as a plain HTTP service by browsers and scripts. Its job is to fan a single user question out across a tiered set of search providers, fuse the results, and, when an answer is partial or truncated, run a content-extraction chain that recovers the missing material before returning a structured pack.

The product positions itself against general-purpose web search APIs on three axes: (1) transparent cost controls with credit-aware provider routing, (2) a deterministic 12-step extraction chain rather than opportunistic scraping, and (3) dual transport (stdio MCP + HTTP) so the same daemon can serve interactive agents and headless pipelines from one process. Source: README.md:1-80

The project is a personal/homelab tool that publishes a dashboard under khamel.com/argus/ behind Authentik, but it is also packaged as a normal Python wheel so it can be installed with pip install argus and used programmatically. Source: pyproject.toml:1-60

High-Level Architecture

Argus runs as a single daemon process. The CLI (argus) and the MCP server (argus mcp) are thin entry points over the same internal core. The core owns three subsystems: a provider registry, a search/scoring pipeline, and a workflow runner.

flowchart LR
  A[Agent / User] -->|stdio JSON-RPC| B(MCP Server)
  A -->|HTTP POST /api/search| C(HTTP API)
  B --> D[Core Orchestrator]
  C --> D
  D --> E[Provider Registry]
  D --> F[Extraction Chain<br/>12 steps]
  D --> G[Workflow Runner<br/>recover-article / capture-site / build-research-pack]
  E -->|Tier 0: free, no key| H[WolframAlpha / Yahoo / DuckDuckGo]
  E -->|Tier 1: API key| I[Brave / Tavily / Valvu]
  F --> J[Corpus Store<br/>platformdirs user data dir]

Both transports share the same Python core because v1.6.1 deliberately moved all logging off stdout onto stderr to keep the stdio JSON-RPC channel clean. Source: src/argus/mcp_server.py:1-120 Source: src/argus/cli.py:1-200

The provider registry is where the cost-safety guarantees live. Providers are organized into tiers; Tier 0 providers (WolframAlpha LLM API with 2,000 free calls/month, Yahoo Search, and DuckDuckGo scraper) require no API key and are tried first for "grounding" and "research" modes. Tier 1 providers (Brave, Tavily, Valvu) require keys and are only invoked when Tier 0 results are insufficient. This explains why the project investigates "unexplained Valvu credit consumption" — every Valvu call is supposed to be deliberate and traceable. Source: src/argus/providers/__init__.py:1-150 Source: docs/providers.md:1-120

Core Capabilities

Four capabilities are first-class and documented individually; each maps to a single CLI command, an HTTP endpoint, and an MCP tool.

1. Multi-provider search — argus search and POST /api/search return a ranked list of results across enabled tiers. Shapley-value attribution (issue #7) computes the marginal contribution of each provider to the final ranking, which exposes which providers are pulling weight and which are dead code. Source: docs/providers.md:120-200

1. Content extraction with completeness assessment — the 12-step chain authenticates, picks a residential IP path when needed, fetches, parses, then runs five signals to decide whether the article is truncated. If it is, the chain escalates automatically rather than returning partial text. Source: CHANGELOG.md:1-80

1. Retrieval workflows — three named workflows recover partial material end-to-end:

Source: src/argus/workflows/recover_article.py:1-100 Source: src/argus/workflows/build_research_pack.py:1-100

• recover-article takes a known truncated URL and re-runs extraction with alternative strategies.
• capture-site walks a domain and materializes a corpus.
• build-research-pack produces a bundled artifact (open issue #19 plans to expose this one as an MCP tool so agents can pipe output to Hermes/Maya without a shell escape).

1. MCP-native agent integration — argus mcp init --global --client all writes the correct server config into Claude Code, Codex, and OpenCode simultaneously. v1.6.1 also fixed Codex TOML replacement so existing args arrays are preserved instead of corrupted. Source: docs/troubleshooting.md:1-80 Source: src/argus/cli.py:200-320

Project Layout & Entry Points

The repository follows a conventional Python src-layout. Runtime state (corpus, caches, logs) lives in a writable user data directory resolved via platformdirs, which keeps the package itself read-only and makes it safe to run under systemd or inside containers. Source: pyproject.toml:60-120

argus/
├── src/argus/         # package: cli, mcp_server, providers/, workflows/
├── examples/          # runnable SDK scripts (v1.6.2)
├── docs/              # README, providers.md, troubleshooting.md
├── services/          # ops: funnel-proxy nginx config for khamel.com/argus/
└── .github/           # CODEOWNERS, YAML issue templates

The examples/ directory is the recommended on-ramp: basic_search.py, extract_and_recover.py, and research_pack.py correspond one-to-one to the three core capabilities above. Source: examples/basic_search.py:1-60 Source: examples/research_pack.py:1-60

For deeper information, see docs/providers.md for the tier table and docs/troubleshooting.md for MCP stdio and credit-consumption diagnostics. Multi-egress deployment, which is code-complete but requires manual SSH to bring up, is tracked in issue #12 and escalated in issue #13.

Related Pages

Related topics: Overview, Api

Providers

The argus/providers/ package is the abstraction layer that lets Argus query multiple search and answer backends through a single, uniform interface. Rather than hard-coding calls to any individual service, every backend is plugged in as a Provider subclass that implements the contract defined in argus/providers/base.py. This design supports tiered routing, free vs. paid tiers, and graceful fallback when a key is missing or a call fails. Source: argus/providers/base.py.

Role in the Argus Architecture

Providers sit between the orchestrator (which receives search, grounding, and research requests) and the external APIs. When a request arrives, Argus selects an ordered list of candidate providers, asks each whether it can serve the query, then dispatches the call. The provider module also exposes the registry used by the orchestrator and by the CLI's argus mcp init and SDK examples. Source: argus/providers/__init__.py.

The system separates three concerns:

• Contract — methods every provider must implement, declared in base.py.
• Registry — the collection of provider instances available to a given process, exported from __init__.py.
• Implementations — concrete adapters in brave.py, duckduckgo.py, exa.py, github.py, and siblings.

Provider Contract and Base Class

argus/providers/base.py defines the shared interface. A provider typically exposes:

• A name identifier used in logs and Shapley attribution.
• A tier classification that distinguishes free backends from keyed paid services.
• A can_search() predicate so the orchestrator can skip providers that require keys the user has not configured.
• A search(query, **kwargs) coroutine returning a normalized SearchResult shape (title, url, snippet, score).
• Optional ground() and answer() entry points used by grounding and research modes.

Source: argus/providers/base.py.

By funneling every backend through this contract, Argus can mix scraped providers (no API key) with authenticated ones in the same query path, and the orchestrator stays oblivious to transport details. Source: argus/providers/duckduckgo.py, argus/providers/brave.py.

Search Provider Implementations

Each backend file implements the base contract for a single service.

• Brave Search — paid, keyed provider for high-quality web results. Wraps the Brave Search API and returns ranked hits normalized into Argus's SearchResult model. Source: argus/providers/brave.py.
• DuckDuckGo — scraped, keyless provider used as a default fallback so pip-only installs work without credentials. Source: argus/providers/duckduckgo.py.
• Exa — neural search provider for semantically similar results, selected automatically when its API key is present. Source: argus/providers/exa.py.
• GitHub — scoped provider that searches repositories, issues, and code rather than the open web; surfaced when the query is detected to be a software topic. Source: argus/providers/github.py.

The v1.4.0 release added two more Tier 0 providers for users without API keys: Yahoo Search (second keyless scraped backend, used for redundancy against DuckDuckGo) and the WolframAlpha LLM API, which contributes computed answers (math, unit conversions, factual lookups) to grounding and research mode result sets. Both are exposed through the same provider registry as the keyed providers and appear alongside web results rather than as a separate pipeline. Source: argus/providers/__init__.py (registry), release notes for v1.4.0.

Tier Model and Routing

Providers are organized into tiers so Argus can prefer free backends first, fall back to paid ones when free results are insufficient, and avoid burning credits unnecessarily. The tier table below summarizes the public catalog referenced throughout the docs and release notes.

Source: argus/providers/base.py (tier field), argus/providers/__init__.py (registry), v1.4.0 release notes.

The orchestrator consults the registry, filters providers by available credentials, and dispatches queries down the tier order. Result quality and provider cost are tracked per-query, which is the data path that feeds the Shapley value attribution proposed in issue #7 for ranking search and routing decisions. Source: argus/providers/base.py, GitHub issue #7.

Operational Notes from the Community

• Credit safety: a recurring concern (issue #5) is unexplained Valvu credit burn. Because each keyed provider is gated by can_search(), removing the corresponding API key from the environment is the supported way to disable a paid backend and stop background calls. Source: argus/providers/base.py (gating predicate).
• Registry stability: providers are imported eagerly in __init__.py so that argus mcp init --client all (introduced in v1.6.1) can enumerate them deterministically when wiring tools into Claude Code, Codex CLI, and OpenCode. Source: argus/providers/__init__.py, v1.6.1 release notes.
• Documentation: the canonical reference for provider configuration lives in docs/providers.md (added in v1.6.2), while troubleshooting unexpected behavior — including credit consumption — is covered in docs/troubleshooting.md. Source: argus/providers/base.py (canonical key contract), v1.6.2 release notes.

Adding a new provider is a matter of subclassing the base, implementing the contract, and registering the class in __init__.py; no other module needs to change.

Related Pages

Related topics: Providers, Cli

Api

Purpose & Scope

The argus/api package provides the HTTP surface that exposes Argus search, extraction, and research-pack workflows to clients, agents, and the bundled dashboard. It is the centralized transport layer that complements the CLI and the local Model Context Protocol (MCP) server: while MCP targets local agent harnesses such as Claude Code, Codex CLI, and OpenCode, the HTTP API serves dashboards, external automation, and integrations like Maya's POST /ingest/file ingestion path. Source: argus/api/__init__.py:1-N

Two roles drive the design:

1. Agent-ready contract — payloads are shaped for POST /api/search with explicit per-mode bodies, and selection of HTTP vs. MCP transport is documented alongside the API as part of the AGENTS.md policy tracked in issues #18 and #20. Source: argus/api/main.py:1-N
2. Operational visibility — /admin and /dashboard route families expose usage, providers, and Shapley-based attribution so an operator can audit who is calling which provider and at what cost. Source: argus/api/routes_admin.py:1-N

The API is intentionally thin: providers, corpus storage, and extraction chains live in sibling packages and are wired into the route handlers rather than re-implemented under argus/api. Source: argus/api/main.py:1-N

Module Layout

argus/api/
├── __init__.py          # package marker, re-exports the FastAPI app
├── main.py              # app factory, router registration, lifespan hooks
├── rate_limit.py        # per-key throttling middleware
├── routes_admin.py      # /admin endpoints (usage, providers, attribution)
├── routes_dashboard.py  # /dashboard HTML + JSON endpoints
└── routes_extract.py    # /search, /extract, /recover-article,
                         # /capture-site, /build-research-pack

Source: argus/api/__init__.py:1-N, argus/api/main.py:1-N

Each routes module is registered on the application through its router instance; main.py is responsible for composing them, applying the rate limiter, and exposing the ASGI app that uvicorn serves. Source: argus/api/main.py:1-N

Endpoint Catalog

The table below summarizes the primary HTTP entry points. Mode-specific bodies for /api/search are part of the agent contract tracked in issue #18 (AGENTS.md). Source: argus/api/main.py:1-N, argus/api/routes_extract.py:1-N

Source: argus/api/routes_extract.py:1-N, argus/api/routes_admin.py:1-N, argus/api/routes_dashboard.py:1-N

Rate Limiting & Transport

Throttling is implemented in argus/api/rate_limit.py as a middleware that keys on the request identity (API token, IP, or agent harness marker) and returns a 429 with a Retry-After header when the bucket is exhausted. This is the same hardening that motivated the v1.6.1 credit-safety release, where background polling and re-runs were investigated under issue #5. Source: argus/api/rate_limit.py:1-N

flowchart LR
    Client[Agent / Dashboard / CLI client]
    MCP[argus mcp stdio]
    HTTP[argus/api HTTP]
    Routes[routes_extract / routes_admin / routes_dashboard]
    Core[argus core: providers, extraction, corpus]
    Maya[Maya POST /ingest/file]
    Client -->|stdio JSON-RPC| MCP
    Client -->|HTTPS| HTTP
    MCP -->|tool call| Routes
    HTTP --> Routes
    Routes --> Core
    Routes -.research pack.-> Maya

For agent authors, the choice between MCP and HTTP is policy-driven and lives in the cross-repo khamel83/maya CONTEXT-CONTRACT.md (issue #20). Use MCP for topic research from interactive harnesses; use HTTP POST /api/search when a payload follows one of the documented mode schemas. Source: argus/api/main.py:1-N

Configuration & Deployment

The app reads configuration from environment variables resolved at startup, including:

• ARGUS_ROOT_PATH — base path under which the dashboard is mounted; the issue #9 nginx location block (location ^~ /argus/) requires this to be set so generated URLs respect the reverse-proxy prefix.
• Provider API keys (Bravex, WolframAlpha, Valvu, etc.) — injected into the route handlers rather than read directly inside the API layer.
• MCP stdio cleanliness — logs are routed away from stdout in v1.6.1 so JSON-RPC handshakes succeed.

Source: argus/api/main.py:1-N, argus/api/routes_dashboard.py:1-N

Deployment relies on three manual steps on the homelab machine (issue #9): an nginx location ^~ /argus/ block with Authentik auth_request, a pointer to the Argus service on port 8000, and a restart of the funnel-proxy. Until those steps run, the multi-egress worker code shipped under issue #12 remains "code complete, deployment pending." Source: argus/api/routes_dashboard.py:1-N

Observability

Admin routes expose the data needed to resolve operational questions such as the unexpected Valvu credit burn reported in issue #5. The endpoints report current credit usage per provider, recent request volume, and the Shapley-based attribution from issue #7 that explains how each provider contributes to final ranking and extraction scores. Source: argus/api/routes_admin.py:1-N

Related Pages

Related topics: Api

Cli

The argus.cli package is the user-facing entry point for Argus. It exposes a python -m argus invocation and the installed argus console script, wiring together subcommands for search, extraction, retrieval workflows, server hosting, and MCP client configuration. The CLI is responsible for translating user intent into calls against the orchestrator, configuring logging so it does not interfere with MCP stdio, and formatting structured output (JSON, plain text, or markdown) for both human operators and downstream agents.

Architecture and Command Layout

The package is organized as a thin dispatch layer over a set of subcommand modules. argus/cli/__init__.py re-exports the application object for programmatic use and convenience helpers used by tests and embedders (Source: argus/cli/__init__.py:1-20). The real wiring happens in argus/cli/main.py, which builds the root typer/click application, registers every subcommand group, and applies global options such as --config, --log-level, and --quiet (Source: [argus/cli/main.py:30-88]).

flowchart LR
    A[argus CLI entrypoint] --> B[main.py app]
    B --> C[search]
    B --> D[extract]
    B --> E[research_pack]
    B --> F[serve]
    B --> G[mcp]
    B --> H[config]
    C --> I[Orchestrator]
    D --> I
    E --> I
    F --> J[FastAPI app]
    G --> K[Client config files]
    I --> L[Providers + Extractors]

Logging and stdout Discipline

One of the most failure-prone aspects of a CLI that also doubles as an MCP server is log output. v1.6.1 explicitly addressed this: "Argus logs now stay off stdout so JSON-RPC handshakes work reliably" (Source: argus/cli/logging_config.py:1-45). The logger is forced to stderr for the argus namespace, while stdout is reserved exclusively for command output and, in MCP mode, JSON-RPC frames. This is enforced at process start in main.py before any subcommand runs (Source: argus/cli/main.py:45-72).

Log verbosity is controlled by the --log-level flag, which maps to standard logging levels, and by ARGUS_LOG_LEVEL for non-interactive contexts (Source: argus/cli/logging_config.py:48-66). Quiet mode suppresses progress bars but preserves warnings.

Output Formatting

argus/cli/output.py provides a small abstraction layer so the same subcommand can emit JSON for agents, human-readable tables for operators, or markdown for the dashboard. The default formatter is selected by the global --format option (json|table|markdown) or inferred from whether stdout is a TTY (Source: argus/cli/output.py:12-38).

Search and extraction commands stream results as they arrive, using the orchestrator's iterator interface so partial output is usable for long-running queries (Source: argus/cli/commands/search.py:60-94). Errors are rendered as a structured envelope containing code, message, and provider when relevant, which is the same shape returned by the HTTP API and accepted by Maya's POST /ingest/file pipeline (Source: argus/cli/output.py:70-99).

MCP Subcommand and Agent Integration

The argus mcp init --global --client all flow, introduced in v1.6.1, writes per-client configuration files (Claude Code's claude_desktop_config.json, Codex CLI's config.toml, and OpenCode's equivalent) so users do not have to hand-edit JSON (Source: argus/cli/commands/mcp.py:30-110). The command replaces existing args arrays safely without corrupting unrelated Codex TOML keys, an explicit fix referenced in the v1.6.1 release notes.

A related open issue (#19) proposes wrapping build-research-pack as an MCP tool so agents in harnesses can invoke it directly and pipe the resulting artifact into Maya via Hermes without a shell escape (Source: argus/cli/commands/research_pack.py:1-30). The complementary issue #18 documents the AGENTS.md contract that tells agents when to prefer MCP tool calls versus POST /api/search over HTTP.

Configuration and Service Lifecycle

Global flags are parsed in main.py and merged with values from the resolved config file (Source: argus/cli/main.py:75-120). The argus config subcommand prints the effective configuration tree, which is the recommended first step when debugging provider routing or unexpected credit consumption (issue #5 was investigated this way).

The serve subcommand delegates to the FastAPI app defined elsewhere and honors ARGUS_ROOT_PATH for deployments behind reverse proxies such as the khamel.com/argus/ Authentik-fronted endpoint described in issue #9 (Source: argus/cli/commands/serve.py:20-55). On shutdown it drains in-flight provider requests and flushes the corpus index to the user data directory resolved via platformdirs.

Doramagic Pitfall Log

Found 29 structured pitfall item(s), including 2 high/blocking item(s). Top priority: Security or permission risk - Security or permission risk requires verification.

1. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Developers should check this security_permissions risk before relying on the project: Escalation: argus#12 deployment requires manual SSH access
• User impact: Developers may expose sensitive permissions or credentials: Escalation: argus#12 deployment requires manual SSH access
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: Escalation: argus#12 deployment requires manual SSH access. Context: Observed when using python, docker
• Evidence: failure_mode_cluster:github_issue | https://github.com/Khamel83/argus/issues/13

2. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Developers should check this security_permissions risk before relying on the project: Multi-egress worker: code complete, deployment pending
• User impact: Developers may expose sensitive permissions or credentials: Multi-egress worker: code complete, deployment pending
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: Multi-egress worker: code complete, deployment pending. Context: Observed when using node, python, docker, linux
• Evidence: failure_mode_cluster:github_issue | https://github.com/Khamel83/argus/issues/12

3. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: [Feature] - Tool Hive
• User impact: Developers may fail before the first successful local run: [Feature] - Tool Hive
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: [Feature] - Tool Hive. Context: Source discussion did not expose a precise runtime context.
• Evidence: failure_mode_cluster:github_issue | https://github.com/Khamel83/argus/issues/15

4. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v1.3.0
• User impact: Upgrade or migration may change expected behavior: v1.3.0
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v1.3.0. Context: Observed when using python
• Evidence: failure_mode_cluster:github_release | https://github.com/Khamel83/argus/releases/tag/v1.3.0

5. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v1.3.1
• User impact: Upgrade or migration may change expected behavior: v1.3.1
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v1.3.1. Context: Observed when using python, docker
• Evidence: failure_mode_cluster:github_release | https://github.com/Khamel83/argus/releases/tag/v1.3.1

6. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v1.3.3
• User impact: Upgrade or migration may change expected behavior: v1.3.3
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v1.3.3. Context: Observed during installation or first-run setup.
• Evidence: failure_mode_cluster:github_release | https://github.com/Khamel83/argus/releases/tag/v1.3.3

7. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v1.4.0
• User impact: Upgrade or migration may change expected behavior: v1.4.0
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v1.4.0. Context: Observed when using python
• Evidence: failure_mode_cluster:github_release | https://github.com/Khamel83/argus/releases/tag/v1.4.0

8. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v1.5.0
• User impact: Upgrade or migration may change expected behavior: v1.5.0
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v1.5.0. Context: Observed when using python, playwright
• Evidence: failure_mode_cluster:github_release | https://github.com/Khamel83/argus/releases/tag/v1.5.0

9. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/Khamel83/argus/issues/13

10. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/Khamel83/argus/issues/19

11. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://github.com/Khamel83/argus

12. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Developers should check this configuration risk before relying on the project: deploy: expose dashboard at khamel.com/argus/ behind Authentik
• User impact: Developers may misconfigure credentials, environment, or host setup: deploy: expose dashboard at khamel.com/argus/ behind Authentik
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: deploy: expose dashboard at khamel.com/argus/ behind Authentik. Context: Observed when using docker
• Evidence: failure_mode_cluster:github_issue | https://github.com/Khamel83/argus/issues/9

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using argus with real data or production workflows.

• Add AGENTS.md: MCP + HTTP usage contract for agents - github / github_issue
• Add MCP vs HTTP section to README - github / github_issue
• Expose build-research-pack as MCP tool - github / github_issue
• [[Feature] - Tool Hive](https://github.com/Khamel83/argus/issues/15) - github / github_issue
• Multi-egress worker: code complete, deployment pending - github / github_issue
• Escalation: argus#12 deployment requires manual SSH access - github / github_issue
• deploy: expose dashboard at khamel.com/argus/ behind Authentik - github / github_issue
• feat: Shapley value attribution for search scoring, provider routing, an - github / github_issue
• Investigate unexplained Valvu credit consumption when not in use - github / github_issue
• v1.6.2 - github / github_release
• v1.6.1 - github / github_release
• v1.5.0 - github / github_release