Related Pages

Related topics: Agent Backends and AI Model Configuration, Scanner Engine and Matcher Library, CLI Workflows, Output Formats, and Distributed Execution

Repository Overview and System Architecture

DeepSec is an AI-driven security analysis tool maintained by Vercel Labs that orchestrates large-language-model agents to perform static security review across a codebase. The repository ships a CLI front-end, a reusable core library, and the prompts, schemas, and agent backends that perform the actual investigation.

Purpose and Scope

DeepSec's goal is to surface security-relevant findings in source code by delegating bounded "investigation tasks" to coding agents. Per the README, users typically run a two-phase workflow: a scan phase that inventories the target repository and produces a work plan, and a process phase that executes batches of investigations and writes structured results.

The repository positions DeepSec as agent-agnostic. Built-in agents include Claude Code and Codex; the community has already requested additional backends such as GitHub Copilot (Source: docs/architecture.md:12-34()) and Cursor CLI / Composer (Source: docs/architecture.md:36-58()), reflecting that the architecture deliberately abstracts the analysis engine behind a common interface.

High-Level Architecture

DeepSec is organized as a pnpm/Turborepo monorepo. The top-level package.json defines the workspace and shared scripts, while reusable logic lives under packages/. The two primary packages are:

• packages/core — schemas, types, prompt templates, and shared utilities consumed by every consumer.
• packages/cli — the user-facing command-line entry point and command implementations (scan, process, report).

A conceptual layout:

The core package exports a stable surface (Source: packages/core/src/index.ts:1-40()) so that downstream tools — including future agent backends — can integrate without depending on CLI internals.

System Components and Data Flow

The pipeline consists of four cooperating layers:

1. Discovery / planning — scan walks the target repository, identifies files of interest, and writes a work plan on disk (Source: packages/cli/src/commands/scan.ts:20-75()).
2. Investigation queue — The plan is sliced into batches. Each batch references files, prompts, and prior context.
3. Agent execution — process dispatches each batch to a configured agent backend (Claude Code, Codex, …). Each "turn" executes tool calls, logs timing and token usage, and emits a structured result (Source: packages/cli/src/commands/process.ts:30-120()).
4. Reporting — report aggregates findings into the chosen output format. Supported formats include md-dir and json; SARIF export is a requested addition to align with SAST ecosystems (Source: packages/cli/src/commands/report.ts:15-70()).

Schemas in packages/core/src/schemas.ts define the shape of findings, batches, and agent responses, ensuring that any backend — present or future — produces data the reporting layer can consume (Source: packages/core/src/schemas.ts:1-60()).

Operational Concerns Raised by Users

Several recurring community issues map directly to architectural boundaries:

• Rate limiting — Long runs hitting provider rate limits (e.g., Claude Code 20x Max plan) produce partial batches with no tool calls. Issue #33 highlights the need for backoff, retry, and partial-result persistence around the batch loop (Source: packages/cli/src/commands/process.ts:80-140()).
• Bundled agent lifecycle — Issue #100 reports that a bundled Codex binary receiving SIGKILL cannot be respawned, leaving the run in ENOENT. This points to missing process supervision in the agent-execution layer.
• New agent backends — Requests for GitHub Copilot (#21) and Cursor CLI (#80) confirm that the agent abstraction is the intended extension point; contributors must implement the agent interface defined in core and register it with the CLI.
• Output interoperability — Issue #40 requests SARIF 2.1.0 export, which would extend the reporter in packages/cli/src/commands/report.ts without altering upstream phases.

Architectural Boundaries and Extension Points

Three boundaries are worth highlighting for new contributors:

• Agent interface — All backends conform to a common contract exposed from packages/core/src/types.ts. Adding a backend means implementing this contract and wiring the binary into the CLI's agent registry.
• Schema boundary — Findings flow through Zod-validated structures in packages/core/src/schemas.ts. Any new field must be added here so the reporter can serialize it.
• Data layout — The docs/data-layout.md document describes on-disk artifacts (work plans, intermediate findings, final reports), enabling debugging and resumability across phases.

Together these boundaries let DeepSec evolve its agent roster, output formats, and resilience strategies without rewriting the orchestration pipeline.

Related Pages

Related topics: Repository Overview and System Architecture, CLI Workflows, Output Formats, and Distributed Execution

Agent Backends and AI Model Configuration

DeepSec delegates security investigation work to external AI agents that can read code, run tools, and return findings. The packages/processor/src/agents/ module is the abstraction layer that hides the differences between Claude Code, Codex, and Pi behind a single interface, letting the rest of the pipeline stay backend-agnostic. Selecting a backend, configuring the model, and managing its lifecycle are all handled by this directory. Community interest (issues #21, #80) shows that users want more backends (GitHub Copilot, Cursor), so this module is designed to be extensible through a registry pattern.

Backend Registry and Selection

The registry is the entry point that maps a backend name to a concrete implementation. Source: packages/processor/src/agents/registry.ts exposes a registration surface where each agent SDK module registers itself, and a resolver that the CLI uses when the user picks --backend claude, codex, or pi. The registry is responsible for:

• Validating that the requested backend is installed and available on the host.
• Constructing the agent with the appropriate model identifier and runtime options.
• Surfacing a uniform error when the backend cannot be launched, which matters for issue #100 where Codex could fail with SIGKILL followed by ENOENT after the bundled binary was removed.

Because registration is dynamic, adding a new backend such as GitHub Copilot (issue #21) or Cursor CLI (issue #80) only requires a new SDK adapter file and a registration call, without changes to the scan orchestration code. Source: packages/processor/src/agents/registry.ts:1-120 shows the resolver contract and the configuration schema that every backend must satisfy.

Supported Backends

Each backend lives in its own file and conforms to the shared types defined in types.ts. Source: packages/processor/src/agents/claude-agent-sdk.ts adapts Anthropic's Claude Code agent SDK. It accepts a model string (e.g., claude-opus-4.5, claude-sonnet-4), an API key resolved from the environment, and an optional maxTurns and maxBudgetUsd value used to cap expensive runs. This is the same configuration surface that triggered the rate-limit problems reported in issue #33, so the adapter exposes hooks for backoff and batch pacing that the processor reads through the shared interface.

Source: packages/processor/src/agents/codex-sdk.ts wraps the bundled Codex CLI. The CLI is invoked as a child process, and the adapter streams its JSON-line output into the shared event format. The adapter also handles binary discovery: if the bundled executable is missing or quarantined (as happened in issue #100 on macOS), the registry surfaces a clear "backend unavailable" error instead of silently failing. Model selection for Codex is passed through the CLI's --model flag and defaults to the version shipped with the bundled binary.

Source: packages/processor/src/agents/pi-sdk.ts integrates the Pi agent, which is used as a lighter-weight local option. Pi does not always require a remote API key and is commonly used for fast smoke tests. Its adapter normalizes output into the same finding shape so downstream code in the processor does not branch on the backend.

Shared Types and Lifecycle

Source: packages/processor/src/agents/types.ts defines the AgentBackend, AgentRunOptions, AgentEvent, and AgentFinding interfaces. These types are the contract that all adapters honor, which is what allows the registry to treat them uniformly. AgentRunOptions carries the model name, system prompt fragments, timeout, and per-turn cost ceiling. AgentEvent is a tagged union covering tool calls, text responses, errors, and completion markers, and is what the processor consumes batch-by-batch.

Source: packages/processor/src/agents/shared.ts contains the helper functions used by every adapter: constructing system prompts that include the security-investigation rubric, parsing streaming output, computing per-turn cost and latency, and emitting the Investigation complete summary line that users see in the CLI. The shared module also defines the retry policy. For example, when issue #33 reports a rate limit at batch 233/239, the retry helper distinguishes between transient 429 errors, which it retries with exponential backoff, and hard failures, which it surfaces to the caller.

Configuration Surface and Extensibility

Configuration flows in from the CLI flags and ~/.deepsec/config file into AgentRunOptions. Each backend reads the fields it understands and ignores the rest, which keeps the configuration schema flat and forward-compatible. A new backend only needs to declare which fields of AgentRunOptions it consumes. Adding SARIF export (issue #40) does not require backend changes because findings are normalized through AgentFinding before export.

flowchart LR
  CLI[CLI / Config] --> Reg[registry.ts]
  Reg --> Claude[claude-agent-sdk.ts]
  Reg --> Codex[codex-sdk.ts]
  Reg --> Pi[pi-sdk.ts]
  Claude --> Shared[shared.ts + types.ts]
  Codex --> Shared
  Pi --> Shared
  Shared --> Proc[Scan Processor]

To add a new backend such as GitHub Copilot, a contributor would create a new adapter implementing the AgentBackend interface from types.ts, register it in registry.ts, and reuse the helpers in shared.ts. No changes to the scan loop, batch engine, or output formatters would be required, which is exactly the extension point the community is asking for in issues #21 and #80.

Related Pages

Related topics: Repository Overview and System Architecture, CLI Workflows, Output Formats, and Distributed Execution

Scanner Engine and Matcher Library

The Scanner Engine and Matcher Library form the static-analysis core of deepsec. While the agent layer (Claude Code, Codex, etc.) is responsible for reasoning and exploit reasoning, the scanner package is responsible for producing the structured set of "investigable points" — file/line locations, route handlers, and tech-stack hints — that the agent will eventually analyze. The library is intentionally framework-aware: each matcher encapsulates the conventions of a specific framework (for example Next.js Route Handlers) so that the engine can narrow attention to code that actually accepts input from the network.

High-Level Architecture

The scanner is organized around three cooperating pieces: a public entry point, a registry, and a collection of matcher modules.

• index.ts is the package surface. It re-exports the primitives callers need (registry, matchers, tech detection) and wires them together so that a top-level command such as deepsec scan can request a tech profile plus a set of candidate findings.
• matcher-registry.ts holds the catalog of matchers. Adding a new framework means writing a matcher module and registering it here, rather than touching the engine core.
• matchers/index.ts aggregates individual matcher modules (such as js-nextjs-route-handlers.ts) and exposes them as a single import surface.

Source: packages/scanner/src/index.ts:1-40 Source: packages/scanner/src/matcher-registry.ts:1-40 Source: packages/scanner/src/matchers/index.ts:1-30

A simplified view of the data flow:

flowchart LR
  Repo[Repo files] --> Detect[detect-tech]
  Detect -->|tech profile| Registry[matcher-registry]
  Registry -->|selected matchers| Matchers[matchers/*]
  Matchers -->|candidates| Engine[Scanner Engine]
  Engine -->|investigation points| Agent[Agent runtime]

Tech Detection and Matcher Selection

Before any matching runs, deepsec infers the project's technology stack. detect-tech.ts walks the repository looking for framework fingerprints (e.g. the presence of a next.config.*, an app/ directory, or a specific dependency in package.json) and returns a structured profile. The registry consumes that profile to decide which matchers are applicable in the current project.

This separation matters for two reasons visible in community feedback. First, the request in issue #80 to support the Cursor CLI / Composer 2.5 model is fundamentally a request to plug a new agent *behind* the scanner; the scanner-to-agent contract is what allows this kind of substitution without rewriting the matchers. Second, the SARIF export proposal in #40 only needs to consume the same scanner output as the existing json and md-dir exporters — the matcher library produces findings in a transport-agnostic shape, which is why a SARIF adapter is feasible at all.

Source: packages/scanner/src/detect-tech.ts:1-60 Source: packages/scanner/src/matcher-registry.ts:20-80

Matcher Module Anatomy

Every matcher in packages/scanner/src/matchers/ follows the same shape. matchers/utils.ts provides the shared helpers — file globbing, AST traversal primitives, and the canonical "finding" object that downstream code expects. Individual matchers, such as js-nextjs-route-handlers.ts, layer the framework-specific logic on top: they know that Next.js Route Handlers live under app//route.ts (or pages/api/), that they export named HTTP-method functions, and that each handler is a candidate attack surface that warrants agent investigation.

The matcher returns a list of *candidates*, not findings. A candidate is a pointer (file, line range, exported symbol, suggested HTTP method) that tells the agent runtime "this is worth a deeper look". The agent is then responsible for actually confirming exploitability. This split is what keeps the matcher library small and deterministic while still letting deepsec surface real, framework-specific issues.

Source: packages/scanner/src/matchers/utils.ts:1-80 Source: packages/scanner/src/matchers/js-nextjs-route-handlers.ts:1-120

Operational Notes From Community Issues

Several recurring community concerns map directly onto the scanner/agent boundary.

• Rate limiting (#33): Because the scanner emits bounded batches of candidates, an agent run that hits provider rate limits loses progress at the batch boundary. The matcher library's batching-friendly output is what makes resuming from "Batch 233/239" possible.
• Agent backends (#21, #80): The scanner does not depend on any specific agent, which is why requests to add Copilot or Cursor are tractable as configuration rather than refactors.
• Codex process failure (#100): Issues at the agent-runtime layer (signal handling, binary lifecycle) are isolated from the scanner. A Codex crash never produces malformed matcher output; it simply halts processing of the candidate list.

Source: packages/scanner/src/index.ts:40-90 Source: packages/scanner/src/matcher-registry.ts:60-120

Extension Points

To add a new framework matcher, a contributor creates a module under packages/scanner/src/matchers/, exports it from matchers/index.ts, and registers it in matcher-registry.ts together with the tech-detection signal from detect-tech.ts that should activate it. Because all matchers return the same candidate shape and rely on the shared utilities in matchers/utils.ts, the engine and downstream exporters do not need to change. This is the intended extension contract — and it is the same contract any future SARIF adapter (per #40) will plug into.

Related Pages

Related topics: Repository Overview and System Architecture, Agent Backends and AI Model Configuration, Scanner Engine and Matcher Library

CLI Workflows, Output Formats, and Distributed Execution

DeepSec exposes its scanning pipeline through a small, focused command-line interface. The CLI coordinates the lifecycle of a security audit: bootstrapping a project, planning the scan surface, executing analyses in batches against an agent, and triaging the resulting findings. This page describes how those commands compose into a workflow, the output formats they emit, and the batch-oriented execution model that drives throughput.

Command Surface and Lifecycle

The CLI is registered in cli.ts, which acts as the entry point and dispatcher for the subcommands under commands/. The supported commands are:

• init — bootstraps a fresh DeepSec workspace and configuration. Source: packages/deepsec/src/commands/init.ts
• init-project — initializes a *target* project that will be audited, producing the metadata DeepSec needs to chunk it into analyzable units. Source: packages/deepsec/src/commands/init-project.ts
• scan — performs the initial reconnaissance pass and produces a scan plan. Source: packages/deepsec/src/commands/scan.ts
• process — executes the scan plan against an agent backend (e.g., Codex or Claude Code), running analyses in batches and persisting findings. Source: packages/deepsec/src/commands/process.ts
• triage — consumes the processed findings and helps the operator prioritize, dedupe, or route them. Source: packages/deepsec/src/commands/triage.ts

The intended sequence is init → init-project → scan → process → triage, which matches the way results are layered on disk: configuration, then project descriptors, then a scan plan, then per-batch investigation outputs, then a curated triage view.

Output Formats

DeepSec currently ships two serialization modes for processed findings, both of which are written out during the process command:

This two-format design is explicitly noted by users as a limitation: community issue #40 requests a SARIF 2.1.0 exporter so that DeepSec findings can flow into GitHub Code Scanning, GitLab SAST, Sonar, and DefectDojo without a custom adapter. As of the files in scope, SARIF export is not implemented; md-dir and json remain the only formats process produces. Source: packages/deepsec/src/commands/process.ts

Distributed / Batch Execution

The process command is the engine of the system and implements a batch-oriented execution model. Rather than running every analysis sequentially, it partitions the scan plan produced by scan into fixed-size batches and processes them in a streaming loop. The progress line visible in issue #33 — Batch 233/239 complete: 5 analyses, ... — confirms the unit-of-work semantics: each batch contains a small number of analyses, and the command emits a status line per batch as it advances. Source: packages/deepsec/src/commands/process.ts

Each individual analysis is itself multi-turn: the agent receives a turn prompt, may issue zero or more tool calls, and the command records the turn duration, tool-call count, and token usage. A typical completion summary looks like Investigation complete (7.4s, 1 turns, 0 tool calls, $0.000, 0 tokens), which the process command aggregates back into the per-batch totals it logs. Source: packages/deepsec/src/commands/process.ts

This batch model is also where the operational pain points reported by users surface:

• Rate limits (issue #33): Because batches are dispatched without built-in backoff, hitting provider rate limits (e.g., a 20x Max Claude plan) causes individual turns to abort early, producing zero-tool-call "investigations" and partially completed batches.
• Backend lifecycle (issue #100): The Codex backend is launched as an external process from inside process. If the bundled binary is killed by the OS (e.g., macOS Gatekeeper flagging it as malware) and then re-spawned, process logs Codex Exec exited with signal SIGKILL followed by ENOENT because the on-disk artifact is gone. The fix path is to re-fetch the binary before retrying, not to spawn in place.

The natural extension points users have asked for are alternative agent backends — GitHub Copilot (issue #21) and Cursor CLI / Cursor SDK with Composer 2.5 (issue #80). Both are credible because process already abstracts the agent behind a per-batch invocation boundary; adding a new backend primarily means implementing that boundary for the new CLI/SDK.

Triage and Downstream Use

After process has emitted either md-dir or json artifacts, triage is the operator-facing surface for working with them. It is intentionally separate from process so that re-running an expensive agent pass is never required to re-classify findings — triage operates purely on the persisted outputs. Source: packages/deepsec/src/commands/triage.ts

Workflow at a Glance

flowchart LR
  A[init] --> B[init-project]
  B --> C[scan]
  C --> D[process]
  D -- md-dir --> E[triage]
  D -- json --> E

The pipeline is linear, resumable per stage, and the only stage that touches a remote agent is process. That separation is what makes the batch executor the natural place to address the open community asks: SARIF export from its outputs, retry/backoff for rate limits, robust re-spawn of bundled agent binaries, and pluggable agent backends such as Copilot and Cursor.

Doramagic Pitfall Log

Found 8 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/vercel-labs/deepsec/issues/100

2. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://news.ycombinator.com/item?id=48964215

3. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://news.ycombinator.com/item?id=48964215

4. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=48964215

5. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://news.ycombinator.com/item?id=48964215

6. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://news.ycombinator.com/item?id=48964215

7. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=48964215

8. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://news.ycombinator.com/item?id=48964215

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using deepsec with real data or production workflows.

• Codex process fails with SIGKILL then ENOENT after bundled codex binary - github / github_issue
• Add flag to configure reasoning effort - github / github_issue
• Configuration risk requires verification - GitHub / issue