# https://github.com/trufflesecurity/trufflehog Project Manual Generated at: 2026-06-19 07:23:52 UTC ## Table of Contents - [Overview and Quick Start](#page-1) - [System Architecture and Data Flow](#page-2) - [Data Sources](#page-3) - [Detectors Library](#page-4) - [Verification and Permission Analysis](#page-5) - [Output Formats, Configuration, and Filtering](#page-6) - [Extensibility and Custom Detectors](#page-7) - [Deployment and CI/CD Integration](#page-8) ## Overview and Quick Start ### Related Pages Related topics: [System Architecture and Data Flow](#page-2), [Deployment and CI/CD Integration](#page-8)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) - [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md)
# Overview and Quick Start ## Purpose and Scope TruffleHog is an open-source secrets **Discovery, Classification, Validation, and Analysis** tool. In this context, a "secret" refers to a credential a machine uses to authenticate itself to another machine — including API keys, database passwords, and private encryption keys. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) The project is organized around four capabilities: - **Discovery** — Locates secrets across Git repositories, chats, wikis, logs, API testing platforms, object stores, filesystems, and more. - **Classification** — Identifies over 800 secret types and maps them to the specific identity they belong to (e.g., AWS, Stripe, Cloudflare, Postgres). - **Validation** — Performs active verification of credentials against their respective APIs to eliminate false positives. Results are reported as `verified`, `unverified`, or `unknown`. - **Analysis** — Performs a deeper inspection of a credential to enumerate its permissions and the resources it has access to. TruffleHog v3 is a complete Go rewrite with 700+ credential detectors that support active verification, native scanning sources (GitHub, GitLab, Docker, filesystems, S3, GCS, CircleCI, TravisCI), binary and document scanning, and integrations as a GitHub Action and pre-commit hook. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) The tool is released under the AGPL 3 license since v3.0, with v2 still available under GPL 2.0 in the repository history. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) ## High-Level Architecture TruffleHog follows a source → detector → output pipeline. Sources enumerate data (e.g., git commits, GitHub issues, S3 objects, Docker layers); detectors scan that data for credential patterns and optionally verify them; results are emitted in human-readable, JSON, or SARIF formats. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) ```mermaid flowchart LR A[Data Sources
git, github, s3, gcs,
docker, filesystem,
gitlab, circleci] --> B[Engine
pkg/engine] B --> C[Detectors
800+ types] C --> D{Verification} D -->|API call| E[verified] D -->|no check| F[unverified] D -->|error| G[unknown] E --> H[Output
stdout / JSON / SARIF] F --> H G --> H H --> I[Optional:
analyze / webhook] ``` The Docker source, documented separately, demonstrates the per-source pattern: it supports remote registries, the local Docker daemon, and tarball files, and walks each image layer plus build history to find embedded secrets. Source: [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) ## Quick Start ### Installation TruffleHog is distributed as a single static binary, a Docker image, a Homebrew package, a GitHub Action, and a pre-commit hook. Artifacts are signed with [Cosign](https://docs.sigstore.dev/cosign/system_config/installation/) and shipped with SHA256 checksums; users can verify signatures against the `trufflehog_{version}_checksums.txt.pem` and `.sig` files downloaded from the GitHub Releases page. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) ### Common Commands The CLI exposes one subcommand per source. The most frequently used entry points are: `git`, `github`, `github-experimental`, `gitlab`, `filesystem`, `s3`, `gcs`, `syslog`, `circleci`, `docker`, `huggingface`, `stdin`, and `analyze`. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) | Use Case | Command | |----------|---------| | Scan a public git repo (verified only) | `trufflehog git https://github.com/trufflesecurity/test_keys --results=verified` | | Scan a GitHub org for verified secrets | `trufflehog github --org=trufflesecurity --results=verified` | | Emit JSON output | `trufflehog git --results=verified --json` | | Scan a filesystem path | `trufflehog filesystem path/to/file1.txt path/to/dir` | | Scan a local git repo (cloned to temp dir for safety, see [CVE-2025-41390](https://github.com/trufflesecurity/trufflehog/security/advisories)) | `trufflehog git file://test_keys --results=verified,unknown` | | Scan an S3 bucket via assumed IAM role | `trufflehog s3 --bucket= --role-arn=` | | Scan a Docker image | `trufflehog docker --image trufflesecurity/secrets --results=verified` | | Scan stdin (e.g., decompressed S3 object) | `aws s3 cp s3://... - \| gunzip -c \| trufflehog stdin` | | Deep permission analysis | `trufflehog analyze` | ### Verification Statuses Every detected credential is reported as one of three statuses: - **verified** — the credential was confirmed valid and active by an API call (e.g., the [AWS detector](pkg/detectors/aws/aws.go) calls `GetCallerIdentity`). - **unverified** — the credential matched a pattern but was not confirmed valid. - **unknown** — verification was attempted but failed (network or API error). Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) ## Extending TruffleHog ### Custom Detectors Generic secret detection is disabled by default to suppress false positives, but users can opt in via a custom detector configuration. A community example lives at `examples/generic.yml` and can be invoked as `trufflehog filesystem --config=$PWD/generic.yml $PWD`. Custom detectors support regex, entropy, captured-group filtering, and excluded word lists, and may optionally POST matches to a webhook for verification — a `200 OK` response marks the result as `verified`. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md), [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) ### Adding New Detectors The project publishes external documentation and tooling for adding new credential detectors. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) A repository-level static-analysis tool, `hack/checksecretparts`, ensures every detector populates the `SecretParts` field on its `detectors.Result` struct. It runs in warning mode by default and can be flipped to fail mode once all detectors are migrated. Source: [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) ### Analyzer Framework For deep credential inspection, analyzers enumerate the permissions a credential has on its target system. Permissions are declared in a YAML file and translated into Go types via `go generate`, using the generator at `pkg/analyzer/generate_permissions`. Source: [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) ## Common Failure Modes and Community Requests Several long-standing community requests shape day-to-day use: - **SARIF 2.1.0 output** for CI security dashboards (Issue #578). - **`.trufflehogignore` file** to suppress specific findings by fingerprint (Issue #2687). - **Recursive filesystem scans** outside of git repositories (Issue #92) — supported via the `filesystem` subcommand. - **New detectors** (e.g., Adobe IMS OAuth2 tokens, Issue #4908) — contributed via the documented detector authoring flow. - **Shallow-since filtering** for git and GitHub scans (Issue #628). The `github-experimental` subcommand additionally enumerates Cross Fork Object References and deleted commits via the `--object-discovery` flag, writing intermediate state to `valid_hidden.txt` and `invalid.txt` under `$HOME/.trufflehog` (cleared with `--delete-cached-data`). Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) For secrets that should be ignored inline, a `trufflehog:ignore` comment on the line containing the match suppresses the finding. Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) ## See Also - [Docker Source](Docker-Source.md) — detailed configuration for the `docker` subcommand. - [Custom Detectors](Custom-Detectors.md) — author and tune user-defined detectors. - [Detector Authoring](Detector-Authoring.md) — contribute new built-in detectors. - [Analyzer Framework](Analyzer-Framework.md) — deep credential permission analysis. --- ## System Architecture and Data Flow ### Related Pages Related topics: [Data Sources](#page-3), [Detectors Library](#page-4), [Verification and Permission Analysis](#page-5)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) - [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go) - [pkg/decoders/decoders.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/decoders/decoders.go)
# System Architecture and Data Flow ## Overview and Purpose TruffleHog is a secrets **Discovery, Classification, Validation, and Analysis** tool. In this context, "secret" refers to a credential a machine uses to authenticate itself to another machine — including API keys, database passwords, private encryption keys, and similar material. The tool is implemented as a Go binary that exposes a CLI (`trufflehog`) and a library surface used by the TruffleHog Enterprise product [Source: [README.md:1-40]()]. Architecturally, TruffleHog is organized as a pipeline: 1. **Sources** produce chunks of content (files, git blobs, API responses, image layers, etc.). 2. **Decoders** normalize and extract secrets from non-plain formats (base64, gzip, etc.). 3. **Detectors** run regex and entropy heuristics against decoded chunks, classified across 800+ credential types. 4. **Verification** performs live API checks to confirm whether a candidate is actually valid. 5. **Analyzers** (optional, via `trufflehog analyze`) inspect a verified credential to enumerate its permissions and reachable resources. This separation lets new sources, decoders, and detectors be added without modifying the core engine, and it allows users to plug in **Custom Detectors** defined in YAML [Source: [examples/README.md:1-15]()]. ## High-Level Architecture ```mermaid flowchart LR CLI[CLI / multi-scan] subgraph Sources GIT[Git] GH[GitHub] GL[GitLab] FS[Filesystem] S3[S3 / GCS] DOCKER[Docker] CI[CircleCI / TravisCI] HF[Hugging Face] end subgraph Engine[pkg/engine] DEC[Decoders] DET[Detectors
~800 types] VER[Verifiers
API calls] RES[Results Channel] end AN[Analyzer
trufflehog analyze] OUT[Output: pretty / JSON / SARIF* / webhook] CLI --> Sources Sources --> DEC DEC --> DET DET --> VER VER --> RES RES --> OUT RES --> AN ``` `*` SARIF output is a commonly requested feature, tracked in community issue [#578](https://github.com/trufflesecurity/trufflehog/issues/578) [Source: [README.md:130-145]()]. ### Sources Layer Each source type implements a common `Source` interface that yields `Chunk` values containing content plus metadata (commit SHA, file path, line number, email, timestamp, repository URL). Examples include: - `git` / `github` / `gitlab` — repository histories, pull-request comments, issue comments, and Cross-Fork Object References via `github-experimental --object-discovery` [Source: [README.md:175-195]()]). - `filesystem` — recursively scans directories and individual files (issue [#92](https://github.com/trufflesecurity/trufflehog/issues/92) requested non-recursive directory support, but the `filesystem` subcommand now covers that use case) [Source: [README.md:215-220]()]). - `s3` / `gcs` — object stores, optionally with IAM role assumption for cross-account scanning [Source: [README.md:255-275]()]). - `docker` — remote registries, the local Docker daemon, or `tarball` files; scans each image layer independently and also inspects build-history metadata for secrets embedded in `RUN` commands [Source: [pkg/sources/docker/README.md:1-30]()]). - `circleci`, `travisci`, `huggingface`, `elasticsearch`, `syslog`, `stdin` — additional integrations. ### Decoders and Detectors Layer Decoders (`pkg/decoders`) peel encodings — base64, URL-encoding, gzip, JSON-in-string, and others — producing plaintext fragments. Detectors (`pkg/detectors/`) apply regex patterns and entropy filters to identify candidate credentials. The detector registry is keyed by detector ID/type (for example `DetectorType: 2, DetectorName: "AWS"` in the JSON output) [Source: [README.md:95-110]()]). A static-analysis tool, `hack/checksecretparts`, enforces that every `detectors.Result{...}` literal populates the new `SecretParts` field, so newly added detectors cannot silently regress structured output [Source: [hack/checksecretparts/README.md:1-35]()]). ### Engine and Verification Layer The engine (`pkg/engine/engine.go`) coordinates chunk production, decoding, detection, and verification. For every candidate, the engine calls the detector's verifier, which performs an API call against the service the credential claims to belong to (for example, AWS's `GetCallerIdentity` for AWS access keys) [Source: [README.md:115-125]()]). Three terminal result states are defined: | Status | Meaning | |--------|---------| | `verified` | The credential was confirmed valid by a live API check. | | `unverified` | The credential matched a detector but verification was disabled or did not run. | | `unknown` | Verification was attempted but failed (network error, API outage, etc.). | Verification results can be filtered via `--results=verified`, `--results=verified,unknown`, etc. [Source: [README.md:125-140]()]. ## Data Flow A typical scan proceeds as follows: 1. The user invokes a subcommand (for example `trufflehog git https://github.com/.../repo --results=verified`). The CLI parses flags, constructs a `Source`, and registers detectors from the default set plus any user-provided custom detectors [Source: [examples/README.md:5-15]()]). 2. The `Source` enumerates units of work (commits, files, image layers, S3 objects) and emits `Chunk`s. For local git scans, the repository is cloned into a temporary directory first to neutralize malicious `.git/config` hooks (mitigation for CVE-2025-41390); `--clone-path` overrides this location, and `--trust-local-git-config` re-enables in-place scanning for trusted repos [Source: [README.md:215-225]()]). 3. Each chunk flows through the decoder pipeline. Decoded fragments re-enter the detector stage, allowing multi-layer encoding chains to be peeled until plaintext is reached. 4. Detectors emit `detectors.Result` values with raw/redacted strings, detector type, decoder used, and source metadata. The engine forwards results through a Go channel. 5. Verifiers act on results that pass preliminary filters (entropy thresholds, exclusion word lists). The outcome (`verified` / `unverified` / `unknown`) is attached to the result [Source: [README.md:115-130]()]). 6. The output writer renders results as pretty text by default; `--json` produces structured JSON, and a webhook can be configured for CI pipelines. Lines containing findings can be suppressed by adding a `trufflehog:ignore` comment on the offending line in the source code [Source: [README.md:155-170]()]). 7. Optionally, the user can run `trufflehog analyze` to take a verified credential and enumerate its blast radius — permissions, accessible resources, and so on. Analyzer packages define their permissions in YAML, generated into Go via `go generate ./...` [Source: [pkg/analyzer/README.md:1-15]()]). ## Key Components and Extensibility | Component | Location | Role | |-----------|----------|------| | CLI | `cmd/trufflehog` (referenced from README) | Argument parsing, source construction, output | | Engine | `pkg/engine/engine.go` | Orchestration of chunks → decoders → detectors → verifiers | | Sources | `pkg/sources/` | Pluggable data ingest (git, github, docker, s3, gcs, filesystem, …) | | Decoders | `pkg/decoders` | Encoding normalization | | Detectors | `pkg/detectors/` | 800+ regex + verification implementations | | Analyzers | `pkg/analyzer` | Post-verification permission/resource inspection | | Custom Detectors | YAML via `--config` | User-defined regex + entropy + webhook verification | | Static checks | `hack/checksecretparts` | Enforces `SecretParts` field is populated | Community requests cluster around three extension points: - **Output formats**: SARIF 2.1.0 (#578) for security tooling integration. - **Ignore lists**: a `.trufflehogignore` file keyed by fingerprint (#2687). - **New detectors**: e.g. Adobe IMS OAuth2 tokens (#4908); the project publishes [Adding_Detectors_external.md](hack/docs/Adding_Detectors_external.md) to onboard contributors. The architecture's reliance on small, well-defined interfaces at each layer keeps each of these additions local — new sources, new decoders, new detectors, and new output sinks can all be added without touching the others. ## See Also - [Docker Source](docker-source.md) — registry/daemon/tarball scanning details - [Custom Detectors](custom-detectors.md) — YAML-defined detectors and webhook verification - [Adding New Detectors](adding-detectors.md) — contribution guide for new credential types - [Analyzer Framework](analyzer-framework.md) — permissions and resource enumeration --- ## Data Sources ### Related Pages Related topics: [System Architecture and Data Flow](#page-2), [Output Formats, Configuration, and Filtering](#page-6)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go) - [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md)
# Data Sources ## Purpose and Scope TruffleHog is a secrets Discovery, Classification, Validation, and Analysis tool. Data Sources are the input adapters that define *where* TruffleHog should look for credentials. A data source ingests content from a particular kind of storage or service (e.g., a Git repository, an S3 bucket, a Docker image, or a local filesystem) and emits a stream of *chunks* that are then handed to the detector engine for regex matching and verification (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). The source layer is intentionally separated from the detector layer: detectors know how to recognize and verify a credential, while sources know how to traverse a system and produce a sequence of byte chunks. This separation is what allows TruffleHog to support many disparate back-ends without duplicating detection logic (Source: [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go)). ## Supported Source Subcommands Each data source is exposed as a top-level CLI subcommand. The README documents the following sources out of the box: `git`, `github`, `github-experimental`, `gitlab`, `filesystem`, `s3`, `gcs`, `syslog`, `circleci`, `docker`, `travisci`, `elasticsearch`, `huggingface`, `stdin`, `analyze`, and `multi-scan` (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). The full subcommand listing appears in the CLI help block reproduced in the same file. | Source Subcommand | Input Type | Notable Flags | |---|---|---| | `git` | Local or remote git repository | `--since-commit`, `--branch`, `--clone-path`, `--trust-local-git-config` | | `github` / `github-experimental` | GitHub orgs, repos, issue/PR comments, hidden commits | `--org`, `--repo`, `--issue-comments`, `--pr-comments`, `--object-discovery`, `--delete-cached-data` | | `gitlab` | GitLab repositories | `--token` | | `filesystem` | Local files and directories (recursive) | positional `...` | | `s3` / `gcs` | Object storage buckets | `--bucket`, `--role-arn`, `--cloud-environment`, `--project-id` | | `docker` | Container images from registries, daemon, or tarballs | `--image` (repeatable), namespace scanning | | `circleci` / `travisci` | CI build logs and configs | `--token` | | `elasticsearch` | Elasticsearch clusters | `--cloud-id`, `--api-key` | | `huggingface` | Models, datasets, spaces, discussions | `--model`, `--dataset`, `--space`, `--org`, `--user`, `--include-discussions`, `--include-prs` | | `syslog` / `stdin` | Streaming inputs | `--format` for syslog | | `multi-scan` | A YAML file describing multiple sources | `--config` | | `analyze` | A previously found credential for permission analysis | n/a | Sources: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md), [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md). ## Architecture and Data Flow ```mermaid flowchart LR A[Data Source
git, github, s3, docker, ...] -->|emits byte chunks| B[Engine
pkg/engine/engine.go] B -->|runs regex detectors| C[Detector Pool
800+ types] C -->|HTTP/API verification| D[Verifier] D -->|verified / unverified / unknown| E[Output
JSON, SARIF, plaintext] E -->|optional deeper analysis| F[Analyzer
pkg/analyzer] ``` Each source implements the same minimal contract: enumerate the data it can see, yield chunks of bytes, and report metadata (commit, file, line, timestamp, repository, etc.). The engine then fans chunks out to detectors concurrently, deduplicates results, and forwards findings to the configured verifier or output sink. The chunk–detector boundary is what enables TruffleHog to scan containers and object stores with the same detector library used for git history (Source: [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go)). The Docker source is illustrative: it supports remote registries, the local daemon, and tarballs, and scans layers plus build history concurrently with authentication and namespace enumeration. Files larger than 50 MB are skipped automatically to keep scans bounded (Source: [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md)). ## Configuration, Filtering, and Multi-Scan Because no two environments are identical, TruffleHog exposes filters that apply across all sources. The most important global flags are: - `--include-detectors` / `--exclude-detectors` — accept protobuf names, numeric IDs, or ID ranges, allowing operators to scope a run to a small subset (e.g., only cloud providers) (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). - `--results=verified,unknown,unverified` — control which finding severities are emitted. - `--no-verification-cache`, `--force-skip-binaries`, `--force-skip-archives`, `--skip-additional-refs`. - `--detector-timeout`, `--archive-max-size`, `--archive-max-depth`, `--archive-timeout` — bound resource consumption. - `--config` — points at a YAML file consumed by the `multi-scan` subcommand, where multiple source connections (e.g., several GitHub repos and S3 buckets) can be enumerated and scanned concurrently. The schema for each connection lives in Truffle Security's [source configuration documentation](https://docs.trufflesecurity.com/scan-data-for-secrets) (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). For local git scanning, the README documents that TruffleHog now clones to a temporary directory before scanning to mitigate CVE-2025-41390 from malicious `.git/config` payloads; users can override the temp path with `--clone-path` or skip the clone step with `--trust-local-git-config` for trusted repos (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). The community has repeatedly asked for incremental scans on git history. Issue #628 requests `--shallow-since` for git/GitHub scans, which would let users bound a run to commits after a timestamp. Related ergonomics requests include #2687 (a `.trufflehogignore` file of fingerprint-based exclusions) and #92 (recursive filesystem scans outside git, which is now satisfied by `trufflehog filesystem`) (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). ## Common Failure Modes and Limitations - **Large files or unusual encodings.** Lines exceeding `bufio`'s default 64 KB token limit were historically skipped; this was fixed in v3.95.6 (PR #5022). - **Hidden or deleted commits on GitHub.** `github-experimental --object-discovery` enumerates Cross Fork Object References but takes 20 minutes to several hours per repository and writes state files under `$HOME/.trufflehog` (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). - **Docker image size.** Files larger than 50 MB are skipped, which can hide long-lived secrets in large blobs (Source: [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md)). - **Detector coverage gaps.** New credential types (e.g., Adobe IMS OAuth2 tokens, requested in #4908) are added incrementally through the documented detector authoring guide at `hack/docs/Adding_Detectors_external.md` (Source: [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)). - **Output format evolution.** Issue #578 requests SARIF 2.1.0 output for CI security tooling integration; consumers that need SARIF today should check the release notes for current support. ## See Also - Detector engine internals: [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go) - Docker source details: [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - Permission analyzers: [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - Custom detector examples: [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - Detector linting: [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) --- ## Detectors Library ### Related Pages Related topics: [System Architecture and Data Flow](#page-2), [Verification and Permission Analysis](#page-5), [Extensibility and Custom Detectors](#page-7)
Related Source Files The following source files were used to generate this page: - [pkg/detectors/detectors.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/detectors.go) - [pkg/detectors/account_filter.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/account_filter.go) - [pkg/detectors/falsepositives.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/falsepositives.go) - [pkg/detectors/endpoint_customizer.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/endpoint_customizer.go) - [pkg/detectors/multi_part_credential_provider.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/multi_part_credential_provider.go) - [pkg/detectors/generic/generic.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/generic/generic.go) - [pkg/detectors/aws/aws.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/aws/aws.go) - [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go)
# Detectors Library ## Overview and Purpose The Detectors Library is the heart of TruffleHog's credential discovery and classification capabilities. It is a large collection of individual Go packages — one per credential type — that recognize, extract, and optionally verify secrets such as API keys, OAuth tokens, database passwords, and private keys. According to the project README, TruffleHog ships with over 800 detectors, each mapping a discovered token to a specific provider identity (AWS, Stripe, Cloudflare, Postgres, etc.) and, when possible, validating the credential against the corresponding API. Source: [README.md:73-83](). Each detector follows a shared Go interface defined in [pkg/detectors/detectors.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/detectors.go). The engine feeds every scanned chunk of content (a file line, a git diff hunk, a Docker layer excerpt) to every registered detector, then aggregates matches into a normalized `Result` struct consumed by all output sinks (console, JSON, SARIF, etc.). Source: [pkg/engine/engine.go:358-365](). ## Core Data Model A detector produces a `detectors.Result` value containing the fields shown below. These fields are uniform across all ~800+ built-in detectors and the user-defined custom detectors, which makes downstream consumers (analyzers, loggers, CI integrations) trivial to write. | Field | Purpose | |-------|---------| | `DetectorType` | Stable numeric ID (e.g., `2` for AWS) used in JSON output | | `DetectorName` | Human-readable label (e.g., `"AWS"`) | | `DecoderName` | How the chunk was decoded (`PLAIN`, `BASE64`, `UTF16`, …) | | `Raw` / `Redacted` | Original match and a redacted preview safe to log | | `Verified` | Tri-state: `verified`, `unverified`, `unknown` | | `ExtraData` | Detector-specific metadata (account ID, ARN, user ID, etc.) | | `StructuredData` | Optional machine-typed payload | A recently introduced `SecretParts` field is required by the static-analysis check in [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md); the linter warns when a `detectors.Result{}` literal is constructed without populating `SecretParts`, and is intended to flip into a gating check once every detector is migrated. ## Verification Pipeline Verification is what separates TruffleHog from a pure regex scanner. For every candidate match, the engine dispatches an asynchronous verification routine whose implementation lives next to each detector. The AWS detector, for instance, calls `GetCallerIdentity` against the AWS API to confirm the key is live and to enrich the finding with the owning `account`, `arn`, and `user_id` (visible in the example JSON in the README). Source: [pkg/detectors/aws/aws.go](). Three result statuses can be returned: - **verified** — credential confirmed live by the upstream API - **unverified** — regex matched but verification was disabled (`--no-verification`) or the secret was invalidated before the call - **unknown** — verification failed because of a transient error (network, rate-limit, 5xx) Shared helpers live alongside the detectors package: - `account_filter.go` — allow/deny accounts at the engine level (useful in CI to ignore dev keys) - `falsepositives.go` — a list of well-known test fixtures (canarytokens, example.com, etc.) that should never count as live - `endpoint_customizer.go` — override the verification URL (e.g., to point at a staging or private-cloud deployment of the provider) - `multi_part_credential_provider.go` — for credentials that need two or more matched fragments (username + token, id + secret) These helpers let detector authors avoid duplicating cross-cutting concerns and keep individual detector files focused on the provider's quirks. Source: [pkg/detectors/detectors.go](). ## Custom and Generic Detectors TruffleHog intentionally does not run "generic API key" detection by default because the false-positive rate is too high. Two opt-in paths exist for users who need broader coverage: 1. **Generic detector** — enabled through `examples/generic.yml`. The README snippet shows the one-line invocation: ```bash trufflehog filesystem --config=$PWD/generic.yml $PWD ``` The generic detector is implemented in [pkg/detectors/generic/generic.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/generic/generic.go). Source: [examples/README.md:5-15](). 2. **User-defined YAML custom detectors** — documented in [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md). They support per-regex filters for **entropy**, **targeted regex** (the entire match), **capture-group regex** (the secret portion), and **excluded word lists**. When configured with a `--verifier` webhook URL, TruffleHog POSTs the match to the endpoint and treats a `200 OK` as `verified`. The same docs warn that this feature is alpha and subject to change. Source: [README.md:305-321](). ```mermaid flowchart LR A[Scanned Chunk] --> B[Engine] B --> C{Detector Pool
~800 detectors} C -->|regex match| D[detectors.Result] D --> E[account_filter] E --> F[falsepositives] F --> G[endpoint_customizer] G --> H{Verifier API
or webhook} H -->|verified| I[verified] H -->|invalid| J[unverified] H -->|error| K[unknown] I --> L[Output Sink
JSON / SARIF / Console] ``` ## Community-Driven Detector Coverage The detector roster is community-extensible. Recent release notes (v3.95.6) illustrate the cadence of detector maintenance: bug fixes such as honoring ignore tags for default-port Postgres URLs and hardening the Enigma detector against false-verified findings. Source: [release notes referenced in community context](). Open feature requests that map directly onto the detectors library include: - **#4908 — Adobe IMS OAuth2 detector.** Proposes a new package under `pkg/detectors/adobeims/` following the same pattern as the existing AWS detector, including a JWT-based verification call to the Adobe IMS endpoint. Source: [community context issue #4908](). - **#2687 — `.trufflehogignore` file.** Requests a fingerprint-based suppression list so teams can mute recurring findings without modifying source. This would integrate with `falsepositives.go` or a new sibling helper. Source: [community context issue #2687](). ## Operational Tips and Failure Modes - A detector that fails to populate `SecretParts` is flagged by `hack/checksecretparts` and may break downstream consumers expecting token fragments. Source: [hack/checksecretparts/README.md:5-12](). - Bufio's default 64 KB token limit can clip extremely long lines; v3.95.6 raised the scanner buffer so single-line payloads (PEM blocks, JWTs, large cookies) are tokenized correctly. Source: [community context v3.95.6 changelog](). - Use `--include-detectors` / `--exclude-detectors` (accepting protobuf names, IDs, or ID ranges) to scope a scan to a subset of the library. Source: [README.md CLI reference](). - Add a `trufflehog:ignore` comment on the offending line to suppress a finding in the local repo context — engine logic is at [pkg/engine/engine.go:358-365](). ## See Also - [Docker Source](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - [Analyzers](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - [Adding Detectors (external guide)](https://github.com/trufflesecurity/trufflehog/blob/main/hack/docs/Adding_Detectors_external.md) - [checksecretparts linter](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) --- ## Verification and Permission Analysis ### Related Pages Related topics: [Detectors Library](#page-4), [Output Formats, Configuration, and Filtering](#page-6)
Related Source Files The following source files were used to generate this page: - [pkg/engine/ahocorasick/ahocorasickcore.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/ahocorasick/ahocorasickcore.go) - [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go) - [pkg/verificationcache/verification_cache.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/verificationcache/verification_cache.go) - [pkg/verificationcache/result_cache.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/verificationcache/result_cache.go) - [pkg/analyzer/analyzers.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/analyzers.go) - [pkg/analyzer/cli.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/cli.go) - [pkg/analyzer/config/config.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/config/config.go) - [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - [pkg/detectors/aws/aws.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/detectors/aws/aws.go) - [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md) - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md)
# Verification and Permission Analysis ## Overview TruffleHog is positioned as a complete **Discovery, Classification, Validation, and Analysis** tool for machine credentials. Two of those phases live in dedicated subsystems that run after the high-throughput Aho-Corasick detector engine flags candidate matches. The Aho-Corasick core performs keyword/prefix pre-filtering so that only promising chunks reach per-detector verification logic, while the engine itself manages the full lifecycle from source ingestion to result emission. - **Validation** — verifying that a discovered credential is actually live. This is implemented per-detector (for example, a `GetCallerIdentity` API call for AWS) and is wrapped by a verification result cache to avoid repeat API calls during long scans. - **Analysis** — once a credential is confirmed valid, the `pkg/analyzer` subsystem inspects it for the permissions and resources it can reach. Source: [README.md]() describes TruffleHog's four-phase mission (Discovery, Classification, Validation, Analysis) and the verified/unverified/unknown result statuses emitted by the validation pipeline. Source: [pkg/engine/ahocorasick/ahocorasickcore.go]() powers the keyword pre-filter that feeds detectors. ## Credential Verification ### Result Statuses Every detector emits a `detectors.Result` whose verification step yields one of three states: - **`verified`** — the credential was confirmed valid and active by API testing. - **`unverified`** — the credential was detected but could not be confirmed valid (it may be invalid, expired, or verification may have been disabled for that detector). - **`unknown`** — verification was attempted but failed because of a network or API error. Source: [README.md]() documents these three result statuses and gives the AWS detector as the canonical example. Source: [pkg/detectors/aws/aws.go]() performs that verification via `GetCallerIdentity`. ### Caching Verification calls are expensive, so TruffleHog caches their outcomes in two cooperating layers: - `pkg/verificationcache/verification_cache.go` deduplicates verification calls — the same credential value passed through the same detector will not trigger two API calls during a single scan. - `pkg/verificationcache/result_cache.go` stores the structured `Result` payload so downstream consumers can re-emit findings without re-scanning. - A CLI flag `--no-verification-cache` disables this caching layer when needed (for example, in CI jobs that always want a fresh check). Source: [pkg/verificationcache/verification_cache.go]() implements the verification-call cache. Source: [pkg/verificationcache/result_cache.go]() implements the result cache. Source: [README.md]() lists the `--no-verification-cache` flag. ### Custom Detectors with Verification Custom regex detectors can opt into HTTP-based verification: TruffleHog POSTs the matched secret as JSON to a configured webhook endpoint; a `200 OK` response marks the credential as verified, and any network or API failure flips it to `unknown`. Filters available on custom detectors include entropy, full-match regex, capture-group regex, and excluded word lists. Note that if a custom detector declares multiple regexes (for example `hogID` and `hogToken`), the filters are applied to each one. Source: [pkg/custom_detectors/CUSTOM_DETECTORS.md]() describes the verification webhook protocol, and `examples/generic_with_filters.yml` is referenced as the canonical filter example. Source: [README.md]() notes that custom detectors are an alpha feature. ### Special Verification Modes TruffleHog ships two notable verification paths that go beyond per-detector API checks. First, **Driftwood** technology instantly verifies private keys against millions of GitHub users and billions of TLS certificates. Second, **generic JWT detection** verifies a subset of JWTs that use public-key cryptography, retrieving the public key to confirm whether the token is live. Source: [README.md]() introduces both Driftwood and the generic JWT verification flow. ## Permission Analysis Once a credential is verified, `trufflehog analyze` runs the analyzer subsystem to enumerate the permissions and accessible resources attached to the identity. ```mermaid flowchart LR A[Verified credential] --> B[analyzer.Spawn] B --> C[Analyzer implementation] C --> D[Permissions YAML] D --> E[go generate] E --> F[Typed Permissions] F --> G[AnalyzerResult] G --> H[analyze CLI output] ``` ### Analyzer Architecture The analyzer package is registered by `pkg/analyzer/analyzers.go`, which provides the dispatch layer that selects a concrete analyzer implementation for a given credential type. Each analyzer (for example `analyzers/twilio/twilio.go`) consumes a credential, calls the relevant vendor API, and returns an `AnalyzerResult` describing the identity, its permissions, and the resources it can reach. Source: [pkg/analyzer/analyzers.go]() dispatches to analyzer implementations. Source: [pkg/analyzer/README.md]() references `analyzers/twilio/twilio.go` as a concrete analyzer and `analyzers/twilio/permissions.yaml` as its permission declaration. ### Permission Definitions Permissions are declared in YAML and code-generated into typed Go constants: - **YAML format**: a `permissions.yaml` file under each analyzer directory. - **Accepted naming styles**: lower snake case (`permission_name:access_level`), kebab case (`permission-name:read`), or dot notation (`permission.name:read`). - **Codegen**: install the generator with `go install github.com/trufflesecurity/trufflehog/v3/pkg/analyzer/generate_permissions`, then run `go generate ./...` to produce the typed `Permission` values used inside `AnalyzerResult.Permissions`. Source: [pkg/analyzer/README.md]() documents the permission format, accepted naming styles, and the codegen workflow. ### Analyzer CLI and Configuration The CLI entry point in `pkg/analyzer/cli.go` is what `trufflehog analyze` actually invokes. Shared configuration for all analyzers lives in `pkg/analyzer/config/config.go`, which controls endpoints, timeouts, and feature flags applied to every analyzer at runtime. Together with `analyzers.go`, these files form the public face of the analysis subsystem. Source: [pkg/analyzer/cli.go]() implements the `analyze` subcommand. Source: [pkg/analyzer/config/config.go]() defines the shared analyzer configuration. ## CLI Usage and Common Failure Modes Typical commands that exercise both subsystems: ```bash # Scan a git repo and verify (AWS credentials will hit GetCallerIdentity) trufflehog git https://github.com/trufflesecurity/test_keys --results=verified # Run deep permission analysis on a verified credential trufflehog analyze # Force a fresh verification (skip cache) trufflehog github --org=trufflesecurity --no-verification-cache ``` Common failure modes to be aware of: - **Verification never completes** — caused by firewalls blocking egress to vendor APIs; the result is recorded as `unknown` rather than `verified`. Source: [README.md]() lists network failures as a trigger for the `unknown` state. - **Custom-detector webhook misconfiguration** — if the verification endpoint does not return `200 OK` or is unreachable, results skew toward `unknown`/`unverified`. Source: [pkg/custom_detectors/CUSTOM_DETECTORS.md]() describes the webhook protocol. - **Stale permissions** — analyzer output reflects the credential's permissions at scan time; rotated or downscoped credentials produce different results on the next run. - **Result filtering** — when a secret is wrapped in a `trufflehog:ignore` source comment, the engine skips it before verification, which can look like a false negative in CI output. Source: [pkg/engine/engine.go]() documents the `trufflehog:ignore` comment handling. ## See Also - Detector Engine and Aho-Corasick Matching - Sources: Git, GitHub, GitLab, S3, Docker - Custom Detectors Configuration --- ## Output Formats, Configuration, and Filtering ### Related Pages Related topics: [System Architecture and Data Flow](#page-2), [Extensibility and Custom Detectors](#page-7), [Deployment and CI/CD Integration](#page-8)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [pkg/output/plain.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/plain.go) - [pkg/output/json.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/json.go) - [pkg/output/legacy_json.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/legacy_json.go) - [pkg/output/github_actions.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/github_actions.go) - [pkg/config/config.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/config/config.go) - [pkg/config/detectors.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/config/detectors.go) - [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) - [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go)
# Output Formats, Configuration, and Filtering TruffleHog provides a flexible output, configuration, and filtering system that lets operators tailor scans for both human inspection and machine-driven CI/CD pipelines. The subsystem spans three concerns: the **output renderers** that serialize findings, the **YAML configuration** that defines data sources and detection rules, and the **filtering flags** that constrain which detectors run and which results are emitted. Together they form the bridge between the scanning [engine](pkg/engine/engine.go) and downstream consumers (SIEM tools, GitHub PRs, custom webhooks, or operator terminals). ## Output Formats TruffleHog supports several output renderers, each implemented as a separate package under `pkg/output/`: | Format | File | Purpose | |--------|------|---------| | Plain | [pkg/output/plain.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/plain.go) | Default human-readable output with emoji markers (`🐷🔑`) for findings | | JSON | [pkg/output/json.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/json.go) | Structured single-line JSON per result, ideal for piping into `jq` | | Legacy JSON | [pkg/output/legacy_json.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/legacy_json.go) | Backwards-compatible JSON schema used by older TruffleHog v2 integrations | | GitHub Actions | [pkg/output/github_actions.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/output/github_actions.go) | Emits [GitHub Actions workflow commands](https://docs.github.com/actions/using-workflows/workflow-commands-for-github-actions) so secrets surface as annotations in PRs | To select a format, TruffleHog uses the global `--json` flag (toggles JSON mode) or source-specific output plugins. A representative JSON record from a verified AWS finding is reproduced in the [README](README.md) and includes `SourceMetadata`, `DetectorType`, `DetectorName`, `DecoderName`, `Verified`, `Raw`, `Redacted`, and `ExtraData` fields. The SARIF 2.1.0 format is a long-standing community request tracked in [issue #578](https://github.com/trufflesecurity/trufflehog/issues/578) for CI security use cases. ## Configuration TruffleHog loads a YAML configuration file via the global `--config=CONFIG` flag. The schema is defined in [pkg/config/config.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/config/config.go) and [pkg/config/detectors.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/config/detectors.go), which together describe the `sources`, `detectors`, and verification sections. A trimmed GitHub source example from the [README](README.md) is: ```yaml sources: - connection: '@type': type.googleapis.com/sources.GitHub repositories: - https://github.com/trufflesecurity/test_keys.git unauthenticated: {} name: example config scan type: SOURCE_TYPE_GITHUB verify: true ``` Multiple `sources` entries are scanned concurrently. The same configuration file may also contain user-defined **Custom Detectors** (regex-based). An example of a generic detector for filesystem scans is shown in [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md): ```bash wget https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/examples/generic.yml trufflehog filesystem --config=$PWD/generic.yml $PWD ``` Custom Detectors may declare a verification webhook. According to the [README](README.md), when a regex matches, TruffleHog POSTs the JSON record to the configured endpoint; a `200 OK` response marks the result as verified, while transport or HTTP failures yield an `unknown` result. Filters supported on custom detectors include Shannon entropy thresholds, regex targeting the whole match, regex targeting the captured secret, and excluded word lists. The detailed syntax is in [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md). ## Filtering TruffleHog exposes several layers of filtering, each implemented as a CLI flag in the engine entry point: ```mermaid flowchart LR A[Raw scan chunks] --> B{--include-detectors
--exclude-detectors} B -->|pass| C[Detector matching] C --> D{--filter-entropy} D -->|pass| E[Verification] E --> F{--results} F -->|filtered| G[Output renderer] ``` | Flag | Effect | Source | |------|--------|--------| | `--results=verified,unknown` | Restrict emitted results to those that were API-verified or had verification errors | [README](README.md) | | `--include-detectors="all"` | Restrict detectors by Protobuf name, ID, or numeric range | [README](README.md) | | `--exclude-detectors=...` | Exclude specific detectors; takes precedence over include list | [README](README.md) | | `--filter-entropy=3.0` | Drop unverified matches below the Shannon entropy threshold | [README](README.md) | | `--no-verification-cache` | Bypass the verification cache to force fresh API calls | [README](README.md) | | `--force-skip-binaries` / `--force-skip-archives` | Skip binary or archive content | [README](README.md) | | `--detector-timeout=30s` | Per-detector time budget per chunk | [README](README.md) | | `--archive-max-size`, `--archive-max-depth`, `--archive-timeout` | Bound archive extraction cost | [README](README.md) | | `--user-agent-suffix=...` | Append a string to the verification HTTP User-Agent | [README](README.md) | In addition to CLI filters, the engine supports in-line suppression via the `trufflehog:ignore` comment on a line containing a secret. This is gated on a code path inside [pkg/engine/engine.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine.go) (lines 358–365). A more durable, fingerprint-based ignore mechanism is a recurring community ask, tracked in [issue #2687](https://github.com/trufflesecurity/trufflehog/issues/2687) for a `.trufflehogignore` file. ## Source-Specific Knobs Several scanners extend the global flag set with source-local options. The Docker source ([pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md)) supports `file://`, `docker://`, and remote registry references, optional file exclusion patterns, a 50 MB per-file cap, and multi-image enumeration under a registry namespace. The Git source supports `--shallow-since` for time-bounded history, requested in [issue #628](https://github.com/trufflesecurity/trufflehog/issues/628), and `--clone-path` plus `--trust-local-git-config` to control how local repos are staged before scanning (mitigating CVE-2025-41390). For the GitHub scanner, the [README](README.md) shows `--repo`, `--org`, `--issue-comments`, and `--pr-comments` flags for fan-out across issues, pull requests, and discussions. ## See Also - [README](README.md) — Quick start, subcommand reference, and CI integration recipes - [pkg/custom_detectors/CUSTOM_DETECTORS.md](pkg/custom_detectors/CUSTOM_DETECTORS.md) — Authoring regex detectors with verification - [pkg/sources/docker/README.md](pkg/sources/docker/README.md) — Docker image scanning reference - [pkg/analyzer/README.md](pkg/analyzer/README.md) — Permission analyzer framework (`trufflehog analyze`) - [examples/README.md](examples/README.md) — Generic detector YAML and example scripts - [hack/checksecretparts/README.md](hack/checksecretparts/README.md) — Lint guard for detector `SecretParts` population --- ## Extensibility and Custom Detectors ### Related Pages Related topics: [Detectors Library](#page-4), [Output Formats, Configuration, and Filtering](#page-6)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [examples/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) - [pkg/custom_detectors/CUSTOM_DETECTORS.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md) - [proto/custom_detectors.proto](https://github.com/trufflesecurity/trufflehog/blob/main/proto/custom_detectors.proto) - [pkg/custom_detectors/custom_detectors.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/custom_detectors.go) - [pkg/custom_detectors/validation.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/validation.go) - [pkg/custom_detectors/regex_varstring.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/regex_varstring.go) - [pkg/pb/custom_detectorspb/custom_detectors.pb.go](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/pb/custom_detectorspb/custom_detectors.pb.go) - [pkg/analyzer/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md) - [hack/checksecretparts/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md)
# Extensibility and Custom Detectors ## Overview and Scope TruffleHog ships with a large library of native detectors — the project advertises "over 700 credential detectors that support active verification against their respective APIs" — but the team explicitly recognises that not every secret type a user cares about can ship in-tree. To address this gap, TruffleHog exposes a **custom detectors** subsystem that lets operators describe new secret patterns declaratively, without recompiling the binary. As the README states, custom detectors are an "alpha" feature that is "subject to change," so consumers should pin versions and review changelogs before upgrading. The extensibility surface is intentionally narrow and is described by a small protobuf schema: - [`proto/custom_detectors.proto`](https://github.com/trufflesecurity/trufflehog/blob/main/proto/custom_detectors.proto) defines the wire-level types (regex patterns, keywords, verification endpoints, product/decoder hints). - [`pkg/custom_detectors/custom_detectors.go`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/custom_detectors.go) implements the runtime that loads user-supplied YAML, evaluates it against chunk data, and emits standard `detectors.Result` records. - [`pkg/custom_detectors/validation.go`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/validation.go) validates user-supplied YAML before it is loaded. - [`pkg/custom_detectors/regex_varstring.go`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/regex_varstring.go) implements the named capture-group / variable-string machinery used inside regex literals. - [`pkg/pb/custom_detectorspb/custom_detectors.pb.go`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/pb/custom_detectorspb/custom_detectors.pb.go) is the generated Go bindings consumed throughout the engine. Configuration is passed at scan time through the `--config` flag, as documented in [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) and [`examples/README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md). ## Custom Detector Configuration A custom detector is a YAML document that maps regex matches to detector metadata. The canonical example in [`examples/README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) demonstrates a **generic detector** — a pattern without a known vendor: ```yaml # examples/generic.yml detectors: - name: generic-api-key keywords: - apikey regex: hogID: '\b((A|a)PIKEY|api|SECRET|TOKEN|KEY)[^a-zA-Z0-9]{0,3}([a-zA-Z0-9]{32,45})' ``` The same README explains why this feature exists: "By default, we do not support generic detection as it would result in lots of false positives. However, if you want to attempt detect generic secrets you can use a custom detector." Source: [`examples/README.md:5-17`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md). A custom detector is invoked with the `--config` flag on any source subcommand. The supported filtering mechanisms, summarised in [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md), are: | Filter | Purpose | |-------------------|----------------------------------------------------------------------------| | `entropy` | Reject matches whose Shannon entropy falls below the configured threshold | | `regex` | Apply a secondary regex against the entire match or a captured secret | | `exclude_words` | Drop matches that contain any listed stop-words (full match or capture) | | `keywords` | Reduce noise by requiring proximity keywords in the surrounding chunk | The README also notes that "if your custom detector has multiple `regex` set (in example `hogID`, and `hogToken`), then the filters get applied to each regex," and points to [`examples/generic_with_filters.yml`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/generic_with_filters.yml) as a worked example. Source: [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md). ## Verification via Webhook Because TruffleHog's value proposition is *verified* secrets rather than raw regex hits, custom detectors can be paired with an external HTTP verification endpoint. The flow is described in the README: ```mermaid sequenceDiagram participant TH as TruffleHog Engine participant CDN as Custom Detector participant EP as User Verification Endpoint TH->>CDN: Scan chunk with custom regex CDN-->>TH: Match + captured secret TH->>EP: POST JSON {match, captured groups} EP-->>TH: 200 OK → verified EP-->>TH: non-2xx → unverified TH->>TH: Emit detectors.Result{Verified=...} ``` "TruffleHog will send a JSON POST request containing the regex matches to a configured webhook endpoint. If the endpoint responds with a `200 OK` response status code, the secret is considered verified. If verification fails due to network/API errors, the result is marked as unknown." Source: [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md). Verification endpoints are configured at the CLI with `--verifier=...` and may be enforced exclusively via `--custom-verifiers-only`. Source: [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md). Setup documentation for the user-side endpoint lives at [`pkg/custom_detectors/CUSTOM_DETECTORS.md`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md), which the README links as "how to setup a custom regex detector with verification server." Source: [`README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/README.md). ## Generic Detection and the Analyzer Subsystem The generic detector example in [`examples/README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md) is invoked as: ```bash trufflehog filesystem --config=$PWD/generic.yml $PWD ``` To isolate only generic hits in JSON output, the README recommends: ```bash trufflehog filesystem --config=$PWD/generic.yml --json --no-verification $PWD \ | awk '/generic-api-key/{print $0}' ``` Source: [`examples/README.md:9-16`](https://github.com/trufflesecurity/trufflehog/blob/main/examples/README.md). Beyond regex-driven discovery, TruffleHog also offers an **analyze** command that performs deeper post-verification analysis of a credential. Permissions for analyzers are declared in YAML and generated into Go types, as described in [`pkg/analyzer/README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md). "The Permissions are initially defined as a yaml file (analyzers/twilio/permissions.yaml). At the top of the analyzer implementation you specify the go generate command." Source: [`pkg/analyzer/README.md:3-9`](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/analyzer/README.md). ## Common Failure Modes and Limits - **Alpha status.** The README labels the entire feature "alpha and subject to change," so YAML schemas may evolve between releases. - **SecretParts population.** A static analysis lint at [`hack/checksecretparts/README.md`](https://github.com/trufflesecurity/trufflehog/blob/main/hack/checksecretparts/README.md) warns when native detector packages construct `detectors.Result` without populating the `SecretParts` field. Custom detectors that bypass native conventions should be reviewed manually to keep verification stable. - **False positives.** Generic patterns deliberately trade recall for noise; the `entropy`, `regex`, and `exclude_words` filters exist to claw back precision. - **Library use.** The README warns that "trufflehog is in heavy development and no guarantees can be made on the stability of the public APIs," which applies to anyone embedding the custom-detector package directly. ## See Also - [Detection Engine and Architecture](#) - [Output Formats and CI Integration (SARIF, JSON)](#) — relates to community request for SARIF 2.1.0 support. - [Sources Overview: GitHub, GitLab, S3, Docker](#) — `--config` is accepted by every source subcommand. - [Adding New Native Detectors](https://github.com/trufflesecurity/trufflehog/blob/main/hack/docs/Adding_Detectors_external.md) — the upstream guide for in-tree contributions. --- ## Deployment and CI/CD Integration ### Related Pages Related topics: [Overview and Quick Start](#page-1), [Output Formats, Configuration, and Filtering](#page-6)
Related Source Files The following source files were used to generate this page: - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) - [action.yml](https://github.com/trufflesecurity/trufflehog/blob/main/action.yml) - [.pre-commit-hooks.yaml](https://github.com/trufflesecurity/trufflehog/blob/main/.pre-commit-hooks.yaml) - [PreCommit.md](https://github.com/trufflesecurity/trufflehog/blob/main/PreCommit.md) - [Dockerfile](https://github.com/trufflesecurity/trufflehog/blob/main/Dockerfile) - [Dockerfile.goreleaser](https://github.com/trufflesecurity/trufflehog/blob/main/Dockerfile.goreleaser) - [.goreleaser.yml](https://github.com/trufflesecurity/trufflehog/blob/main/.goreleaser.yml) - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md)
# Deployment and CI/CD Integration TruffleHog is designed to fit into a wide variety of deployment topologies, from local developer machines to fully automated CI/CD pipelines. The project ships multiple distribution channels (Homebrew tap, static binaries, signed container images, and a first-party GitHub Action) so that the same scanner can be invoked consistently across pull-request checks, scheduled scans, and pre-commit hooks. This page documents the supported deployment surfaces, the configuration surface that CI engineers interact with, and the artifact verification workflow that protects the supply chain. ## Distribution and Container Images TruffleHog is published through several channels that are each suitable for different CI environments. The `README.md` documents four primary installation paths: a Homebrew formula (`brew install trufflehog`), Docker images published as `trufflesecurity/trufflehog`, pre-built binaries downloaded from GitHub Releases, and a one-line `install.sh` script (Source: [README.md]()). The container images are produced from `Dockerfile.goreleaser` (used for the multi-arch release images) and a slimmer `Dockerfile` used for development. The Docker image is the most common CI vector because it does not require a Go toolchain on the runner and works on both x86_64 and ARM hosts. The README illustrates the canonical invocation, which mounts the current working directory and selects a source subcommand (Source: [README.md]()): ```bash docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys ``` For Apple Silicon and Windows runners, the README also documents platform- and shell-specific mount syntax variations, ensuring the same image works on `linux/amd64` and `linux/arm64` hosts (Source: [README.md]()). The Docker source's own documentation in `pkg/sources/docker/README.md` describes the image-reference grammar accepted by the scanner — `nginx:latest` for remote registries, `docker://nginx:latest` for the local daemon, and `file:///path/to/image.tar` for tarballs — which is the same grammar CI jobs must use when passing `--image` flags (Source: [pkg/sources/docker/README.md]()). ## GitHub Actions Integration TruffleHog ships a first-party composite GitHub Action declared in `action.yml`. The README shows two common usage patterns: scanning on every push and pull request, and scanning a specific range of commits using the `base` and `head` inputs (Source: [README.md]()). The advanced "Scan entire branch" pattern demonstrates that leaving `base` empty while setting `head` to the current ref is the supported way to perform a full-branch scan (Source: [README.md]()). A typical workflow invocation looks like the following (Source: [README.md]()): ```yaml - uses: trufflesecurity/trufflehog@main with: path: base: head: # optional extra_args: --log-level=2 --results=verified,unknown ``` For performance-sensitive PR pipelines, the README documents a depth-aware checkout pattern: it computes the number of commits in the push or PR, adds two, and passes the result to `actions/checkout`'s `fetch-depth`. This avoids fetching the full history on every event while still giving TruffleHog the commits it needs to diff against `base` (Source: [README.md]()). When wiring the action into multi-stage pipelines, the additional inputs (`--issue-comments`, `--pr-comments`) can be forwarded through `extra_args` to expand coverage to GitHub Issues and Pull Requests bodies (Source: [README.md]()). The `extra_args` string is the canonical escape hatch for any CLI flag, which means the GitHub Action inherits the full TruffleHog CLI surface — including the JSON output mode required by security dashboards and the SARIF output format requested by community issue #578 ("Sarif 2.1.0 output support for CI security") (Source: [README.md]()). ## Other CI Integrations and Pre-commit Hooks Beyond GitHub Actions, the README documents a drop-in GitLab CI configuration. The example installs TruffleHog via the `install.sh` script inside an Alpine job, then invokes the `filesystem` subcommand against a configurable `SCAN_PATH` (Source: [README.md]()). The same approach generalizes to any container-based CI: install the binary, authenticate against the target source (Git, GitHub, S3, GCS, etc.), and invoke the appropriate subcommand. For pre-commit workflows, TruffleHog publishes a hook definition in `.pre-commit-hooks.yaml` and supporting documentation in `PreCommit.md`. The hook lets developers scan staged changes locally before they ever reach a CI runner, which complements — rather than replaces — server-side scanning. Pre-commit integration is documented alongside the action in the README, reinforcing the project's "shift left" strategy for secret detection (Source: [README.md]()). The README also highlights the `--shallow-since` and `--since-commit` flags requested in community issue #628, which are the supported way to bound git history scans in CI when a full clone is impractical (Source: [README.md]()). ## Artifact Verification and Supply Chain Security Release artifacts — binaries, checksums, and container images — are signed with Cosign. The README documents the verification workflow in three steps: download the artifact, the `trufflehog_{version}_checksums.txt` file, its `.pem` certificate, and its `.sig` signature; run `cosign verify-blob` with the `certificate-identity-regexp` and `--certificate-oidc-issuer` flags; and finally validate SHA-256 sums with `sha256sum --ignore-missing -c` (Source: [README.md]()). CI pipelines that pull TruffleHog from the GitHub Releases API can layer the same checks into their job definitions to guarantee they execute an authentic binary. The release pipeline itself is defined in `.goreleaser.yml`, which orchestrates the cross-platform builds, the Docker image publication, and the checksum generation. `Dockerfile.goreleaser` is the build context consumed by that pipeline. Together, these files describe a reproducible supply chain: any consumer can rebuild the images, compare checksums, and validate Cosign signatures before promoting a TruffleHog version across environments (Source: [.goreleaser.yml](), [Dockerfile.goreleaser]()). For local repository scans, the README also notes that TruffleHog clones local git repos to a temporary directory before scanning, following Git's security best practices (CVE-2025-41390). CI jobs that scan untrusted checkouts should leave this default enabled and pass `--clone-path` only when a managed temp location is required (Source: [README.md]()). ## See Also - [README.md](https://github.com/trufflesecurity/trufflehog/blob/main/README.md) — Top-level usage, installation, and CI recipes - [action.yml](https://github.com/trufflesecurity/trufflehog/blob/main/action.yml) — GitHub Action input/output contract - [PreCommit.md](https://github.com/trufflesecurity/trufflehog/blob/main/PreCommit.md) — Pre-commit hook setup - [.goreleaser.yml](https://github.com/trufflesecurity/trufflehog/blob/main/.goreleaser.yml) — Release and signing pipeline definition - [pkg/sources/docker/README.md](https://github.com/trufflesecurity/trufflehog/blob/main/pkg/sources/docker/README.md) — Docker source image-reference grammar --- --- ## Pitfall Log Project: trufflesecurity/trufflehog Summary: Found 10 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification. ## 1. Installation risk - Installation risk requires verification - Severity: medium - Evidence strength: runtime_trace - Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Repro command: `docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github` - Evidence: identity.distribution | https://github.com/trufflesecurity/trufflehog ## 2. Configuration risk - Configuration risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/5039 ## 3. Capability evidence risk - Capability evidence risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: README/documentation is current enough for a first validation pass. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: capability.assumptions | https://github.com/trufflesecurity/trufflehog ## 4. Maintenance risk - Maintenance risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog ## 5. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: downstream_validation.risk_items | https://github.com/trufflesecurity/trufflehog ## 6. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: risks.scoring_risks | https://github.com/trufflesecurity/trufflehog ## 7. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/1969 ## 8. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/5048 ## 9. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: issue_or_pr_quality=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog ## 10. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: release_recency=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog