Related Pages

Related topics: System Architecture and Data Flow, Deployment and CI/CD Integration

Overview and Quick Start

Purpose and Scope

TruffleHog is an open-source secrets Discovery, Classification, Validation, and Analysis tool. In this context, a "secret" refers to a credential a machine uses to authenticate itself to another machine — including API keys, database passwords, and private encryption keys. Source: README.md

The project is organized around four capabilities:

• Discovery — Locates secrets across Git repositories, chats, wikis, logs, API testing platforms, object stores, filesystems, and more.
• Classification — Identifies over 800 secret types and maps them to the specific identity they belong to (e.g., AWS, Stripe, Cloudflare, Postgres).
• Validation — Performs active verification of credentials against their respective APIs to eliminate false positives. Results are reported as verified, unverified, or unknown.
• Analysis — Performs a deeper inspection of a credential to enumerate its permissions and the resources it has access to.

TruffleHog v3 is a complete Go rewrite with 700+ credential detectors that support active verification, native scanning sources (GitHub, GitLab, Docker, filesystems, S3, GCS, CircleCI, TravisCI), binary and document scanning, and integrations as a GitHub Action and pre-commit hook. Source: README.md

The tool is released under the AGPL 3 license since v3.0, with v2 still available under GPL 2.0 in the repository history. Source: README.md

High-Level Architecture

TruffleHog follows a source → detector → output pipeline. Sources enumerate data (e.g., git commits, GitHub issues, S3 objects, Docker layers); detectors scan that data for credential patterns and optionally verify them; results are emitted in human-readable, JSON, or SARIF formats. Source: README.md

flowchart LR
    A[Data Sources<br/>git, github, s3, gcs,<br/>docker, filesystem,<br/>gitlab, circleci] --> B[Engine<br/>pkg/engine]
    B --> C[Detectors<br/>800+ types]
    C --> D{Verification}
    D -->|API call| E[verified]
    D -->|no check| F[unverified]
    D -->|error| G[unknown]
    E --> H[Output<br/>stdout / JSON / SARIF]
    F --> H
    G --> H
    H --> I[Optional:<br/>analyze / webhook]

The Docker source, documented separately, demonstrates the per-source pattern: it supports remote registries, the local Docker daemon, and tarball files, and walks each image layer plus build history to find embedded secrets. Source: pkg/sources/docker/README.md

Quick Start

Installation

TruffleHog is distributed as a single static binary, a Docker image, a Homebrew package, a GitHub Action, and a pre-commit hook. Artifacts are signed with Cosign and shipped with SHA256 checksums; users can verify signatures against the trufflehog_{version}_checksums.txt.pem and .sig files downloaded from the GitHub Releases page. Source: README.md

Common Commands

The CLI exposes one subcommand per source. The most frequently used entry points are: git, github, github-experimental, gitlab, filesystem, s3, gcs, syslog, circleci, docker, huggingface, stdin, and analyze. Source: README.md

Verification Statuses

Every detected credential is reported as one of three statuses:

• verified — the credential was confirmed valid and active by an API call (e.g., the AWS detector calls GetCallerIdentity).
• unverified — the credential matched a pattern but was not confirmed valid.
• unknown — verification was attempted but failed (network or API error).

Source: README.md

Extending TruffleHog

Custom Detectors

Generic secret detection is disabled by default to suppress false positives, but users can opt in via a custom detector configuration. A community example lives at examples/generic.yml and can be invoked as trufflehog filesystem --config=$PWD/generic.yml $PWD. Custom detectors support regex, entropy, captured-group filtering, and excluded word lists, and may optionally POST matches to a webhook for verification — a 200 OK response marks the result as verified. Source: README.md, examples/README.md

Adding New Detectors

The project publishes external documentation and tooling for adding new credential detectors. Source: README.md

A repository-level static-analysis tool, hack/checksecretparts, ensures every detector populates the SecretParts field on its detectors.Result struct. It runs in warning mode by default and can be flipped to fail mode once all detectors are migrated. Source: hack/checksecretparts/README.md

Analyzer Framework

For deep credential inspection, analyzers enumerate the permissions a credential has on its target system. Permissions are declared in a YAML file and translated into Go types via go generate, using the generator at pkg/analyzer/generate_permissions. Source: pkg/analyzer/README.md

Common Failure Modes and Community Requests

Several long-standing community requests shape day-to-day use:

• SARIF 2.1.0 output for CI security dashboards (Issue #578).
• .trufflehogignore file to suppress specific findings by fingerprint (Issue #2687).
• Recursive filesystem scans outside of git repositories (Issue #92) — supported via the filesystem subcommand.
• New detectors (e.g., Adobe IMS OAuth2 tokens, Issue #4908) — contributed via the documented detector authoring flow.
• Shallow-since filtering for git and GitHub scans (Issue #628).

The github-experimental subcommand additionally enumerates Cross Fork Object References and deleted commits via the --object-discovery flag, writing intermediate state to valid_hidden.txt and invalid.txt under $HOME/.trufflehog (cleared with --delete-cached-data). Source: README.md

For secrets that should be ignored inline, a trufflehog:ignore comment on the line containing the match suppresses the finding. Source: README.md

See Also

• Docker Source — detailed configuration for the docker subcommand.
• Custom Detectors — author and tune user-defined detectors.
• Detector Authoring — contribute new built-in detectors.
• Analyzer Framework — deep credential permission analysis.

Related Pages

Related topics: Data Sources, Detectors Library, Verification and Permission Analysis

System Architecture and Data Flow

Overview and Purpose

TruffleHog is a secrets Discovery, Classification, Validation, and Analysis tool. In this context, "secret" refers to a credential a machine uses to authenticate itself to another machine — including API keys, database passwords, private encryption keys, and similar material. The tool is implemented as a Go binary that exposes a CLI (trufflehog) and a library surface used by the TruffleHog Enterprise product Source: [README.md:1-40].

Architecturally, TruffleHog is organized as a pipeline:

1. Sources produce chunks of content (files, git blobs, API responses, image layers, etc.).
2. Decoders normalize and extract secrets from non-plain formats (base64, gzip, etc.).
3. Detectors run regex and entropy heuristics against decoded chunks, classified across 800+ credential types.
4. Verification performs live API checks to confirm whether a candidate is actually valid.
5. Analyzers (optional, via trufflehog analyze) inspect a verified credential to enumerate its permissions and reachable resources.

This separation lets new sources, decoders, and detectors be added without modifying the core engine, and it allows users to plug in Custom Detectors defined in YAML Source: [examples/README.md:1-15].

High-Level Architecture

flowchart LR
    CLI[CLI / multi-scan]
    subgraph Sources
      GIT[Git]
      GH[GitHub]
      GL[GitLab]
      FS[Filesystem]
      S3[S3 / GCS]
      DOCKER[Docker]
      CI[CircleCI / TravisCI]
      HF[Hugging Face]
    end
    subgraph Engine[pkg/engine]
      DEC[Decoders]
      DET[Detectors<br/>~800 types]
      VER[Verifiers<br/>API calls]
      RES[Results Channel]
    end
    AN[Analyzer<br/>trufflehog analyze]
    OUT[Output: pretty / JSON / SARIF* / webhook]

    CLI --> Sources
    Sources --> DEC
    DEC --> DET
    DET --> VER
    VER --> RES
    RES --> OUT
    RES --> AN

* SARIF output is a commonly requested feature, tracked in community issue #578 Source: [README.md:130-145].

Sources Layer

Each source type implements a common Source interface that yields Chunk values containing content plus metadata (commit SHA, file path, line number, email, timestamp, repository URL). Examples include:

• git / github / gitlab — repository histories, pull-request comments, issue comments, and Cross-Fork Object References via github-experimental --object-discovery Source: [README.md:175-195]).
• filesystem — recursively scans directories and individual files (issue #92 requested non-recursive directory support, but the filesystem subcommand now covers that use case) Source: [README.md:215-220]).
• s3 / gcs — object stores, optionally with IAM role assumption for cross-account scanning Source: [README.md:255-275]).
• docker — remote registries, the local Docker daemon, or tarball files; scans each image layer independently and also inspects build-history metadata for secrets embedded in RUN commands Source: [pkg/sources/docker/README.md:1-30]).
• circleci, travisci, huggingface, elasticsearch, syslog, stdin — additional integrations.

Decoders and Detectors Layer

Decoders (pkg/decoders) peel encodings — base64, URL-encoding, gzip, JSON-in-string, and others — producing plaintext fragments. Detectors (pkg/detectors/<name>) apply regex patterns and entropy filters to identify candidate credentials. The detector registry is keyed by detector ID/type (for example DetectorType: 2, DetectorName: "AWS" in the JSON output) Source: [README.md:95-110]).

A static-analysis tool, hack/checksecretparts, enforces that every detectors.Result{...} literal populates the new SecretParts field, so newly added detectors cannot silently regress structured output Source: [hack/checksecretparts/README.md:1-35]).

Engine and Verification Layer

The engine (pkg/engine/engine.go) coordinates chunk production, decoding, detection, and verification. For every candidate, the engine calls the detector's verifier, which performs an API call against the service the credential claims to belong to (for example, AWS's GetCallerIdentity for AWS access keys) Source: [README.md:115-125]).

Three terminal result states are defined:

Verification results can be filtered via --results=verified, --results=verified,unknown, etc. Source: [README.md:125-140].

Data Flow

A typical scan proceeds as follows:

1. The user invokes a subcommand (for example trufflehog git https://github.com/.../repo --results=verified). The CLI parses flags, constructs a Source, and registers detectors from the default set plus any user-provided custom detectors Source: [examples/README.md:5-15]).
2. The Source enumerates units of work (commits, files, image layers, S3 objects) and emits Chunks. For local git scans, the repository is cloned into a temporary directory first to neutralize malicious .git/config hooks (mitigation for CVE-2025-41390); --clone-path overrides this location, and --trust-local-git-config re-enables in-place scanning for trusted repos Source: [README.md:215-225]).
3. Each chunk flows through the decoder pipeline. Decoded fragments re-enter the detector stage, allowing multi-layer encoding chains to be peeled until plaintext is reached.
4. Detectors emit detectors.Result values with raw/redacted strings, detector type, decoder used, and source metadata. The engine forwards results through a Go channel.
5. Verifiers act on results that pass preliminary filters (entropy thresholds, exclusion word lists). The outcome (verified / unverified / unknown) is attached to the result Source: [README.md:115-130]).
6. The output writer renders results as pretty text by default; --json produces structured JSON, and a webhook can be configured for CI pipelines. Lines containing findings can be suppressed by adding a trufflehog:ignore comment on the offending line in the source code Source: [README.md:155-170]).
7. Optionally, the user can run trufflehog analyze to take a verified credential and enumerate its blast radius — permissions, accessible resources, and so on. Analyzer packages define their permissions in YAML, generated into Go via go generate ./... Source: [pkg/analyzer/README.md:1-15]).

Key Components and Extensibility

Community requests cluster around three extension points:

• Output formats: SARIF 2.1.0 (#578) for security tooling integration.
• Ignore lists: a .trufflehogignore file keyed by fingerprint (#2687).
• New detectors: e.g. Adobe IMS OAuth2 tokens (#4908); the project publishes Adding_Detectors_external.md to onboard contributors.

The architecture's reliance on small, well-defined interfaces at each layer keeps each of these additions local — new sources, new decoders, new detectors, and new output sinks can all be added without touching the others.

See Also

• Docker Source — registry/daemon/tarball scanning details
• Custom Detectors — YAML-defined detectors and webhook verification
• Adding New Detectors — contribution guide for new credential types
• Analyzer Framework — permissions and resource enumeration

Related Pages

Related topics: System Architecture and Data Flow, Output Formats, Configuration, and Filtering

Data Sources

Purpose and Scope

TruffleHog is a secrets Discovery, Classification, Validation, and Analysis tool. Data Sources are the input adapters that define *where* TruffleHog should look for credentials. A data source ingests content from a particular kind of storage or service (e.g., a Git repository, an S3 bucket, a Docker image, or a local filesystem) and emits a stream of *chunks* that are then handed to the detector engine for regex matching and verification (Source: README.md).

The source layer is intentionally separated from the detector layer: detectors know how to recognize and verify a credential, while sources know how to traverse a system and produce a sequence of byte chunks. This separation is what allows TruffleHog to support many disparate back-ends without duplicating detection logic (Source: pkg/engine/engine.go).

Supported Source Subcommands

Each data source is exposed as a top-level CLI subcommand. The README documents the following sources out of the box: git, github, github-experimental, gitlab, filesystem, s3, gcs, syslog, circleci, docker, travisci, elasticsearch, huggingface, stdin, analyze, and multi-scan (Source: README.md). The full subcommand listing appears in the CLI help block reproduced in the same file.

Sources: README.md, pkg/sources/docker/README.md.

Architecture and Data Flow

flowchart LR
    A[Data Source<br/>git, github, s3, docker, ...] -->|emits byte chunks| B[Engine<br/>pkg/engine/engine.go]
    B -->|runs regex detectors| C[Detector Pool<br/>800+ types]
    C -->|HTTP/API verification| D[Verifier]
    D -->|verified / unverified / unknown| E[Output<br/>JSON, SARIF, plaintext]
    E -->|optional deeper analysis| F[Analyzer<br/>pkg/analyzer]

Each source implements the same minimal contract: enumerate the data it can see, yield chunks of bytes, and report metadata (commit, file, line, timestamp, repository, etc.). The engine then fans chunks out to detectors concurrently, deduplicates results, and forwards findings to the configured verifier or output sink. The chunk–detector boundary is what enables TruffleHog to scan containers and object stores with the same detector library used for git history (Source: pkg/engine/engine.go).

The Docker source is illustrative: it supports remote registries, the local daemon, and tarballs, and scans layers plus build history concurrently with authentication and namespace enumeration. Files larger than 50 MB are skipped automatically to keep scans bounded (Source: pkg/sources/docker/README.md).

Configuration, Filtering, and Multi-Scan

Because no two environments are identical, TruffleHog exposes filters that apply across all sources. The most important global flags are:

• --include-detectors / --exclude-detectors — accept protobuf names, numeric IDs, or ID ranges, allowing operators to scope a run to a small subset (e.g., only cloud providers) (Source: README.md).
• --results=verified,unknown,unverified — control which finding severities are emitted.
• --no-verification-cache, --force-skip-binaries, --force-skip-archives, --skip-additional-refs.
• --detector-timeout, --archive-max-size, --archive-max-depth, --archive-timeout — bound resource consumption.
• --config — points at a YAML file consumed by the multi-scan subcommand, where multiple source connections (e.g., several GitHub repos and S3 buckets) can be enumerated and scanned concurrently. The schema for each connection lives in Truffle Security's source configuration documentation (Source: README.md).

For local git scanning, the README documents that TruffleHog now clones to a temporary directory before scanning to mitigate CVE-2025-41390 from malicious .git/config payloads; users can override the temp path with --clone-path or skip the clone step with --trust-local-git-config for trusted repos (Source: README.md).

The community has repeatedly asked for incremental scans on git history. Issue #628 requests --shallow-since for git/GitHub scans, which would let users bound a run to commits after a timestamp. Related ergonomics requests include #2687 (a .trufflehogignore file of fingerprint-based exclusions) and #92 (recursive filesystem scans outside git, which is now satisfied by trufflehog filesystem) (Source: README.md).

Common Failure Modes and Limitations

• Large files or unusual encodings. Lines exceeding bufio's default 64 KB token limit were historically skipped; this was fixed in v3.95.6 (PR #5022).
• Hidden or deleted commits on GitHub. github-experimental --object-discovery enumerates Cross Fork Object References but takes 20 minutes to several hours per repository and writes state files under $HOME/.trufflehog (Source: README.md).
• Docker image size. Files larger than 50 MB are skipped, which can hide long-lived secrets in large blobs (Source: pkg/sources/docker/README.md).
• Detector coverage gaps. New credential types (e.g., Adobe IMS OAuth2 tokens, requested in #4908) are added incrementally through the documented detector authoring guide at hack/docs/Adding_Detectors_external.md (Source: README.md).
• Output format evolution. Issue #578 requests SARIF 2.1.0 output for CI security tooling integration; consumers that need SARIF today should check the release notes for current support.

See Also

• Detector engine internals: pkg/engine/engine.go
• Docker source details: pkg/sources/docker/README.md
• Permission analyzers: pkg/analyzer/README.md
• Custom detector examples: examples/README.md
• Detector linting: hack/checksecretparts/README.md

Related Pages

Related topics: System Architecture and Data Flow, Verification and Permission Analysis, Extensibility and Custom Detectors

Detectors Library

Overview and Purpose

The Detectors Library is the heart of TruffleHog's credential discovery and classification capabilities. It is a large collection of individual Go packages — one per credential type — that recognize, extract, and optionally verify secrets such as API keys, OAuth tokens, database passwords, and private keys. According to the project README, TruffleHog ships with over 800 detectors, each mapping a discovered token to a specific provider identity (AWS, Stripe, Cloudflare, Postgres, etc.) and, when possible, validating the credential against the corresponding API. Source: README.md:73-83.

Each detector follows a shared Go interface defined in pkg/detectors/detectors.go. The engine feeds every scanned chunk of content (a file line, a git diff hunk, a Docker layer excerpt) to every registered detector, then aggregates matches into a normalized Result struct consumed by all output sinks (console, JSON, SARIF, etc.). Source: pkg/engine/engine.go:358-365.

Core Data Model

A detector produces a detectors.Result value containing the fields shown below. These fields are uniform across all ~800+ built-in detectors and the user-defined custom detectors, which makes downstream consumers (analyzers, loggers, CI integrations) trivial to write.

A recently introduced SecretParts field is required by the static-analysis check in hack/checksecretparts/README.md; the linter warns when a detectors.Result{} literal is constructed without populating SecretParts, and is intended to flip into a gating check once every detector is migrated.

Verification Pipeline

Verification is what separates TruffleHog from a pure regex scanner. For every candidate match, the engine dispatches an asynchronous verification routine whose implementation lives next to each detector. The AWS detector, for instance, calls GetCallerIdentity against the AWS API to confirm the key is live and to enrich the finding with the owning account, arn, and user_id (visible in the example JSON in the README). Source: pkg/detectors/aws/aws.go.

Three result statuses can be returned:

• verified — credential confirmed live by the upstream API
• unverified — regex matched but verification was disabled (--no-verification) or the secret was invalidated before the call
• unknown — verification failed because of a transient error (network, rate-limit, 5xx)

Shared helpers live alongside the detectors package:

• account_filter.go — allow/deny accounts at the engine level (useful in CI to ignore dev keys)
• falsepositives.go — a list of well-known test fixtures (canarytokens, example.com, etc.) that should never count as live
• endpoint_customizer.go — override the verification URL (e.g., to point at a staging or private-cloud deployment of the provider)
• multi_part_credential_provider.go — for credentials that need two or more matched fragments (username + token, id + secret)

These helpers let detector authors avoid duplicating cross-cutting concerns and keep individual detector files focused on the provider's quirks. Source: pkg/detectors/detectors.go.

Custom and Generic Detectors

TruffleHog intentionally does not run "generic API key" detection by default because the false-positive rate is too high. Two opt-in paths exist for users who need broader coverage:

``bash trufflehog filesystem --config=$PWD/generic.yml $PWD `` The generic detector is implemented in pkg/detectors/generic/generic.go. Source: examples/README.md:5-15.

1. Generic detector — enabled through examples/generic.yml. The README snippet shows the one-line invocation:

1. User-defined YAML custom detectors — documented in pkg/custom_detectors/CUSTOM_DETECTORS.md. They support per-regex filters for entropy, targeted regex (the entire match), capture-group regex (the secret portion), and excluded word lists. When configured with a --verifier webhook URL, TruffleHog POSTs the match to the endpoint and treats a 200 OK as verified. The same docs warn that this feature is alpha and subject to change. Source: README.md:305-321.

flowchart LR
    A[Scanned Chunk] --> B[Engine]
    B --> C{Detector Pool<br/>~800 detectors}
    C -->|regex match| D[detectors.Result]
    D --> E[account_filter]
    E --> F[falsepositives]
    F --> G[endpoint_customizer]
    G --> H{Verifier API<br/>or webhook}
    H -->|verified| I[verified]
    H -->|invalid| J[unverified]
    H -->|error| K[unknown]
    I --> L[Output Sink<br/>JSON / SARIF / Console]

Community-Driven Detector Coverage

The detector roster is community-extensible. Recent release notes (v3.95.6) illustrate the cadence of detector maintenance: bug fixes such as honoring ignore tags for default-port Postgres URLs and hardening the Enigma detector against false-verified findings. Source: release notes referenced in community context.

Open feature requests that map directly onto the detectors library include:

• #4908 — Adobe IMS OAuth2 detector. Proposes a new package under pkg/detectors/adobeims/ following the same pattern as the existing AWS detector, including a JWT-based verification call to the Adobe IMS endpoint. Source: community context issue #4908.
• #2687 — .trufflehogignore file. Requests a fingerprint-based suppression list so teams can mute recurring findings without modifying source. This would integrate with falsepositives.go or a new sibling helper. Source: community context issue #2687.

Operational Tips and Failure Modes

• A detector that fails to populate SecretParts is flagged by hack/checksecretparts and may break downstream consumers expecting token fragments. Source: hack/checksecretparts/README.md:5-12.
• Bufio's default 64 KB token limit can clip extremely long lines; v3.95.6 raised the scanner buffer so single-line payloads (PEM blocks, JWTs, large cookies) are tokenized correctly. Source: community context v3.95.6 changelog.
• Use --include-detectors / --exclude-detectors (accepting protobuf names, IDs, or ID ranges) to scope a scan to a subset of the library. Source: README.md CLI reference.
• Add a trufflehog:ignore comment on the offending line to suppress a finding in the local repo context — engine logic is at pkg/engine/engine.go:358-365.

See Also

• Docker Source
• Analyzers
• Adding Detectors (external guide)
• checksecretparts linter

Related Pages

Related topics: Detectors Library, Output Formats, Configuration, and Filtering

Verification and Permission Analysis

Overview

TruffleHog is positioned as a complete Discovery, Classification, Validation, and Analysis tool for machine credentials. Two of those phases live in dedicated subsystems that run after the high-throughput Aho-Corasick detector engine flags candidate matches. The Aho-Corasick core performs keyword/prefix pre-filtering so that only promising chunks reach per-detector verification logic, while the engine itself manages the full lifecycle from source ingestion to result emission.

• Validation — verifying that a discovered credential is actually live. This is implemented per-detector (for example, a GetCallerIdentity API call for AWS) and is wrapped by a verification result cache to avoid repeat API calls during long scans.
• Analysis — once a credential is confirmed valid, the pkg/analyzer subsystem inspects it for the permissions and resources it can reach.

Source: README.md describes TruffleHog's four-phase mission (Discovery, Classification, Validation, Analysis) and the verified/unverified/unknown result statuses emitted by the validation pipeline. Source: pkg/engine/ahocorasick/ahocorasickcore.go powers the keyword pre-filter that feeds detectors.

Credential Verification

Result Statuses

Every detector emits a detectors.Result whose verification step yields one of three states:

• verified — the credential was confirmed valid and active by API testing.
• unverified — the credential was detected but could not be confirmed valid (it may be invalid, expired, or verification may have been disabled for that detector).
• unknown — verification was attempted but failed because of a network or API error.

Source: README.md documents these three result statuses and gives the AWS detector as the canonical example. Source: pkg/detectors/aws/aws.go performs that verification via GetCallerIdentity.

Caching

Verification calls are expensive, so TruffleHog caches their outcomes in two cooperating layers:

• pkg/verificationcache/verification_cache.go deduplicates verification calls — the same credential value passed through the same detector will not trigger two API calls during a single scan.
• pkg/verificationcache/result_cache.go stores the structured Result payload so downstream consumers can re-emit findings without re-scanning.
• A CLI flag --no-verification-cache disables this caching layer when needed (for example, in CI jobs that always want a fresh check).

Source: pkg/verificationcache/verification_cache.go implements the verification-call cache. Source: pkg/verificationcache/result_cache.go implements the result cache. Source: README.md lists the --no-verification-cache flag.

Custom Detectors with Verification

Custom regex detectors can opt into HTTP-based verification: TruffleHog POSTs the matched secret as JSON to a configured webhook endpoint; a 200 OK response marks the credential as verified, and any network or API failure flips it to unknown. Filters available on custom detectors include entropy, full-match regex, capture-group regex, and excluded word lists. Note that if a custom detector declares multiple regexes (for example hogID and hogToken), the filters are applied to each one.

Source: pkg/custom_detectors/CUSTOM_DETECTORS.md describes the verification webhook protocol, and examples/generic_with_filters.yml is referenced as the canonical filter example. Source: README.md notes that custom detectors are an alpha feature.

Special Verification Modes

TruffleHog ships two notable verification paths that go beyond per-detector API checks. First, Driftwood technology instantly verifies private keys against millions of GitHub users and billions of TLS certificates. Second, generic JWT detection verifies a subset of JWTs that use public-key cryptography, retrieving the public key to confirm whether the token is live.

Source: README.md introduces both Driftwood and the generic JWT verification flow.

Permission Analysis

Once a credential is verified, trufflehog analyze runs the analyzer subsystem to enumerate the permissions and accessible resources attached to the identity.

flowchart LR
    A[Verified credential] --> B[analyzer.Spawn]
    B --> C[Analyzer implementation]
    C --> D[Permissions YAML]
    D --> E[go generate]
    E --> F[Typed Permissions]
    F --> G[AnalyzerResult]
    G --> H[analyze CLI output]

Analyzer Architecture

The analyzer package is registered by pkg/analyzer/analyzers.go, which provides the dispatch layer that selects a concrete analyzer implementation for a given credential type. Each analyzer (for example analyzers/twilio/twilio.go) consumes a credential, calls the relevant vendor API, and returns an AnalyzerResult describing the identity, its permissions, and the resources it can reach.

Source: pkg/analyzer/analyzers.go dispatches to analyzer implementations. Source: pkg/analyzer/README.md references analyzers/twilio/twilio.go as a concrete analyzer and analyzers/twilio/permissions.yaml as its permission declaration.

Permission Definitions

Permissions are declared in YAML and code-generated into typed Go constants:

• YAML format: a permissions.yaml file under each analyzer directory.
• Accepted naming styles: lower snake case (permission_name:access_level), kebab case (permission-name:read), or dot notation (permission.name:read).
• Codegen: install the generator with go install github.com/trufflesecurity/trufflehog/v3/pkg/analyzer/generate_permissions, then run go generate ./... to produce the typed Permission values used inside AnalyzerResult.Permissions.

Source: pkg/analyzer/README.md documents the permission format, accepted naming styles, and the codegen workflow.

Analyzer CLI and Configuration

The CLI entry point in pkg/analyzer/cli.go is what trufflehog analyze actually invokes. Shared configuration for all analyzers lives in pkg/analyzer/config/config.go, which controls endpoints, timeouts, and feature flags applied to every analyzer at runtime. Together with analyzers.go, these files form the public face of the analysis subsystem.

Source: pkg/analyzer/cli.go implements the analyze subcommand. Source: pkg/analyzer/config/config.go defines the shared analyzer configuration.

CLI Usage and Common Failure Modes

Typical commands that exercise both subsystems:

# Scan a git repo and verify (AWS credentials will hit GetCallerIdentity)
trufflehog git https://github.com/trufflesecurity/test_keys --results=verified

# Run deep permission analysis on a verified credential
trufflehog analyze

# Force a fresh verification (skip cache)
trufflehog github --org=trufflesecurity --no-verification-cache

Common failure modes to be aware of:

• Verification never completes — caused by firewalls blocking egress to vendor APIs; the result is recorded as unknown rather than verified. Source: README.md lists network failures as a trigger for the unknown state.
• Custom-detector webhook misconfiguration — if the verification endpoint does not return 200 OK or is unreachable, results skew toward unknown/unverified. Source: pkg/custom_detectors/CUSTOM_DETECTORS.md describes the webhook protocol.
• Stale permissions — analyzer output reflects the credential's permissions at scan time; rotated or downscoped credentials produce different results on the next run.
• Result filtering — when a secret is wrapped in a trufflehog:ignore source comment, the engine skips it before verification, which can look like a false negative in CI output. Source: pkg/engine/engine.go documents the trufflehog:ignore comment handling.

See Also

• Detector Engine and Aho-Corasick Matching
• Sources: Git, GitHub, GitLab, S3, Docker
• Custom Detectors Configuration

Related Pages

Related topics: System Architecture and Data Flow, Extensibility and Custom Detectors, Deployment and CI/CD Integration

Output Formats, Configuration, and Filtering

TruffleHog provides a flexible output, configuration, and filtering system that lets operators tailor scans for both human inspection and machine-driven CI/CD pipelines. The subsystem spans three concerns: the output renderers that serialize findings, the YAML configuration that defines data sources and detection rules, and the filtering flags that constrain which detectors run and which results are emitted. Together they form the bridge between the scanning engine and downstream consumers (SIEM tools, GitHub PRs, custom webhooks, or operator terminals).

Output Formats

TruffleHog supports several output renderers, each implemented as a separate package under pkg/output/:

To select a format, TruffleHog uses the global --json flag (toggles JSON mode) or source-specific output plugins. A representative JSON record from a verified AWS finding is reproduced in the README and includes SourceMetadata, DetectorType, DetectorName, DecoderName, Verified, Raw, Redacted, and ExtraData fields. The SARIF 2.1.0 format is a long-standing community request tracked in issue #578 for CI security use cases.

Configuration

TruffleHog loads a YAML configuration file via the global --config=CONFIG flag. The schema is defined in pkg/config/config.go and pkg/config/detectors.go, which together describe the sources, detectors, and verification sections. A trimmed GitHub source example from the README is:

sources:
- connection:
    '@type': type.googleapis.com/sources.GitHub
    repositories:
    - https://github.com/trufflesecurity/test_keys.git
    unauthenticated: {}
  name: example config scan
  type: SOURCE_TYPE_GITHUB
  verify: true

Multiple sources entries are scanned concurrently. The same configuration file may also contain user-defined Custom Detectors (regex-based). An example of a generic detector for filesystem scans is shown in examples/README.md:

wget https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/examples/generic.yml
trufflehog filesystem --config=$PWD/generic.yml $PWD

Custom Detectors may declare a verification webhook. According to the README, when a regex matches, TruffleHog POSTs the JSON record to the configured endpoint; a 200 OK response marks the result as verified, while transport or HTTP failures yield an unknown result. Filters supported on custom detectors include Shannon entropy thresholds, regex targeting the whole match, regex targeting the captured secret, and excluded word lists. The detailed syntax is in pkg/custom_detectors/CUSTOM_DETECTORS.md.

Filtering

TruffleHog exposes several layers of filtering, each implemented as a CLI flag in the engine entry point:

flowchart LR
    A[Raw scan chunks] --> B{--include-detectors<br/>--exclude-detectors}
    B -->|pass| C[Detector matching]
    C --> D{--filter-entropy}
    D -->|pass| E[Verification]
    E --> F{--results}
    F -->|filtered| G[Output renderer]

In addition to CLI filters, the engine supports in-line suppression via the trufflehog:ignore comment on a line containing a secret. This is gated on a code path inside pkg/engine/engine.go (lines 358–365). A more durable, fingerprint-based ignore mechanism is a recurring community ask, tracked in issue #2687 for a .trufflehogignore file.

Source-Specific Knobs

Several scanners extend the global flag set with source-local options. The Docker source (pkg/sources/docker/README.md) supports file://, docker://, and remote registry references, optional file exclusion patterns, a 50 MB per-file cap, and multi-image enumeration under a registry namespace. The Git source supports --shallow-since for time-bounded history, requested in issue #628, and --clone-path plus --trust-local-git-config to control how local repos are staged before scanning (mitigating CVE-2025-41390). For the GitHub scanner, the README shows --repo, --org, --issue-comments, and --pr-comments flags for fan-out across issues, pull requests, and discussions.

See Also

• README — Quick start, subcommand reference, and CI integration recipes
• pkg/custom_detectors/CUSTOM_DETECTORS.md — Authoring regex detectors with verification
• pkg/sources/docker/README.md — Docker image scanning reference
• pkg/analyzer/README.md — Permission analyzer framework (trufflehog analyze)
• examples/README.md — Generic detector YAML and example scripts
• hack/checksecretparts/README.md — Lint guard for detector SecretParts population

Related Pages

Related topics: Detectors Library, Output Formats, Configuration, and Filtering

Extensibility and Custom Detectors

Overview and Scope

TruffleHog ships with a large library of native detectors — the project advertises "over 700 credential detectors that support active verification against their respective APIs" — but the team explicitly recognises that not every secret type a user cares about can ship in-tree. To address this gap, TruffleHog exposes a custom detectors subsystem that lets operators describe new secret patterns declaratively, without recompiling the binary. As the README states, custom detectors are an "alpha" feature that is "subject to change," so consumers should pin versions and review changelogs before upgrading.

The extensibility surface is intentionally narrow and is described by a small protobuf schema:

• proto/custom_detectors.proto defines the wire-level types (regex patterns, keywords, verification endpoints, product/decoder hints).
• pkg/custom_detectors/custom_detectors.go implements the runtime that loads user-supplied YAML, evaluates it against chunk data, and emits standard detectors.Result records.
• pkg/custom_detectors/validation.go validates user-supplied YAML before it is loaded.
• pkg/custom_detectors/regex_varstring.go implements the named capture-group / variable-string machinery used inside regex literals.
• pkg/pb/custom_detectorspb/custom_detectors.pb.go is the generated Go bindings consumed throughout the engine.

Configuration is passed at scan time through the --config flag, as documented in README.md and examples/README.md.

Custom Detector Configuration

A custom detector is a YAML document that maps regex matches to detector metadata. The canonical example in examples/README.md demonstrates a generic detector — a pattern without a known vendor:

# examples/generic.yml
detectors:
  - name: generic-api-key
    keywords:
      - apikey
    regex:
      hogID: '\b((A|a)PIKEY|api|SECRET|TOKEN|KEY)[^a-zA-Z0-9]{0,3}([a-zA-Z0-9]{32,45})'

The same README explains why this feature exists: "By default, we do not support generic detection as it would result in lots of false positives. However, if you want to attempt detect generic secrets you can use a custom detector." Source: examples/README.md:5-17.

A custom detector is invoked with the --config flag on any source subcommand. The supported filtering mechanisms, summarised in README.md, are:

The README also notes that "if your custom detector has multiple regex set (in example hogID, and hogToken), then the filters get applied to each regex," and points to examples/generic_with_filters.yml as a worked example. Source: README.md.

Verification via Webhook

Because TruffleHog's value proposition is *verified* secrets rather than raw regex hits, custom detectors can be paired with an external HTTP verification endpoint. The flow is described in the README:

sequenceDiagram
    participant TH as TruffleHog Engine
    participant CDN as Custom Detector
    participant EP as User Verification Endpoint
    TH->>CDN: Scan chunk with custom regex
    CDN-->>TH: Match + captured secret
    TH->>EP: POST JSON {match, captured groups}
    EP-->>TH: 200 OK → verified
    EP-->>TH: non-2xx → unverified
    TH->>TH: Emit detectors.Result{Verified=...}

"TruffleHog will send a JSON POST request containing the regex matches to a configured webhook endpoint. If the endpoint responds with a 200 OK response status code, the secret is considered verified. If verification fails due to network/API errors, the result is marked as unknown." Source: README.md. Verification endpoints are configured at the CLI with --verifier=... and may be enforced exclusively via --custom-verifiers-only. Source: README.md.

Setup documentation for the user-side endpoint lives at pkg/custom_detectors/CUSTOM_DETECTORS.md, which the README links as "how to setup a custom regex detector with verification server." Source: README.md.

Generic Detection and the Analyzer Subsystem

The generic detector example in examples/README.md is invoked as:

trufflehog filesystem --config=$PWD/generic.yml $PWD

To isolate only generic hits in JSON output, the README recommends:

trufflehog filesystem --config=$PWD/generic.yml --json --no-verification $PWD \
  | awk '/generic-api-key/{print $0}'

Source: examples/README.md:9-16.

Beyond regex-driven discovery, TruffleHog also offers an analyze command that performs deeper post-verification analysis of a credential. Permissions for analyzers are declared in YAML and generated into Go types, as described in pkg/analyzer/README.md. "The Permissions are initially defined as a yaml file (analyzers/twilio/permissions.yaml). At the top of the analyzer implementation you specify the go generate command." Source: pkg/analyzer/README.md:3-9.

Common Failure Modes and Limits

• Alpha status. The README labels the entire feature "alpha and subject to change," so YAML schemas may evolve between releases.
• SecretParts population. A static analysis lint at hack/checksecretparts/README.md warns when native detector packages construct detectors.Result without populating the SecretParts field. Custom detectors that bypass native conventions should be reviewed manually to keep verification stable.
• False positives. Generic patterns deliberately trade recall for noise; the entropy, regex, and exclude_words filters exist to claw back precision.
• Library use. The README warns that "trufflehog is in heavy development and no guarantees can be made on the stability of the public APIs," which applies to anyone embedding the custom-detector package directly.

See Also

• Detection Engine and Architecture
• Output Formats and CI Integration (SARIF, JSON) — relates to community request for SARIF 2.1.0 support.
• Sources Overview: GitHub, GitLab, S3, Docker — --config is accepted by every source subcommand.
• Adding New Native Detectors — the upstream guide for in-tree contributions.

Related Pages

Related topics: Overview and Quick Start, Output Formats, Configuration, and Filtering

Deployment and CI/CD Integration

TruffleHog is designed to fit into a wide variety of deployment topologies, from local developer machines to fully automated CI/CD pipelines. The project ships multiple distribution channels (Homebrew tap, static binaries, signed container images, and a first-party GitHub Action) so that the same scanner can be invoked consistently across pull-request checks, scheduled scans, and pre-commit hooks. This page documents the supported deployment surfaces, the configuration surface that CI engineers interact with, and the artifact verification workflow that protects the supply chain.

Distribution and Container Images

TruffleHog is published through several channels that are each suitable for different CI environments. The README.md documents four primary installation paths: a Homebrew formula (brew install trufflehog), Docker images published as trufflesecurity/trufflehog, pre-built binaries downloaded from GitHub Releases, and a one-line install.sh script (Source: README.md). The container images are produced from Dockerfile.goreleaser (used for the multi-arch release images) and a slimmer Dockerfile used for development.

The Docker image is the most common CI vector because it does not require a Go toolchain on the runner and works on both x86_64 and ARM hosts. The README illustrates the canonical invocation, which mounts the current working directory and selects a source subcommand (Source: README.md):

docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys

For Apple Silicon and Windows runners, the README also documents platform- and shell-specific mount syntax variations, ensuring the same image works on linux/amd64 and linux/arm64 hosts (Source: README.md). The Docker source's own documentation in pkg/sources/docker/README.md describes the image-reference grammar accepted by the scanner — nginx:latest for remote registries, docker://nginx:latest for the local daemon, and file:///path/to/image.tar for tarballs — which is the same grammar CI jobs must use when passing --image flags (Source: pkg/sources/docker/README.md).

GitHub Actions Integration

TruffleHog ships a first-party composite GitHub Action declared in action.yml. The README shows two common usage patterns: scanning on every push and pull request, and scanning a specific range of commits using the base and head inputs (Source: README.md). The advanced "Scan entire branch" pattern demonstrates that leaving base empty while setting head to the current ref is the supported way to perform a full-branch scan (Source: README.md).

A typical workflow invocation looks like the following (Source: README.md):

- uses: trufflesecurity/trufflehog@main
  with:
    path:
    base:
    head: # optional
    extra_args: --log-level=2 --results=verified,unknown

For performance-sensitive PR pipelines, the README documents a depth-aware checkout pattern: it computes the number of commits in the push or PR, adds two, and passes the result to actions/checkout's fetch-depth. This avoids fetching the full history on every event while still giving TruffleHog the commits it needs to diff against base (Source: README.md). When wiring the action into multi-stage pipelines, the additional inputs (--issue-comments, --pr-comments) can be forwarded through extra_args to expand coverage to GitHub Issues and Pull Requests bodies (Source: README.md).

The extra_args string is the canonical escape hatch for any CLI flag, which means the GitHub Action inherits the full TruffleHog CLI surface — including the JSON output mode required by security dashboards and the SARIF output format requested by community issue #578 ("Sarif 2.1.0 output support for CI security") (Source: README.md).

Other CI Integrations and Pre-commit Hooks

Beyond GitHub Actions, the README documents a drop-in GitLab CI configuration. The example installs TruffleHog via the install.sh script inside an Alpine job, then invokes the filesystem subcommand against a configurable SCAN_PATH (Source: README.md). The same approach generalizes to any container-based CI: install the binary, authenticate against the target source (Git, GitHub, S3, GCS, etc.), and invoke the appropriate subcommand.

For pre-commit workflows, TruffleHog publishes a hook definition in .pre-commit-hooks.yaml and supporting documentation in PreCommit.md. The hook lets developers scan staged changes locally before they ever reach a CI runner, which complements — rather than replaces — server-side scanning. Pre-commit integration is documented alongside the action in the README, reinforcing the project's "shift left" strategy for secret detection (Source: README.md).

The README also highlights the --shallow-since and --since-commit flags requested in community issue #628, which are the supported way to bound git history scans in CI when a full clone is impractical (Source: README.md).

Artifact Verification and Supply Chain Security

Release artifacts — binaries, checksums, and container images — are signed with Cosign. The README documents the verification workflow in three steps: download the artifact, the trufflehog_{version}_checksums.txt file, its .pem certificate, and its .sig signature; run cosign verify-blob with the certificate-identity-regexp and --certificate-oidc-issuer flags; and finally validate SHA-256 sums with sha256sum --ignore-missing -c (Source: README.md). CI pipelines that pull TruffleHog from the GitHub Releases API can layer the same checks into their job definitions to guarantee they execute an authentic binary.

The release pipeline itself is defined in .goreleaser.yml, which orchestrates the cross-platform builds, the Docker image publication, and the checksum generation. Dockerfile.goreleaser is the build context consumed by that pipeline. Together, these files describe a reproducible supply chain: any consumer can rebuild the images, compare checksums, and validate Cosign signatures before promoting a TruffleHog version across environments (Source: .goreleaser.yml, Dockerfile.goreleaser).

For local repository scans, the README also notes that TruffleHog clones local git repos to a temporary directory before scanning, following Git's security best practices (CVE-2025-41390). CI jobs that scan untrusted checkouts should leave this default enabled and pass --clone-path only when a managed temp location is required (Source: README.md).

See Also

• README.md — Top-level usage, installation, and CI recipes
• action.yml — GitHub Action input/output contract
• PreCommit.md — Pre-commit hook setup
• .goreleaser.yml — Release and signing pipeline definition
• pkg/sources/docker/README.md — Docker source image-reference grammar

Doramagic Pitfall Log

Found 10 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: identity.distribution | https://github.com/trufflesecurity/trufflehog

2. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/5039

3. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/trufflesecurity/trufflehog

4. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog

5. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/trufflesecurity/trufflehog

6. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/trufflesecurity/trufflehog

7. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/1969

8. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/trufflesecurity/trufflehog/issues/5048

9. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog

10. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/trufflesecurity/trufflehog

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using trufflehog with real data or production workflows.

• Dependency Dashboard - github / github_issue
• Filesystem Directory Transversal Issue - TruffleHog Fails to Discover Se - github / github_issue
• nearly there butu wont win that easy... - github / github_issue
• v3.95.6 - github / github_release
• v3.95.5 - github / github_release
• v3.95.4 - github / github_release
• v3.95.3 - github / github_release
• v3.95.2 - github / github_release
• v3.95.1 - github / github_release
• v3.95.0 - github / github_release
• v3.94.3 - github / github_release
• v3.94.2 - github / github_release