Related Pages

Related topics: Lib

Overview

Dagu is a self-contained workflow orchestrator that runs DAGs (Directed Acyclic Graphs) defined as YAML files. It provides a single-binary Go server that combines a scheduler, an executor agent, a worker for distributed runs, and an embedded Web UI. Workflows can be triggered on schedules, queued, run interactively, or rerun selectively through the UI, CLI, or REST API.

The project is currently published as a series of 2.x releases (latest line: v2.10.7) with a companion Helm chart at helm-dagu-1.0.11. Source: README.md:1-40.

Core Components and Runtime Layout

A Dagu installation is composed of several cooperating subsystems that live inside the internal/ tree:

• Scheduler — keeps cron expressions, queue state, and next-run metadata. Schedules can be defined per profile, so a DAG may run on different cron strings in prod versus dev. Source: README.md:60-90.
• Agent (internal/agent) — the per-DAG runner invoked when dagu start executes a DAG; it materializes the execution graph, walks the DAG, and writes attempt metadata through the file-backed stores. Source: Dockerfile:1-40.
• Worker (internal/worker) — polls for distributed DAG runs dispatched by coordinators and shells out to dagu start so that remote workers can execute without sharing a filesystem tree. Source: internal/runtime/builtin/docker/keepalive/README.md:1-20.
• Server / Web UI — exposes REST endpoints, the Web UI, and the API docs page; serves DAG definitions, status, and live logs. Source: README.md:40-60.

The Docker image is a thin wrapper around the dagu binary and is the basis for both single-container and Kubernetes deployments. Source: Dockerfile:1-40.

Deployment Options

Dagu supports four deployment paths, each documented in its own subdirectory:

The Helm chart is also published as a GitHub release artifact so it can be added with helm repo add dagu https://dagucloud.github.io/dagu. Source: charts/dagu/README.md:1-40.

For Kubernetes installs, the manifests configure persistent volumes that the worker, coordinator, scheduler, and UI server must all be able to read and write. A long-standing community discussion (issue #1260) tracks the desire to relax that requirement so workers can pull DAGs from remote storage instead of a shared filesystem. Source: deploy/k8s/README.md:1-40.

Key Features

Dagu's feature set spans authoring, execution, and operations:

• YAML DAG authoring with steps, dependencies, preconditions, retry policies, and stdout.outputs for capturing values into run outputs. Source: README.md:90-140.
• Lifecycle handlers — handler_on blocks support .init, .exit, .success, .failure, and .cancel. Note community-known gaps: onInit records a stdout path but is not surfaced on the Web UI handler table (issue #2429), and stdout.outputs declared on a handler step is captured but not currently published to the run's outputs (issue #2428). Source: README.md:140-180.
• Profiles and per-profile schedules, allowing the same DAG to express different cron expressions and parameters depending on the active profile (added in v2.10.0). Source: README.md:180-220.
• Preconditions with explicit eval distinguishing value comparison from dynamic evaluation (v2.10.4). Source: README.md:220-260.
• Executors including command, docker, http (with output file capture, v2.10.0), and others. Source: README.md:260-300.
• Distributed execution via coordinator/worker split, with hardened file-backed record recovery and cancellation handling during startup (v2.10.5). Source: README.md:300-340.
• Built-in authentication with corrected long-lived token TTL handling so sessions do not expire mid-life due to browser timer limits (v2.10.6). Source: README.md:340-380.
• Rerun modes — single-step retry is supported today; an open request (issue #2388) asks for an explicit "rerun selected step and all downstream steps" mode. Source: README.md:380-420.
• Web UI improvements including saved Kanban views, relative timestamps, searchable ANSI-colored logs, and automatic expansion of failed steps (v2.10.7). Source: README.md:420-460.

The internal/runtime/builtin/docker/keepalive package is used by the Docker executor to keep long-running containers alive across scheduler restarts. Source: internal/runtime/builtin/docker/keepalive/README.md:1-20.

Operational Notes and Known Gaps

A few recurring operational themes surface in the issue tracker and release notes:

• Queued runs that fail before startup can become stuck and be re-dispatched on every tick (issue #2436). Workarounds are to ensure the runnable path is valid or to manually purge the queue.
• Subdirectories under dagsDir are not enumerated by the UI/Definitions list today; only top-level YAML files appear (issue #2400).
• Authentication currently relies on built-in username/password plus tokens; OAuth/OpenID providers (Google, GitHub, Keycloak, Twitter) are tracked as an open feature request (issue #443).
• Licensing is under review — the maintainers are evaluating a move from GPLv3 to a more permissive license such as LGPL-3.0, MIT, or Apache-2.0 (issue #1393).
• Worker sync mode is the leading distributed-execution enhancement request (issue #1260), motivated by the requirement today that all components share a filesystem tree.

Dagu's design — single binary, file-backed stores, YAML-first authoring — keeps operational surface area small while still supporting local, container, and orchestrated deployments. Source: README.md:460-520.

Related Pages

Related topics: Overview, Lib

Lib

The ui/src/lib directory is a shared utility module inside Dagu's React/TypeScript frontend. It collects pure, presentation-agnostic helpers that are reused across pages, components, hooks, and API clients. Unlike feature folders (such as features/ or components/), the lib module contains no React components of its own — it only exports functions, constants, and types. Centralizing these helpers keeps the UI codebase DRY and gives consumers a single, predictable place to import foundational logic such as authentication, validation, time formatting, ANSI rendering, and project-wide constants.

Purpose and Scope

The lib module serves three main concerns:

• Cross-cutting helpers — utilities like ansi.tsx and dagRunTiming.ts that any feature can import without pulling in side effects.
• Authentication glue — authHeaders.ts and authSession.ts encode how the frontend talks to Dagu's protected endpoints. As noted in v2.10.6, long-lived token handling was hardened here so that sessions are no longer cleared prematurely by the browser's timer limit. Source: ui/src/lib/authHeaders.ts:1-40
• Shared constants and validators — constants.ts and dag-validation.ts provide values and structural checks that the editor, run page, and DAG listing rely on consistently.

Because the module is intentionally side-effect free, importing from lib is safe in both client and server contexts, which lets the codebase use it during build-time rendering as well as at runtime.

Module Structure

Authentication Helpers

authSession.ts owns the lifecycle of a logged-in user: it persists tokens, tracks expiration, and surfaces whether the next request needs a refresh. authHeaders.ts consumes that state and produces request headers on demand so each fetch call does not have to re-derive them. The split exists because header construction is called on every network request while session lifetime checks are called less frequently, so separating them avoids unnecessary work on the hot path. Source: ui/src/lib/authSession.ts:1-60

Log Rendering with ANSI

The ansi.tsx file converts ANSI color and control sequences that originate from dagu start subprocess output into React elements that the UI can render alongside plain text. This is essential because v2.10.7 introduced searchable ANSI-colored logs and automatic failed-step expansion; without this helper the new log viewer would render escape codes as literal characters. The module exposes a parser plus a component wrapper so logs can be streamed incrementally without re-parsing unchanged prefixes. Source: ui/src/lib/ansi.tsx:1-80

DAG Validation

dag-validation.ts implements schema-level checks for DAG definitions before they are persisted or queued. It catches issues such as duplicate step IDs, invalid handler_on lifecycle handlers, and malformed stdout.outputs declarations. This validation layer complements the server-side validation performed by internal/agent and is especially relevant for the open issues around lifecycle handlers — for example, the bug where stdout.outputs declared on a handler_on step is captured but never published to the DAG run outputs. Source: ui/src/lib/dag-validation.ts:1-120

Run Timing

dagRunTiming.ts formats run start times, computes elapsed durations, and produces relative timestamps such as "started 3 minutes ago". It also handles schedule edge cases (daylight saving, queued runs that fail before startup) so that the UI shows consistent timing information regardless of how a run was initiated. This file is tightly coupled to the fix delivered in v2.10.5 around queued-run recovery and with v2.10.1's reduced server-side polling for next-run data. Source: ui/src/lib/dagRunTiming.ts:1-90

Data Flow

The following diagram summarizes how lib interacts with the rest of the frontend when rendering a DAG run page:

flowchart LR
  A[React Component] -->|imports| B[ui/src/lib]
  B --> C[authSession]
  B --> D[authHeaders]
  B --> E[ansi.tsx]
  B --> F[dag-validation.ts]
  B --> G[dagRunTiming.ts]
  B --> H[constants.ts]
  C --> D
  D --> I[fetch / API client]
  E --> A
  F --> A
  G --> A
  H --> A

Components call into the helpers, the helpers return primitives or styled nodes, and only authHeaders talks to the network layer — keeping network access localized. This pattern means that if authentication behavior changes (as it did in v2.10.6), only authSession.ts and authHeaders.ts need updating.

Related Community Context

Several ongoing community discussions touch this module indirectly. The re-licensing effort in #1393 does not affect code, but the worker sync mode request in #1260 and the OAuth/OpenID authentication tracking in #443 both propose changes that would land in authSession.ts and authHeaders.ts. The handler-related issues (#2428, #2429) and the queued-run bug (#2436) signal where dag-validation.ts and dagRunTiming.ts will need future enhancements to surface lifecycle handler output and to recover gracefully when runs fail before startup.

Doramagic Pitfall Log

Found 26 structured pitfall item(s), including 1 high/blocking item(s). Top priority: Security or permission risk - Security or permission risk requires verification.

1. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/dagucloud/dagu/issues/2388

2. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: identity.distribution | https://github.com/dagucloud/dagu

3. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: bug: stdout.outputs declared on a handler_on step is captured but never published to the DAG run outputs
• User impact: Developers may fail before the first successful local run: bug: stdout.outputs declared on a handler_on step is captured but never published to the DAG run outputs
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: bug: stdout.outputs declared on a handler_on step is captured but never published to the DAG run outputs. Context: Observed when using node, python, linux
• Evidence: failure_mode_cluster:github_issue | https://github.com/dagucloud/dagu/issues/2428

4. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: bug: queued run that fails before startup is never dequeued and retries forever
• User impact: Developers may fail before the first successful local run: bug: queued run that fails before startup is never dequeued and retries forever
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: bug: queued run that fails before startup is never dequeued and retries forever. Context: Observed when using macos
• Evidence: failure_mode_cluster:github_issue | https://github.com/dagucloud/dagu/issues/2436

5. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: helm-dagu-1.0.10
• User impact: Upgrade or migration may change expected behavior: helm-dagu-1.0.10
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: helm-dagu-1.0.10. Context: Observed during installation or first-run setup.
• Evidence: failure_mode_cluster:github_release | https://github.com/dagucloud/dagu/releases/tag/helm-dagu-1.0.10

6. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: helm-dagu-1.0.11
• User impact: Upgrade or migration may change expected behavior: helm-dagu-1.0.11
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: helm-dagu-1.0.11. Context: Observed during installation or first-run setup.
• Evidence: failure_mode_cluster:github_release | https://github.com/dagucloud/dagu/releases/tag/helm-dagu-1.0.11

7. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: question: onInit handler is recorded but never displayed — is that intended?
• User impact: Developers may fail before the first successful local run: question: onInit handler is recorded but never displayed — is that intended?
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: question: onInit handler is recorded but never displayed — is that intended?. Context: Observed when using python, linux
• Evidence: failure_mode_cluster:github_issue | https://github.com/dagucloud/dagu/issues/2429

8. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Developers should check this installation risk before relying on the project: v2.10.0
• User impact: Upgrade or migration may change expected behavior: v2.10.0
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v2.10.0. Context: Observed during installation or first-run setup.
• Evidence: failure_mode_cluster:github_release | https://github.com/dagucloud/dagu/releases/tag/v2.10.0

9. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/dagucloud/dagu/issues/2436

10. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/dagucloud/dagu/issues/2429

11. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Developers should check this configuration risk before relying on the project: v2.10.4
• User impact: Upgrade or migration may change expected behavior: v2.10.4
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v2.10.4. Context: Observed when using windows
• Evidence: failure_mode_cluster:github_release | https://github.com/dagucloud/dagu/releases/tag/v2.10.4

12. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Developers should check this configuration risk before relying on the project: v2.10.5
• User impact: Upgrade or migration may change expected behavior: v2.10.5
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: v2.10.5. Context: Source discussion did not expose a precise runtime context.
• Evidence: failure_mode_cluster:github_release | https://github.com/dagucloud/dagu/releases/tag/v2.10.5

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using dagu with real data or production workflows.

• question: onInit handler is recorded but never displayed — is that int - github / github_issue
• bug: stdout.outputs declared on a handler_on step is captured but ne - github / github_issue
• bug: queued run that fails before startup is never dequeued and retries - github / github_issue
• feat: rerun a selected step and all downstream steps - github / github_issue
• helm-dagu-1.0.11 - github / github_release
• v2.10.7 - github / github_release
• v2.10.6 - github / github_release
• v2.10.5 - github / github_release
• v2.10.4 - github / github_release
• v2.10.1 - github / github_release
• Installation risk requires verification - GitHub / issue
• Installation risk requires verification - GitHub / issue