<p align="center">
<img src="./assets/images/dagu-logo-dark.png#gh-dark-mode-only" width="300" alt="dagu-logo">
<img src="./assets/images/dagu-logo-light.png#gh-light-mode-only" width="300" alt="dagu-logo">
</p>
<p align="center">
<a href="https://goreportcard.com/report/github.com/yohamta/dagu">
<img src="https://goreportcard.com/badge/github.com/yohamta/dagu" />
</a>
<a href="https://codecov.io/gh/dagu-dev/dagu">
<img src="https://codecov.io/gh/dagu-dev/dagu/branch/main/graph/badge.svg?token=CODZQP61J2" />
</a>
<a href="https://github.com/dagu-dev/dagu/releases">
<img src="https://img.shields.io/github/release/yohamta/dagu.svg" />
</a>
<a href="https://godoc.org/github.com/yohamta/dagu">
<img src="https://godoc.org/github.com/yohamta/dagu?status.svg" />
</a>
<img src="https://github.com/dagu-dev/dagu/actions/workflows/test.yaml/badge.svg" />
</p>
<div align="center">
[Installation](#installation) | [Community](https://discord.gg/gpahPUjGRk) | [Quick Start](#️quick-start-guide)
</div>
<h1><b>Dagu</b></h1>
Dagu is a powerful Cron alternative that comes with a Web UI. It allows you to define dependencies between commands as a [Directed Acyclic Graph (DAG)](https://en.wikipedia.org/wiki/Directed_acyclic_graph) in a declarative [YAML format](https://dagu.readthedocs.io/en/latest/yaml_format.html). Additionally, Dagu natively supports running Docker containers, making HTTP requests, and executing commands over SSH.
- [Documentation](https://dagu.readthedocs.io)
- [Discord Community](https://discord.gg/gpahPUjGRk)
## <a name='Highlights'></a>**Highlights**
- Single binary file installation
- Declarative YAML format for defining DAGs
- Web UI for visually managing, rerunning, and monitoring pipelines
- Use existing programs without any modification
- Self-contained, with no need for a DBMS
## <a name='Contents'></a>**Contents**
<!-- vscode-markdown-toc -->
* [**Highlights**](#Highlights)
* [**Contents**](#Contents)
* [**Features**](#Features)
* [**Usecase**](#Usecase)
* [**Web UI**](#WebUI)
* [DAG Details](#DAGDetails)
* [DAGs List](#DAGsList)
* [Search DAGs](#SearchDAGs)
* [Execution History](#ExecutionHistory)
* [DAG Execution Log](#DAGExecutionLog)
* [**Installation**](#Installation)
* [Via Homebrew](#ViaHomebrew)
* [Via Bash script](#ViaBashscript)
* [Via Docker](#ViaDocker)
* [Via GitHub Release Page](#ViaGitHubReleasePage)
* [️**Quick Start Guide**](#QuickStartGuide)
* [1. Launch the Web UI](#LaunchtheWebUI)
* [2. Create a New DAG](#CreateaNewDAG)
* [3. Edit the DAG](#EdittheDAG)
* [4. Execute the DAG](#ExecutetheDAG)
* [**CLI**](#CLI)
* [**Documentation**](#Documentation)
* [**Running as a daemon**](#Runningasadaemon)
* [**Example Workflow**](#ExampleWorkflow)
* [**Motivation**](#Motivation)
* [**Why Not Use an Existing Workflow Scheduler Like Airflow?**](#WhyNotUseanExistingWorkflowSchedulerLikeAirflow)
* [**How It Works**](#HowItWorks)
* [**License**](#License)
* [**Support and Community**](#SupportandCommunity)
<!-- vscode-markdown-toc-config
numbering=false
autoSave=true
/vscode-markdown-toc-config -->
<!-- /vscode-markdown-toc -->
## <a name='Features'></a>**Features**
- Web User Interface
- Command Line Interface (CLI) with several commands for running and managing DAGs
- YAML format for defining DAGs, with support for various features including:
- Execution of custom code snippets
- Parameters
- Command substitution
- Conditional logic
- Redirection of stdout and stderr
- Lifecycle hooks
- Repeating task
- Automatic retry
- Executors for running different types of tasks:
- Running arbitrary Docker containers
- Making HTTP requests
- Sending emails
- Running jq command
- Executing remote commands via SSH
- Email notification
- Scheduling with Cron expressions
- REST API Interface
- Basic Authentication over HTTPS
## <a name='Usecase'></a>**Usecase**
- **Data Pipeline Automation:** Schedule ETL tasks for data processing and centralization.
- **Infrastructure Monitoring:** Periodically check infrastructure components with HTTP requests or SSH commands.
- **Automated Reporting:** Generate and send periodic reports via email.
- **Batch Processing:** Schedule batch jobs for tasks like data cleansing or model training.
- **Task Dependency Management:** Manage complex workflows with interdependent tasks.
- **Microservices Orchestration:** Define and manage dependencies between microservices.
- **CI/CD Integration:** Automate code deployment, testing, and environment updates.
- **Alerting System:** Create notifications based on specific triggers or conditions.
- **Custom Task Automation:** Define and schedule custom tasks using code snippets.
## <a name='WebUI'></a>**Web UI**
### <a name='DAGDetails'></a>DAG Details
It shows the real-time status, logs, and DAG configurations. You can edit DAG configurations on a browser.
You can switch to the vertical graph with the button on the top right corner.
### <a name='DAGsList'></a>DAGs List
It shows all DAGs and the real-time status.
### <a name='SearchDAGs'></a>Search DAGs
It greps given text across all DAGs.
### <a name='ExecutionHistory'></a>Execution History
It shows past execution results and logs.
### <a name='DAGExecutionLog'></a>DAG Execution Log
It shows the detail log and standard output of each execution and step.
## <a name='Installation'></a>**Installation**
You can install Dagu quickly using Homebrew or by downloading the latest binary from the Releases page on GitHub.
### <a name='ViaHomebrew'></a>Via Homebrew
```sh
brew install yohamta/tap/dagu
```
Upgrade to the latest version:
```sh
brew upgrade yohamta/tap/dagu
```
### <a name='ViaBashscript'></a>Via Bash script
```sh
curl -L https://raw.githubusercontent.com/yohamta/dagu/main/scripts/downloader.sh | bash
```
### <a name='ViaDocker'></a>Via Docker
```sh
docker run \
--rm \
-p 8080:8080 \
-v $HOME/.dagu/dags:/home/dagu/.dagu/dags \
-v $HOME/.dagu/data:/home/dagu/.dagu/data \
-v $HOME/.dagu/logs:/home/dagu/.dagu/logs \
yohamta/dagu:latest
```
### <a name='ViaGitHubReleasePage'></a>Via GitHub Release Page
Download the latest binary from the [Releases page](https://github.com/dagu-dev/dagu/releases) and place it in your `$PATH` (e.g. `/usr/local/bin`).
## <a name='QuickStartGuide'></a>️**Quick Start Guide**
### <a name='LaunchtheWebUI'></a>1. Launch the Web UI
Start the server and scheduler with the command `dagu start-all` and browse to `http://127.0.0.1:8080` to explore the Web UI.
### <a name='CreateaNewDAG'></a>2. Create a New DAG
Navigate to the DAG List page by clicking the menu in the left panel of the Web UI. Then create a DAG by clicking the `New DAG` button at the top of the page. Enter `example` in the dialog.
*Note: DAG (YAML) files will be placed in `~/.dagu/dags` by default. See [Configuration Options](https://dagu.readthedocs.io/en/latest/config.html) for more details.*
### <a name='EdittheDAG'></a>3. Edit the DAG
Go to the `SPEC` Tab and hit the `Edit` button. Copy & Paste the following example and click the `Save` button.
Example:
```yaml
schedule: "* * * * *" # Run the DAG every minute
steps:
- name: s1
command: echo Hello Dagu
- name: s2
command: echo done!
depends:
- s1
```
### <a name='ExecutetheDAG'></a>4. Execute the DAG
You can execute the example by pressing the `Start` button. You can see "Hello Dagu" in the log page in the Web UI.
## <a name='CLI'></a>**CLI**
```sh
# Runs the DAG
dagu start [--params=<params>] <file>
# Displays the current status of the DAG
dagu status <file>
# Re-runs the specified DAG run
dagu retry --req=<request-id> <file>
