📚 taskctl - Awesome Go Library for Build Automation
Concurrent task runner.
Detailed Description of taskctl
taskctl - concurrent task runner, developer's routine tasks automation toolkit
Stand with Ukraine!
While Russia is destroying my home and my country, killing my friends and neighbors - any russian company, organization, or citizen, who do nothing about it, is prohibited from using this package. For others - please, pray for us, share information about war crimes Russia is conducting in Ukraine, do everything you can to urge your governments to be on the right side of history. Ukraine will prevail! Good triumph over evil! Русский военный корабль, иди нах#й!
Simple modern alternative to GNU Make. taskctl is concurrent task runner that allows you to design you routine tasks and development pipelines in nice and neat way in human-readable format (YAML, JSON or TOML). Given a pipeline (composed of tasks or other pipelines) it builds a graph that outlines the execution plan. Each task my run concurrently or cascade. Beside pipelines, each single task can be started manually or triggered by built-in filesystem watcher.
Features
- human-readable configuration format (YAML, JSON or TOML)
- concurrent tasks execution
- highly customizable execution plan
- cross platform
- import local or remote configurations
- integrated file watcher (live reload)
- customizable execution contexts
- different output types
- embeddable task runner
- interactive prompt
- handy autocomplete
- and many more...
tasks:
lint:
command:
- golint $(go list ./... | grep -v /vendor/)
- go vet $(go list ./... | grep -v /vendor/)
test:
allow_failure: true
command: go test ./....
build:
command: go build -o bin/app ./...
env:
GOOS: linux
GOARCH: amd64
before: rm -rf bin/*
pipelines:
release:
- task: lint
- task: test
- task: build
depends_on: [lint, test]
According to this plan lint
and test
will run concurrently, build
will start only when both lint
and test
finished.
Contents
- Getting started
- Configuration
- Tasks
- Pipelines
- Filesystem watchers
- Contexts
- Output formats
- Embeddable task runner
- FAQ
- Autocomplete
- Similar projects
- How to contribute?
- License
Getting started
Install
MacOS
brew tap taskctl/taskctl
brew install taskctl
Linux
sudo wget https://github.com/taskctl/taskctl/releases/latest/download/taskctl_linux_amd64 -O /usr/local/bin/taskctl
sudo chmod +x /usr/local/bin/taskctl
Ubuntu Linux
sudo snap install --classic taskctl
deb/rpm:
Download the .deb or .rpm from the releases page and install with dpkg -i
and rpm -i
respectively.
Windows
scoop bucket add taskctl https://github.com/taskctl/scoop-taskctl.git
scoop install taskctl
Installation script
curl -sL https://raw.githubusercontent.com/taskctl/taskctl/master/install.sh | sh
From sources
git clone https://github.com/taskctl/taskctl
cd taskctl
go build -o taskctl .
Docker images
Docker images available on Docker hub
Usage
taskctl
- run interactive task prompttaskctl pipeline1
- run single pipelinetaskctl task1
- run single tasktaskctl pipeline1 task1
- run one or more pipelines and/or taskstaskctl watch watcher1 watcher2
- start one or more watchers
Configuration
taskctl uses config file (tasks.yaml
or taskctl.yaml
) where your tasks and pipelines stored.
Config file includes following sections:
- tasks
- pipelines
- watchers
- contexts
- variables
Config file may import other config files, directories or URLs.
import:
- .tasks/database.yaml
- .tasks/lint/
- https://raw.githubusercontent.com/taskctl/taskctl/master/docs/example.yaml
Example
Config file example
Global configuration
taskctl has global configuration stored in $HOME/.taskctl/config.yaml
file. It is handy to store system-wide tasks, reusable contexts, defaults etc.
Tasks
Task is a foundation of taskctl. It describes one or more commands to run, their environment, executors and attributes such as working directory, execution timeout, acceptance of failure, etc.
tasks:
lint:
allow_failure: true
command:
- golint $(go list ./... | grep -v /vendor/)
- go vet $(go list ./... | grep -v /vendor/)
build:
command: go build ./...
env:
GOOS: linux
GOARCH: amd64
env_file: /data/.env
after: rm -rf tmp/*
variations:
- GOARCH: amd64
- GOARCH: arm
GOARM: 7
Task definition takes following parameters:
command
- one or more commands to runvariations
- list of variations (env variables) to apply to commandcontext
- execution context's nameenv
- environment variables. All existing environment variables will be passed automaticallyenv_file
- env file ink=v
format to read variables fromdir
- working directory. Current working directory by defaulttimeout
- command execution timeout (default: none)allow_failure
- if set totrue
failed commands will not interrupt execution (default:false
)after
- command that will be executed after command completesbefore
- command that will be executed before task startsexportAs
- env variable name to store task's output (default:TASK_NAME_OUTPUT
, whereTASK_NAME
is actual task's name)condition
- condition to check before running taskvariables
- task's variablesinteractive
- iftrue
provides STDIN to commands (default:false
)
Tasks variables
Each task, stage and context has variables to be used to render task's fields - command
, dir
.
Along with globally predefined, variables can be set in a task's definition.
You can use those variables according to text/template
documentation.
Predefined variables are:
.Root
- root config file directory.Dir
- config file directory.TempDir
- system's temporary directory.Args
- provided arguments as a string.ArgsList
- array of provided arguments.Task.Name
- current task's name.Context.Name
- current task's execution context's name.Stage.Name
- current stage's name.Output
- previous command's output.Tasks.Task1.Output
-task1
last command output
Variables can be used inside task definition. For example:
tasks:
task1:
dir: "{{ .Root }}/some-dir"
command:
- echo "My name is {{ .Task.Name }}"
- echo {{ .Output }} # My name is task1
- echo "Sleep for {{ .sleep }} seconds"
- sleep {{ .sleep | default 10 }}
- sleep {{ .sleep }}
variables:
sleep: 3
Pass CLI arguments to task
Any command line arguments succeeding --
are passed to each task via .Args
, .ArgsList
variables or ARGS
environment variable.
Given this definition:
lint1:
command: go lint {{.Args}}
lint2:
command: go lint {{index .ArgsList 1}}
the resulting command is:
$ taskctl lint1 -- package.go
# go lint package.go
$ taskctl lint2 -- package.go main.go
# go lint main.go
Storing task's output
Task output automatically stored to the variable named like this - .Tasks.TaskName.Output
, where TaskName
is the actual task's name.
It is also stored to TASK_NAME_OUTPUT
environment variable. It's name can be changed by a task's exportAs
parameter.
Those variables will be available to all dependent stages.
Tasks variations
Task may run in one or more variations. Variations allows to reuse task with different env variables:
tasks:
build:
command:
- GOOS=${GOOS} GOARCH=amd64 go build -o bin/taskctl_${GOOS} ./cmd/taskctl
env:
GOFLAGS: -ldflags=-s -ldflags=-w
variations:
- GOOS: linux
- GOOS: darwin
- GOOS: windows
this config will run build 3 times with different GOOS
Task conditional execution
The following task will run only when there are any changes that are staged but not committed:
tasks:
build:
command:
- ...build...
condition: git diff --exit-code
Pipelines
Pipeline is a set of stages (tasks or other pipelines) to be executed in a certain order. Stages may be executed in parallel or one-by-one. Stage may override task's environment, variables etc.
This pipeline:
pipelines:
pipeline1:
- task: start task
- task: task A
depends_on: "start task"
- task: task B
depends_on: "start task"
- task: task C
depends_on: "start task"
- task: task D
depends_on: "task C"
- task: task E
depends_on: ["task A", "task B", "task D"]
- task: finish
depends_on: ["task E"]
will result in an execution plan like this:
Stage definition takes following parameters:
name
- stage name. If not set - referenced task or pipeline name will be used.task
- task to execute on this stagepipeline
- pipeline to execute on this stageenv
- environment variables. All existing environment variables will be passed automaticallydepends_on
- name of stage on which this stage depends on. This stage will be started only after referenced stage is completed.allow_failure
- iftrue
failing stage will not interrupt pipeline execution.false
by defaultcondition
- condition to check before running stagevariables
- stage's variables
Taskctl output formats
Taskctl has several output formats:
raw
- prints raw commands outputprefixed
- strips ANSI escape sequences where possible, prefixes command output with task's namecockpit
- tasks dashboard
Filesystem watchers
Watcher watches for changes in files selected by provided patterns and triggers task anytime an event has occurred.
watchers:
watcher1:
watch: ["README.*", "pkg/**/*.go"] # Files to watch
exclude: ["pkg/excluded.go", "pkg/excluded-dir/*"] # Exclude patterns
events: [create, write, remove, rename, chmod] # Filesystem events to listen to
task: task1 # Task to run when event occurs
Patterns
Thanks to doublestar taskctl supports the following special terms within include and exclude patterns:
Special Terms | Meaning |
---|---|
* | matches any sequence of non-path-separators |
** | matches any sequence of characters, including path separators |
? | matches any single non-path-separator character |
[class] | matches any single non-path-separator character against a class of characters (details) |
{alt1,...} | matches a sequence of characters if one of the comma-separated alternatives matches |
Any character with a special meaning can be escaped with a backslash (\
).
Contexts
Contexts allow you to set up execution environment, variables, binary which will run your task, up/down commands etc.
contexts:
local:
executable:
bin: /bin/zsh
args:
- -c
env:
VAR_NAME: VAR_VALUE
variables:
sleep: 10
quote: "'" # will quote command with provided symbol: "/bin/zsh -c 'echo 1'"
before: echo "I'm local context!"
after: echo "Have a nice day!"
Context has hooks which may be triggered once before first context usage or every time before task with this context will run.
context:
docker-compose:
executable:
bin: docker-compose
args: ["exec", "api"]
up: docker-compose up -d api
down: docker-compose down api
local:
after: rm -rf var/*
Docker context
alpine:
executable:
bin: /usr/local/bin/docker
args:
- run
- --rm
- alpine:latest
env:
DOCKER_HOST: "tcp://0.0.0.0:2375"
before: echo "SOME COMMAND TO RUN BEFORE TASK"
after: echo "SOME COMMAND TO RUN WHEN TASK FINISHED SUCCESSFULLY"
tasks:
mysql-task:
context: alpine
command: uname -a
Embeddable task runner
taskctl may be embedded into any go program. Additional information may be found on taskctl's pkg.go.dev page
Runner
t := task.FromCommands("go fmt ./...", "go build ./..")
r, err := NewTaskRunner()
if err != nil {
return
}
err = r.Run(t)
if err != nil {
fmt.Println(err, t.ExitCode, t.ErrorMessage())
}
fmt.Println(t.Output())
Scheduler
format := task.FromCommands("go fmt ./...")
build := task.FromCommands("go build ./..")
r, _ := runner.NewTaskRunner()
s := NewScheduler(r)
graph, err := NewExecutionGraph(
&Stage{Name: "format", Task: format},
&Stage{Name: "build", Task: build, DependsOn: []string{"format"}},
)
if err != nil {
return
}
err = s.Schedule(graph)
if err != nil {
fmt.Println(err)
}
FAQ
How does it differ from go-task/task?
It's amazing how solving same problems lead to same solutions. taskctl and go-task have a lot of concepts in common but also have some differences.
- Main is pipelines. Pipelines and stages allows more precise workflow design because same tasks may have different dependencies (or no dependencies) in different scenarios.
- Contexts allow you to set up execution environment and binary which will run your task.
Autocomplete
Bash
Add to ~/.bashrc or ~/.profile
. <(taskctl completion bash)
ZSH
Add to ~/.zshrc
. <(taskctl completion zsh)
Similar projects
How to contribute?
Feel free to contribute in any way you want. Share ideas, submit issues, create pull requests. You can start by improving this README.md or suggesting new features Thank you!
License
This project is licensed under the GNU GPLv3 - see the LICENSE.md file for details
Authors
- Yevhen Terentiev - trntv See also the list of contributors who participated in this project.