Task automation in Go

Please ⭐ Star this repository if you find it valuable and worth maintaining.

Slides.

Table of Contents:

• Description
• Quick start
• Repository template
• Examples
• Defining tasks
• Running programs
• Wrapper scripts
• Using middlewares
• Customizing
• Alternatives
  • Make
  • Mage
  • Task
  • Bazel
• Contributing
• License

goyek (/ˈɡɔɪæk/ 🔊 listen) is a task automation tool. As opposed to many other tools, it is just a Go library with API inspired by testing, cobra, flag, http packages.

Here are some good parts:

• It is cross-platform and shell independent.
• No binary installation is needed.
• It is easy to debug, like a regular Go application.
• The tasks are defined similarly to cobra commands.
• The task actions look like a Go unit test. You may even use testify or fluentassert for asserting.
• You can reuse code like in any Go application. It may be helpful to use packages like fsnotify and viper.
• It is highly customizable.
• It does not use any third-party dependency other than the Go standard library. You can find supplumental features in goyek/x.
• Minimal supported Go version is 1.11.

Supplemental packages from github.com/goyek/x are used for convinence.

The convention is to have the build automation in the /build directory (or even Go module).

Put the following content in /build/hello.go:

package main

import (
	"flag"

	"github.com/goyek/goyek/v2"
	"github.com/goyek/x/cmd"
)

var msg = flag.String("msg", "greeting message", "Hello world!")

var hello = flow.Define(goyek.Task{
	Name:  "hello",
	Usage: "demonstration",
	Action: func(a *goyek.A) {
		a.Log(*msg)
		cmd.Exec(a, "go version")
	},
})

Put the following content in /build/main.go:

package main

import (
	"github.com/goyek/goyek/v2"
	"github.com/goyek/x/boot"
)

func main() {
	goyek.SetDefault(hello)
	boot.Main()
}

Run:

$ go mod tidy

$ go run ./build -h
Usage of build: [flags] [--] [tasks]
Tasks:
  hello  demonstration
Flags:
  -dry-run
        print all tasks without executing actions
  -long-run duration
        print when a task takes longer (default 1m0s)
  -msg string
        Hello world! (default "greeting message")
  -no-color
        disable colorizing output
  -no-deps
        do not process dependencies
  -skip comma-separated tasks
        skip processing the comma-separated tasks
  -v    print all tasks as they are run

$ go run ./build -v
===== TASK  hello
      hello.go:16: greeting message
      hello.go:17: Exec: go version
go version go1.19.3 windows/amd64
----- PASS: hello (0.12s)
ok      0.123s

You can use goyek/template to create a new repository.

For an existing repository you can copy most of its files.

• example_test.go - demonstrative examples
• goyek/template - Go application repository template
• fluentassert - Go library
• build - dogfooding
• splunk-otel-go - multi-module repository
• goyek/demo and goyek/workflow - demonstratation of the reusability potential

Use Define to register a a task.

You can add dependencies to already defineded tasks using Task.Deps. The dependencies are running in sequential order. Each task runs at most once.

The Task.Action is a function which executes when a task is running. A task can have only dependencies and no action to act as a pipeline.

The Task.Parallel can be set to allow a task to be run in parallel with other parallel tasks.

A default task can be assigned using SetDefault.

You can use the cmd.Exec convenient function from goyek/x that should cover most use cases.

Alternatively, you may prefer create your own helpers like Exec in build/exec.go.

#60 and #307 explain why this feature is not out-of-the-box.

Instead of executing go run ./build, you may create wrapper scripts, which you can invoke from any location.

Bash - goyek.sh:

#!/bin/bash
set -euo pipefail

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd "$DIR"
go run ./build "$@"

PowerShell - goyek.ps1:

& go run .\build $args
exit $global:LASTEXITCODE

If /build is a separate Go module, check the goyek.sh and goyek.ps1 scripts.

Call the Use function to setup a task runner interceptor (middleware).

You can use a middleware, for example to: generate a task execution report, add retry logic, export task execution telemetry, etc.

You can use some reusable middlewares from the middleware package. ReportStatus is the most commonly used middleware.

Notice that the boot.Main convenient function from goyek/x sets the most commonly used middlewares and defines flags to configure them.

You can customize the default output by using:

• SetOutput
• SetLogger
• SetUsage
• Execute (instead of Main)

You can also study how github.com/goyek/x is customizing the default behavior.

While Make (Makefile) is currently the de facto standard, it has some pitfalls:

• Requires to learn Make (and often Bash).
• It is hard to develop a Makefile which is truly cross-platform.
• Debugging and testing Make targets is not fun.

goyek is intended to be simpler, easier to learn, more portable, while still being able to handle most use cases.

Mage is a framework/tool which magically discovers the targets from magefiles, which results in some drawbacks.

• It requires using build tags.
• Reusing tasks is hacky.
• It needs installation or use of zero install option, which is slow.
• Debugging is complex.
• It is magical by design (of course, one may like it).

goyek is a non-magical alternative for Mage. It is easier to customize and extend as it is a library that offers extension points. Write regular Go code without build tags and tricky imports.

While Task is simpler and easier to use than Make, but it still has similar problems:

• Requires to learn Task's YAML structure and the minimalistic, cross-platform interpreter.
• Debugging and testing tasks is not easy.
• Hard to make reusable tasks.
• Requires to install the tool.

Bazel is a very sophisticated tool which is created to efficiently handle complex and long-running build pipelines. It requires the build target inputs and outputs to be fully specified.

goyek is just a simple Go library. However, nothing prevents you from, for example, using the github.com/magefile/mage/target package to make your automation more efficient.

See CONTRIBUTING.md if you want to help us.

goyek is licensed under the terms of the MIT license.

Note: goyek was named taskflow before v0.3.0.