convey

Convey is a pipeline for running containers and protecting the host as much as possible.

It does this by creating a volume and making it available to each item in the pipeline.

License

GPLv3+

Concepts

Convey is build around a concept that I call a build container. A build container is a container that has all of the dependencies to turn an input into something else. This could be source code or even just data processing. For example the convey/go-build image is a simple wrapper around the golang image that just builds a project instead of installing it.

Convey is built around this concept to allow you to build/process anything on any platform while at the same time protecting the host from a malicious config. This protection is provided by not exposing dangerous options like volume mounts or port forwards while running containers. This is accomplished by creating a volume a data volume at the start of the build plan and volume mounting it into containers as they are run. This allows you to pass state between modular tasks that can be reused between multiple plans.

Once built, convey can build itself entirely. This is done by using an import task to copy the source code into the workspace. Once that stage is complete, it runs another stage that builds for linux-x86_64, windows-x86_64, and darwin-x86_64 in parallel. The final stage will take those compiled binaries and export them back to the hosts file system.

Of course you don't have to export back to the host file system. Since these are just containers, you could instead upload them to your staging environment, packagecloud.io, bintray.com, whatever. Of you could export to the host and create another plan that will import the artifacts and then publish them.

Usage

usage: convey [<flags>] [<plan>]

Flags:
      --help               Show context-sensitive help (also try --help-long and --help-man).
  -f, --config=convey.yml  The config file name to use
  -e, --env=ENV ...        Set environment variables
  -v, --verbose            Be more verbose
  -S, --force-sequential   Don't run anything concurrently
      --color              Enable colorized output
  -g, --graphviz           Output a graphviz diagram of the config file
  -L, --list-tasks         List the supported tasks

Args:
  [<plan>]  The plan to run

Configuration

Configuration is done via a file named convey.yaml. This file defines the tasks as well as the plans. There are three top level-items: environment, plans, and tasks.

Tasks

The tasks section defines each task. By defining them separately from the plan the tasks can be reused in multiple plans.

There are many tasks types that you can use by setting the type attribute. However if the type attribute is not set, it default to run.

Build task

A build task will build a docker image.

Attributes

Example

build-image:
  type: build
  dockerfile: Dockerfile
  tag: myimage:latest

Export tasks

An export task copies files from the volume to the host.

Attributes

Example

export:
  type: export
  files:
    - logo.png
    - binary:binary-linux-x86_64

Import tasks

An import task copies files from the host to the volume. It has one required attribute named sources. This is a list of files relative to the directory that convey was run from that will be copied into the volume.

Attributes

Example

import:
  type: import
  files:
    - Dockerfile
    - src:source

Login task

A login task will run docker login to login to a registry.

Attributes

Example

registry-login:
  type: login
  username: superuser1
  password: abc123

Logout task

A logout task will log you out from a Docker registry.

Attributes

Example

registry-logout:
  type: logout
  server: registry.my.domain:5000

Pull tasks

A pull task pulls an image.

Attributes

Example

pull-alpine:
  type: pull
  image: gliderlabs/alpine:edge

Push tasks

A push task pushes an image.

Attributes

Example

push-image:
  type: push
  image: registry.my.domain:5000/newimage:latest

Remove tasks

A remove task removes an image.

Attributes

Example

remove-image:
  type: remove
  image: registry.my.domain:5000/newimage:latest

Run tasks

A run task runs an image.

Attributes

Example

build-golang:
  type: run
  image: golang:onbuild

Tag Tasks

A tag task will tag an existing image.

Attributes

Example

tag-development:
  source: registry.my.domain:5000/newimage:latest
  destination: registry.my.domain:5000/newimage:development

Environment

The environment section defines a list of environment variables to set for the run.

It should be specified in a NAME or NAME=VALUE format. If no value is provided, the value of the variable from the host will be provided if it is available.

Example

environment:
  - BUILD=development
  - SECRET_TOKEN

Plans

The plans sections defines your execution plans which tie everything together by organizing tasks into stages into the order that they will be executed.

Each plan is defined by a name in the plans section.

Example

plans:
  plan1:
    ...
  plan2:
    ...

Plan

Plans themselves are pretty straight forward. They group a stages with an optional environment.

Attributes

Stages

Stages group tasks together into a logical unit. They can be run sequentially or concurrently.

Attributes