Habana-Container-Runtime
A modified version of runc adding a custom pre-start hook to all containers
If environment variable HABANA_VISIBLE_DEVICES
is set in the OCI spec, the hook will configure Habana device access for the container by leveraging habana-container-cli
from project libhabana-container.
Build from source
All binaries are build under dist/{BINARY_NAME}_architecture/{BINARY_NAME}.
Available architectures:
- linux_amd64
- linux_386
- linux_arm64
Build binaries
# Build all binaries
make build-binary
# Build only habana-container-runtime
make build-runtime
# Build only habana-container-hook
make build-hook
# Build only habana-container-cli (libhabana-container)
make build-cli
After building the binaries, copy the config from packaging/config.toml
into /etc/habana-container-runtime/config.toml
and edit for your
environment.
Build package from source
You must have docker installed. Building the packages is done using goreleaser.
make release
Artifacts are found under dist/
folder.
Install pre-built package
Installation and usage guides can be found in the habana.ai docs
Docker Engine setup
Daemon configuration file
sudo tee /etc/docker/daemon.json <<EOF
{
"runtimes": {
"habana": {
"path": "/usr/bin/habana-container-runtime",
"runtimeArgs": []
}
}
}
EOF
sudo systemctl restart docker
You can optionally reconfigure the default runtime by adding the following to /etc/docker/daemon.json
:
"default-runtime": "habana"
ContainerD Setup
Containerd configuration file
sudo tee /etc/containerd/config.toml <<EOF
disabled_plugins = []
version = 2
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "habana"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.habana]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.habana.options]
BinaryName = "/usr/bin/habana-container-runtime"
[plugins."io.containerd.runtime.v1.linux"]
runtime = "habana-container-runtime"
EOF
sudo systemctl restart containerd
CRI-O Setup
Create new config file at /etc/crio/crio.conf.d/99-habana-ai.conf
.
CRI-O configuration file
[crio.runtime]
default_runtime = "habana-ai"
[crio.runtime.runtimes.habana-ai]
runtime_path = "/usr/local/habana/bin/habana-container-runtime"
monitor_env = [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
]
Restart crio service: systemctl restart crio.service
Usage example
Currently habana-container-runtime has to be used with habana-container-hook and libhabana-container
Bellow is the case when host machine has 8 Habana devices and mount all
by HABANA_VISIBLE_DEVICES=all
docker run --rm --runtime=habana -e HABANA_VISIBLE_DEVICES=all ubuntu:22.04 /bin/bash -c "ls /dev/accel/*"
/dev/accel/accel0
/dev/accel/accel1
/dev/accel/accel2
/dev/accel/accel3
/dev/accel/accel4
/dev/accel/accel5
/dev/accel/accel6
/dev/accel/accel7
/dev/accel/accel_controlD0
/dev/accel/accel_controlD1
/dev/accel/accel_controlD2
/dev/accel/accel_controlD3
/dev/accel/accel_controlD4
/dev/accel/accel_controlD5
/dev/accel/accel_controlD6
/dev/accel/accel_controlD7
Environment variables (OCI spec)
Each environment variable maps to an command-line argument for habana-container-cli
from libhabana-container.
HABANA_VISIBLE_DEVICES
This variable controls which Habana devices will be made accessible inside the container.
Possible values
0,1,2
…: a comma-separated list of index(es).
all
: all Habana devices will be accessible, this is the default value in our container images.
HABANA_RUNTIME_ERROR
Auto generated
Variable hold the last error from the runtime flow. The runtime
does not fail the pod creation in most cases, so we propagate the error inside the container for debugging purposes.
Config
See options here
Issues and Contributing