Action Environment¶
In order to successfully build a project, each action needs to run in an environment that matches its requirements. For example, an action running the C++ compiler gcc may require a Linux environment with an appropriate version of gcc installed. At the same time, actions must be isolated from each other to prevent build breakages, non-deterministic outputs, and cache poisoning.
This document outlines the facilities provided by the EngFlow Remote Execution service to configure the action's runtime environment and isolating actions from each other.
Action isolation (also known as sandboxing) prevents actions from interfering with each other and with the underlying system. This avoids build breakages that might otherwise occur and makes the build more reliable. Restricting CPU usage also decreases action time variance, making the behavior of multi-threaded actions more predictable. These properties are especially important for release builds.
The EngFlow Remote Execution Service supports multiple levels of isolation depending on the host system (Linux, MacOS), and the deployment architecture.
On Linux, we support Docker, sandbox, and local execution.
On MacOS, we support local execution only due to restrictions of the underlying platform.
You have to enable at least one execution mechanism to be able to run actions.
Linux Docker Actions¶
On Linux, we recommend using Docker execution and configuring it to isolate
actions, see --allow_docker. This mode provides the flexibility
of running different actions with different Docker images, different network
settings, and different machine resources (such as CPU and memory).
This mode is only available on Linux, and only if the worker service can subprocess to Docker. More specifically, it is currently only available in VM- and bare-metal deployments, but not in Kubernetes-based deployments where the worker service is already running inside a Docker container.
Client-side, you can request different Docker images for individual actions, enable or disable system capabilities, enable networking, and allow sibling Docker containers. See Docker platform options for more documentation.
Server-side, we recommend restricting CPU, memory, file, and process usage such that individual actions cannot exhaust the machine resources and make other actions running on the same machine fail. If you need to provide additional resources to individual actions, you should setup worker pools and configure those actions to run in a separate pool.
Linux Sandbox Actions¶
Linux-based deployments can alternatively enforce isolation using a sandboxing
tool, see --allow_sandbox.
As of 2020-06-08, the EngFlow Remote Execution Service supports the linux-sandbox
tool developed as part of the Bazel project (included in the EngFlow Debian
package, so you do not need to install the tool yourself).
Client-side, you can disable sandboxing for individual actions, request network access, and set the user id inside the sandbox. See Sandbox platform options for more documentation.
Server-side, the linux-sandbox tool creates a user namespace, a process ID
namespace, a mount namespace, and a network namespace. This allows restricting
the access to the users, processes, file system, and network. See the
--sandbox_* flags for more
details.
Local Actions¶
In all deployments, the least reliable mechanism for action execution is local
action execution, see --allow_local. In this mode, actions are
run as direct subprocesses of the worker service under the same user with
equivalent access to machine resources.
Linux Process Wrapper Actions¶
On Linux, we recommend using the process-wrapper tool for all local actions,
which was developed as part of the Bazel project (included in the EngFlow Debian
package, so you don't need to install the tool yourself), see
--use_process_wrapper. We have updated the process wrapper to
also optionally set CPU affinity, see
--process_wrapper_cpu_limit=set.
The process wrapper primarily enforces termination of all sub-processes started during an action before exiting itself.