Skip to content

Action Environment

In order to successfully build a project, each action needs to run in an environment that matches its requirements. For example, an action running the C++ compiler gcc may require a Linux environment with an appropriate version of gcc installed. At the same time, actions must be isolated from each other to prevent build breakages, non-deterministic outputs, and cache poisoning.

This document outlines the facilities provided by the EngFlow Remote Execution service to configure the action's runtime environment and isolating actions from each other.

Action isolation (also known as sandboxing) prevents actions from interfering with each other and with the underlying system. This avoids build breakages that might otherwise occur and makes the build more reliable. Restricting CPU usage also decreases action time variance, making the behavior of multi-threaded actions more predictable. These properties are especially important for release builds.

The EngFlow Remote Execution Service supports multiple levels of isolation depending on the host system (Linux, MacOS), and the deployment architecture.

On Linux, we support Docker, sandbox, and local execution.

On MacOS, we support local execution only due to restrictions of the underlying platform.

You have to enable at least one execution mechanism to be able to run actions.

Linux Docker Actions

On Linux, we recommend using Docker execution and configuring it to isolate actions, see --allow_docker. This mode provides the flexibility of running different actions with different Docker images, different network settings, and different machine resources (such as CPU and memory).

This mode is only available on Linux, and only if the worker service can subprocess to Docker. More specifically, it is currently only available in VM- and bare-metal deployments, but not in Kubernetes-based deployments where the worker service is already running inside a Docker container.

Client-side, you can request different Docker images for individual actions, enable or disable system capabilities, enable networking, and allow sibling Docker containers. See Docker platform options for more documentation.

Server-side, we recommend restricting CPU, memory, file, and process usage such that individual actions cannot exhaust the machine resources and make other actions running on the same machine fail. If you need to provide additional resources to individual actions, you should setup worker pools and configure those actions to run in a separate pool.

Linux Sandbox Actions

Linux-based deployments can alternatively enforce isolation using a sandboxing tool, see --allow_sandbox.

As of 2020-06-08, the EngFlow Remote Execution Service supports the linux-sandbox tool developed as part of the Bazel project (included in the EngFlow Debian package, so you do not need to install the tool yourself).

Client-side, you can disable sandboxing for individual actions, request network access, and set the user id inside the sandbox. See Sandbox platform options for more documentation.

Server-side, the linux-sandbox tool creates a user namespace, a process ID namespace, a mount namespace, and a network namespace. This allows restricting the access to the users, processes, file system, and network. See the --sandbox_* flags for more details.

Local Actions

In all deployments, the least reliable mechanism for action execution is local action execution, see --allow_local. In this mode, actions are run as direct subprocesses of the worker service under the same user with equivalent access to machine resources.

Linux Process Wrapper Actions

On Linux, we recommend using the process-wrapper tool for all local actions, which was developed as part of the Bazel project (included in the EngFlow Debian package, so you don't need to install the tool yourself), see --use_process_wrapper. We have updated the process wrapper to also optionally set CPU affinity, see --process_wrapper_cpu_limit=set.

The process wrapper primarily enforces termination of all sub-processes started during an action before exiting itself.