Create a Bazel C++ toolchain configuration for remote actions¶
To construct C/C++ commands that can run remotely, Bazel needs a C++ toolchain configuration.
For local builds, Bazel configures this automatically based on toolchains installed on your host system. This may not be desirable: different developers may have different toolchains installed.
For remote builds, you need to configure a toolchain explicitly because Bazel doesn't know what's installed in the remote environment.
This page explains how to configure a Bazel C++ toolchain configuration for remote execution. You must have already created a container image with your C++ toolchain installed. You'll also need a host machine with Docker installed with the same OS and architecture as the remote execution environment.
Generating configuration with
rbe_configs_gen is a tool that can automatically configure a toolchain for you, given a Docker image tag. Visit that page and follow the instructions there to build and run it.
rbe_configs_gen works for Linux only. You may need to install additional tools such as Java in your container image, or pass flags like
rbe_configs_gen to work.
Generating configuration with Bazel¶
You can generate your C++ toolchain configuration with Bazel by building a C++ target locally in a container, then extracting the automatic configuration. This works more reliably and has fewer restrictions than
rbe_configs_gen, but it's a more manual process.
Before you begin, you'll need a Docker tag for your container image. We'll use
engflow-container-image in the instructions here. If it's in a remote registry, you must be able to
docker pull from that registry (
docker login first).
To build your configuration:
Run a new container based on your image. The next few steps should be performed inside the container.
Create a Bazel C++ workspace, then build the target. You may need to set the
CCenvironment variable if Bazel doesn't find your compiler automatically.
local_config_ccrepository, and copy the files. They are short enough that you can read them, then copy from your terminal.
On the host machine, create a directory in the repository where you want to use remote execution. The directory name is up to you, but its name should reflect the remote environment. If you use remote execution on multiple platforms, you'll probably want multiple directories. In our example repository, we have
platform/windows_x64. Paste the
local_config_ccfiles into that new directory.
BUILDfile, keeping only the toolchains you need, discarding others. You may combine it with an existing
BUILDfile, perhaps containing other toolchains and a
platformdefinition. In particular, keep the following targets:
toolchain: this is the top-level definition for the C++ toolchains. It has a
cc_toolchainlabels. Keep only the entries for
cc_toolchaintargets you need. The
--crosstool_topflag will refer to this target.
cc-compiler-k8: this contains high-level information for an individual toolchain. There may be multiple toolchains, and the name may be different, depending on your CPU architecture (
x86_64). You can drop the
armeabi-v7atarget; this is for Android only.
local: this describes your toolchain in detail. It lists paths of commands, paths of include dirctories, and flags to pass. You can modify this if you want to change which flags are passed automatically.
empty: used in places where the toolchain doesn't depend on anything inside the Bazel workspace. You can replace
toolchaintarget to the
BUILDfile. This is part of Bazel's toolchain system, and adding this will help Bazel select your toolchain automatically. Make sure the
toolchainattribute refers to a
In your project's
.bazelrcfile, ensure that your toolchain is registered when you are using your remote configuration. Ensure that
--host_crosstool_toprefer to your
cc_toolchain_suitetarget, and that
--host_compilerrefer to the CPU and compiler for
Verify your new toolchain configuration works by building the same C++ target using your new remote configuration from a compatible host environment. You may need to configure a
platformand add additional flags to
.bazelrcto enable remote execution, if you haven't done so already. Refer to Bazel First-Time Setup.