Skip to content

Bazel First-Time Setup

This document describes how to configure Bazel to connect to an EngFlow Remote Execution cluster.

1. Prepare a Docker container for remote actions

Remote actions are executed in a Docker container to ensure a consistent execution environment. The Docker container must include all system dependencies of actions. For example, a C++ compiler may be required.

If there are no special requirements, Bazel toolchains project provides Ubuntu 16.04 base images.

2. Create a remote configuration

In Bazel, a platform expresses an environment where an action can run. When performing remote execution, Bazel is instructed to build for a special EngFlow platform. This platform carries the Docker image and other environmental configuration to the remote execution service.

Once a Docker container is ready, use the rbe_configs_gen tool to generate Bazel configuration for the image. For example, the following will generate configuration for a Ubuntu 20.04 image into a engflow_config directory:

Bash
1
2
3
4
5
6
7
rbe_configs_gen \
    --toolchain_container=gcr.io/engflow-public/bazel/ubuntu2004-bazel-java11@sha256:c7b874d00b5a5c0cdc9e2d24eea11f3ca689723a42d6c6d088e9e659d389a0f1 \
    --exec_os=linux \
    --target_os=linux \
    --generate_java_configs=false \
    --output_config_path engflow_config \
    --output_src_root .

rbe_configs_gen generates a remote platform target for EngFlow in engflow_config/config/BUILD. It also outputs configuration for the C++ toolchain in the container into engflow_config/cc/.

If you already have a custom C++ compiler crosstool, it's likely easiest to manually write the EngFlow platform rule rather than using rbe_configs_gen. Here is an example engflow_config/config/BUILD for an x86_64 Linux image:

BUILD
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
platform(
    name = "platform",
    constraint_values = [
        "@platforms//os:linux",
        "@platforms//cpu:x86_64",
    ],
    exec_properties = {
        "container-image": "docker://gcr.io/bazel-public/ubuntu2004-java11@sha256:69a78f121230c6d5cbfe2f4af8ce65481aa3f2acaaaf8e899df335f1ac1b35b5",
    },
)

3. Set up .bazelrc.

Configure Bazel to use the EngFlow platform in .bazelrc.

  1. Add common flags:

    .bazelrc
    1
2
3
4
5
6
7
8
    build:engflow --define=EXECUTOR=remote
build:engflow --disk_cache=
build:engflow --experimental_inmemory_dotd_files
build:engflow --experimental_inmemory_jdeps_files
build:engflow --incompatible_strict_action_env=true
build:engflow --remote_timeout=600
build:engflow --nolegacy_important_outputs
build:engflow --grpc_keepalive_time=30s

  2. Add platform flags:

    .bazelrc
    1
2
    build:engflow --host_platform=//engflow_config/config:platform
build:engflow --platforms=//engflow_config/config:platform

  3. If not using a custom C++ toolchain already, add flags for the container's C++ toolchain:

    .bazelrc
    1
2
    build:engflow --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
build:engflow --crosstool_top=//engflow_config/cc:toolchain

  4. If Java is required, optionally use the hermetic JDK shipped with Bazel:

    .bazelrc
    1
    build:engflow --java_runtime_version=remotejdk_11

  5. Add the remote executor flags. For the <CLUSTER_ENDPOINT>, use the host name or IP address of the Load Balancer. You may also have to specify the port, if using a non-standard one. The default ports are 443 for grpc/s, and 80 for http/s.

    If you use TLS with our certificate (engflow-ca.crt):

    .bazelrc
    1
2
3
4
5
    build:engflow --remote_cache=grpcs://<CLUSTER_ENDPOINT>
build:engflow --remote_executor=grpcs://<CLUSTER_ENDPOINT>
build:engflow --bes_backend=grpcs://<CLUSTER_ENDPOINT>
build:engflow --bes_results_url=https://<CLUSTER_ENDPOINT>/invocation/
build:engflow --tls_certificate=<PATH>/engflow-remote-execution/engflow-ca.crt

    If you don't use TLS at all:

    .bazelrc
    1
2
3
4
    build:engflow --remote_cache=grpc://<CLUSTER_ENDPOINT>
build:engflow --remote_executor=grpc://<CLUSTER_ENDPOINT>
build:engflow --bes_backend=grpc://<CLUSTER_ENDPOINT>
build:engflow --bes_results_url=http://<CLUSTER_ENDPOINT>/invocation/

Usage

We recommend using the latest available Bazel version.

Bash
1
bazel build --config=engflow //foo:bar