Skip to content

AWS Private Link

EngFlow AWS clusters can be configured to allow access via AWS Private Link. If you are interested in enabling this feature please reach out to your EngFlow support engineer.

Note there are costs associated with use of AWS Private Link. See https://aws.amazon.com/privatelink/pricing/ and contact your EngFlow support engineer to get a detailed cost estimate.

Benefits

  • "AWS PrivateLink is a highly available, scalable technology that you can use to privately connect your VPC to services as if they were in your VPC." - https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html
  • Once configured for your EngFlow cluster, you can securely connect to the EngFlow cluster without having any traffic flow through the public Internet.
  • Once you have contacted your EngFlow support engineer to use Private Link, they will request the AWS principals (account IDs) associated to the resources (e.g., EC2 instances) that will connect via Private Link.
  • The EngFlow support engineer will configure your cluster and provide:
    • The name of the private endpoint, by default it will be set to be the same as the name of the public endpoint, i.e., <CLUSTER_NAME>.cluster.engflow.com. If you want to use a different name, please let your EngFlow support engineer know which name you want (e.g,private.<CLUSTER_NAME>.cluster.engflow.com), and adjust the instructions below to use the different name of the private endpoint.
    • The Service name of a VPC Endpoint Service used for your Private Link connection. This Service name is of the form com.amazonaws.vpce.<SOME_AZ>.vpce-svc-<RANDOM_HASH>.

Define and configure a VPC endpoint to access the private endpoint

  • Service consumers can access the VPC Endpoint Service for your EngFlow cluster.
    • Service consumers are Bazel clients that must access the EngFlow cluster via the private endpoint.
  • To allow this access, you must set up a VPC endpoint in your consumer account:
    • Go to the project where the service consumer(s) reside (E.g., project that hosts EC2 instances where Bazel clients run), and go to VPC > Endpoints > Create Endpoint.
    • In Endpoint settings > Service Category select PrivateLink Ready partner services.
    • In service settings put the Service name provided by the EngFlow support engineer.
      • Select Verify service to make sure the Service name is correct and authorized.
    • Select the VPC where the service consumer(s) reside.
    • Select the Subnets where the service consumer(s) reside.
    • Select or create security groups associated to the service consumer(s).
  • Wait for the Endpoint to show up with State as Available.
    • If troubleshooting is needed, take note of the DNS names. The main one should be of the form vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com

  • Enable Private DNS names for the VPC Endpoint:

    • Select the VPC Endpoint you created and in Actions select Modify private DNS name.
    • Click on the checkbox in section Enable private DNS names to turn on Enable for this endpoint.

  • Note: you can repeat the steps above and create multiple VPC Endpoints for all the VPCs where service consumer(s) reside.

  • Note: It is possible to configure the VPC endpoint via Terraform. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint for more details.

Configure Bazel to use the private endpoint

The following flag must be set in your .bazelrc file's configuration for remote execution (e.g., build:<CLUSTER_NAME>)

Text Only
1
build:<CLUSTER_NAME> --remote_executor=grpcs://<CLUSTER_NAME>.cluster.engflow.com

I.e., set the value of the remote_executor flag using the name of the private endpoint. If you had already configured this value for the public endpoint, and the name of the private endpoint is the same as the name for the public endpoint, you do not need to make any changes.

Troubleshooting

Verify the endpoint is reachable and private DNS configured

You can run nslookup, from an EC2 instance inside your consumer VPC, using as target the private endpoint name to verify if your instance can access the endpoints:

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
> nslookup <CLUSTER_NAME>.cluster.engflow.com
Server:         10.1.0.2
Address:        10.1.0.2#53

Non-authoritative answer:
Name:   <CLUSTER_NAME>.cluster.engflow.com
Address: 10.1.XX.XXX
Name:   <CLUSTER_NAME>.cluster.engflow.com
Address: 10.1.XX.XXX
Name:   <CLUSTER_NAME>.cluster.engflow.com
Address: 10.1.XX.XXX
Where 10.1.XX.XXX, 10.1.YY.YYY and 10.1.ZZ.ZZZ are the IP addresses of the private DNS configured for your VPC Endpoint

If the output of nslookup <CLUSTER_NAME>.cluster.engflow.com instead looks like the below:

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
> nslookup <CLUSTER_NAME>.cluster.engflow.com
Server:         10.1.0.2
Address:        10.1.0.2#53

Non-authoritative answer:
<CLUSTER_NAME>.cluster.engflow.com        canonical name = tf-lb-<SOME_HASH>.elb.<AVAILABILITY_ZONE>.amazonaws.com.
Name:   tf-lb-<SOME_HASH>.elb.<AVAILABILITY_ZONE>.amazonaws.com
Address: 10.1.XX.XXX
Name:   tf-lb-<SOME_HASH>.elb.<AVAILABILITY_ZONE>.amazonaws.com
Address: 10.1.YY.YYY
Name:   tf-lb-<SOME_HASH>.elb.<AVAILABILITY_ZONE>.amazonaws.com
Address: 10.1.ZZ.ZZZ
Where tf-lb-<SOME_HASH>.elb.<AVAILABILITY_ZONE>.amazonaws.com is the ID of the private load balancer and 10.1.XX.XXX, 10.1.YY.YYY and 10.1.ZZ.ZZZ are its IP addresses within the VPC where the private LB resides.

This response might be indicative that you have not enabled the private DNS names for your VPC endpoint. Make sure this feature is enabled and try again.

Verify the VPC endpoint is reachable

You can also run nslookup, from an EC2 instance inside your consumer VPC, using as target the Service name of the VPC Endpoint Service to verify if your instance can access it correctly:

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
> nslookup vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com
Server:         10.1.0.2
Address:        10.1.0.2#53

Non-authoritative answer:
Name:   vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com
Address: 10.1.XX.XXX
Name:   vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com
Address: 10.1.YY.YYY
Name:   vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com
Address: 10.1.ZZ.ZZZ

Where vpce-<VPC_endpoint_ID_#>-<RANDOM_HASH1>.vpce-svc<RANDOM_HASH2>.<REGION>.vpce.amazonaws.com is the Service name of the VPC Endpoint Service and 10.1.XX.XXX, 10.1.YY.YYY and 10.1.ZZ.ZZZ are its IP addresses within the VPC where the EC2 instance resides. Note these IP addresses must be reachable, i.e., check your Security Group rules and network ACLs do not block access to these IPs.

If the commands above fail, double check the setup for your VPC Endpoint and if everything looks correct, contact your EngFlow support engineer.

If the commands above succeed, but you still get an error running builds, make sure the VPC's Network ACLs or Security groups are not blocking access to the private DNS IPs.

Further reading