Build analytics table schema

Build analytics captures action execution data from the EngFlow profile and stores it as rows in Parquet files. The schema is documented below.

Field Name	Data Type	Required	Description	REAPI v2/source
instance_name	String	YES	Tenant that the invocation and/or executions were submitted under.	instance_name (present in most Request messages)
invocation_id	String	YES	Unique identifier for an invocation.	RequestMetadata.tool_invocation_id
action_digest	String	YES	Digest of the action being executed.	ActiveActionMetadata#getActionDigest
result_digest	String	YES	Digest of the result of the action execution.	CompactDigests.hash(ExecuteResponse.result)
tool_name	String	YES	Name of the tool, e.g. Bazel.	RequestMetadata.tool_details.tool_name
tool_version	String		Version of the tool used for the request, e.g. 5.0.3.	RequestMetadata.tool_details.tool_version
action_id	String		Identifier for the action.	RequestMetadata.action_id
target_id	String		Identifier for the execution target.	RequestMetadata.target_id
action_mnemonic	String		Mnemonic representing the action.	RequestMetadata.action_mnemonic
configuration_id	String		An identifier for the configuration in which the target was built.	RequestMetadata.configuration_id
start_timestamp	Timestamp	YES	When the worker started executing the action.	ExecutedActionMetadata.worker_start_timestamp
end_timestamp	Timestamp		When the worker completed the action, including all stages.	ExecutedActionMetadata.worker_completed_timestamp
queued_timestamp	Timestamp		When the action was added to the queue.	ExecutedActionMetadata.queued_timestamp
input_fetch_start_timestamp	Timestamp		When the worker started fetching action inputs.	ExecutedActionMetadata.input_fetch_start_timestamp
input_fetch_end_timestamp	Timestamp		When the worker finished fetching action inputs.	ExecutedActionMetadata.input_fetch_completed_timestamp
execution_start_timestamp	Timestamp		When the worker started executing the action command. This value will be null if there was a cache hit.	ExecutedActionMetadata.execution_start_timestamp
execution_end_timestamp	Timestamp		When the worker completed executing the action command. This value will be null if there was a cache hit.	ExecutedActionMetadata.execution_completed_timestamp
output_upload_start_timestamp	Timestamp		When the worker started uploading action outputs.	ExecutedActionMetadata.output_upload_start_timestamp
output_upload_end_timestamp	Timestamp		When the worker finished uploading action outputs.	ExecutedActionMetadata.output_upload_completed_timestamp
exit_code	Integer		The exit code of the command.	ActionResult.exit_code
status	String		A text status describing an execution error. Empty in case of success.	ExecuteReponse.status
runner	String			if ExecuteResponse.cached_result: remote cache hit else: remote
cache_hit	Boolean		True if the result was served from cache, false if it was executed.	ExecuteResponse.cached_result
remotable	Boolean		Whether the action was allowed to run remotely.	Always true
remote_cacheable	Boolean		Whether the action was allowed to be cached remotely.	Action.do_not_cache
pool_id	String		Uniquely identifies the cluster pool this action ran on.
pool_id_requested	String		The pool explicitly requested by the client, before alias resolution.
pool_id_alias	String		The synthetic pool alias used for pool family routing. Absent when pool family routing was not used.
pool_id_resolution	String		How the pool was resolved: explicit, mnemonic_override, pool_family_alias, or weighted_pool.
pool_id_weighted_target	String		The target pool after weighted routing. Absent when weighted routing was not applied.
pool_id_weighted_weight	Integer		The configured weight (0-100) at the time of weighted routing. Absent when weighted routing was not applied.
affinity_key	String		Used to schedule an action on an executor that has run similar actions.
executor_id	String		Uniquely identifies the executor this action ran on.	WorkerUnit#toString()
instance_type	String		What kind of machine this action ran on, for example c6i.8xlarge. Absent when the executor does not run on a cloud instance.
resources	Struct		Representation of the rusage structure returned by getrusage(2) with some additional information.
resources.ram_limit	String
resources.rusage_utime_ms	Long		User CPU time used, in milliseconds
resources.rusage_stime_ms	Long		System CPU time used, in milliseconds
resources.rusage_maxrss	Long		Maximum resident set size
resources.rusage_ixrss	Long		Integral shared memory size
resources.rusage_idrss	Long		Integral unshared data size
resources.rusage_isrss	Long		Integral unshared stack size
resources.rusage_minflt	Long		Page reclaims (soft page faults)
resources.rusage_majflt	Long		Page faults (hard page faults)
resources.rusage_nswap	Long		Swaps
resources.rusage_inblock	Long		Block input operations
resources.rusage_oublock	Long		Block output operations
resources.rusage_msgsnd	Long		IPC messages sent
resources.rusage_msgrcv	Long		IPC messages received
resources.rusage_nsignals	Long		Signals received
resources.rusage_nvcsw	Long		Voluntary context switches
resources.rusage_nivcsw	Long		Involuntary context switches
resources.tags	Map		User-provided key values via Bazel remote headers. For example: --remote_header=x-engflow-resource-tag=my_key=my_value. Limitations: Maximum of 5 tags per action. Keys must be 1-128 characters, start with a lowercase letter, and contain only lowercase letters, digits, underscores, and dashes. Values must be 1-128 characters and contain only letters, digits, and the characters !#$%*+.?@^_~-.
resources.tags.key	String		Key in the resource tag key-value pair. Must match the pattern [a-z][a-z0-9_-]{0,127}.
resources.tags.value	String		Value in the resource tag key-value pair. Must match the pattern [A-Za-z0-9!#$%*+.?@^_~-]{1,128}.
psi	Struct		Linux Pressure Stall Information, in milliseconds. Note that the Linux kernel reports PSI in microseconds. For more information, see: https://www.kernel.org/doc/html/v6.1/accounting/psi.html
psi.cpu_some	LongType		CPU some stall time, in milliseconds
psi.cpu_full	LongType		CPU full stall time, in milliseconds
psi.memory_some	LongType		Memory some stall time, in milliseconds
psi.memory_full	LongType		Memory full stall time, in milliseconds
psi.io_some	LongType		IO some stall time, in milliseconds
psi.io_full	LongType		IO full stall time, in milliseconds
input_tree_stats	Struct
input_tree_stats.max_depth	Integer
input_tree_stats.total_dirs	Integer
input_tree_stats.total_files	Integer
input_tree_stats.total_metadata_size	Long
input_tree_stats.total_file_size	Long
input_tree_stats.distinct_digest_total_size	Long
cas_read_stats	Struct
cas_read_stats.local_count	Integer
cas_read_stats.distributed_count	Integer
cas_read_stats.external_storage_count	Integer
cas_read_stats.distributed_longest_read	Long
cas_read_stats.external_storage_longest_read	Long
cas_read_stats.distributed_mbps	Double
cas_read_stats.external_storage_mbps	Double
output_stats	Struct
output_stats.files_uploaded	Integer
output_stats.bytes_uploaded	Long
platform	Struct
platform.cpu_arch	String
platform.os_family	String
correlated_invocations_id	String		Identifier that ties multiple invocations (identified by their invocation_id strings) together.	RequestMetadata.correlated_invocations_id
requester	String		Identifies the person or entity that requested the invocation.
invocation_metadata	Struct		Metadata about the invocation's CI and SCM context.	N/A. invocation_metadata information comes from BES keywords.
invocation_metadata.ci	Struct		CI system metadata.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.ci.pipeline_name	String		Name of the CI pipeline.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.ci.job_name	String		Name of the CI job.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.ci.uri	String		URI of the CI job.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.scm	Struct		Source control metadata.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.scm.repository	String		Repository URL or identifier.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.scm.branch	String		Branch name.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.scm.commit_id	String		Commit identifier.	N/A. invocation_metadata.* information comes from BES keywords.
invocation_metadata.scm.commit_modified	Boolean		Whether the commit was modified (e.g. uncommitted changes).	N/A. invocation_metadata.* information comes from BES keywords.
scm_runner	String
historical_action_id	String		When this execution is coalesced into another action's result, this value is set to the execution_id of the action that actually ran, otherwise this value is equal to this action's execution_id.
execution_id	String		Unique identifier for an execution as generated by remote execution.	google.longrunning.Operation#name from Execution/Execute
oom_kills	Long		Number of times the kernel OOM killer killed a process in the action's container. An OOM-killed action usually also reports exit_code 137. Absent unless the action ran in Docker on Linux.