Audit logs¶
Beta
This feature is currently in beta. Contact EngFlow if you'd like to use it.
EngFlow audit logs record security-relevant events on your cluster, such as user authentication and access to cached build artifacts. Each log entry captures who performed an action, from where, and what the outcome was.
Audit logs are stored separately from application logs because they contain sensitive information, including user email addresses, IP addresses, and user agent strings.
Logged events¶
Audit log entries are modeled after the Elastic Common Schema (ECS) and cover the following categories of events:
| Category | Value | Events | Volume and Cost |
|---|---|---|---|
| Authentication | auth |
User sign in and sign out on the EngFlow Build and Test UI | Low volume, low cost. Scales with the number of interactive users. |
| IAM | iam |
User and group creation, update, and deletion using the SCIM API; role assignment on groups using the GroupService API |
Low volume, low cost. Scales with identity provider sync frequency. |
| File | file |
Action cache and Content Addressable Storage (CAS) reads, writes, and deletes | Very high volume, very high cost. Scales with build and remote-execution traffic: one entry per cache and CAS RPC. |
| Secret | secret |
Instance secret creation, retrieval, and deletion using the Secret API |
Low volume, low cost. Scales with how often secrets are created, read, or rotated. |
Each entry is a JSON object that includes the event action and outcome, the email and roles of the user who performed the action, the source IP address, and the user agent string.
Excluding categories¶
You can request EngFlow to configure audit logging for your cluster to exclude categories based on requirements and cost constraints.
Recommended: exclude file events
Because file events are logged for every cache and CAS operation, enabling this category logs at roughly the same volume as your build and remote-execution traffic itself. This is comparable to enabling S3 data event logging in AWS CloudTrail (object-level GetObject/PutObject/DeleteObject events), as opposed to the default management-event logging: the additional detail comes at a substantial, traffic-proportional cost in log storage and ingestion. We recommend excluding file unless you specifically need an audit trail of cache and CAS access.
Log destinations¶
EngFlow uses Fluent Bit to ship audit logs from the cluster host to your cloud logging backend.
Audit logs are shipped to Google Cloud Logging. Logs are associated with the gce_instance resource type and grouped under the engflow_logs job label, where you can query them using Log Explorer.
Audit logs are shipped to Amazon CloudWatch Logs. They are sent to a separate audit log group and you can query them using CloudWatch Logs Insights.
Log schema¶
See Audit log schema for the full schema and field reference.
Enabling audit logs¶
Contact EngFlow to enable audit logs for your cluster.