Table: aws_cloudtrail_log - Query AWS CloudTrail logs
The aws_cloudtrail_log table allows you to query data from AWS CloudTrail logs. This table provides detailed information about API calls made within your AWS account, including the event name, source IP address, user identity, and more.
Configure
Create a partition for aws_cloudtrail_log (examples):
vi ~/.tailpipe/config/aws.tpc
connection "aws" "logging_account" { profile = "my-logging-account"}
partition "aws_cloudtrail_log" "my_logs" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "aws-cloudtrail-logs-bucket" }}
Collect
Collect logs for all aws_cloudtrail_log partitions:
tailpipe collect aws_cloudtrail_log
Or for a single partition:
tailpipe collect aws_cloudtrail_log.my_logs
Query
Explore 100+ example queries for this table →
Root activity
Find any actions taken by the root user.
select event_time, event_name, source_ip_address, user_agent, aws_region, recipient_account_id as account_idfrom aws_cloudtrail_logwhere user_identity.type = 'Root'order by event_time desc;
Top 10 events
List the top 10 events and how many times they were called.
select event_source, event_name, count(*) as event_countfrom aws_cloudtrail_loggroup by event_source, event_name,order by event_count desclimit 10;
High volume S3 access requests
Find users generating a high volume of S3 access requests to identify potential anomalous activity.
select user_identity.arn as user_arn, count(*) as event_count, date_trunc('minute', event_time) as event_minutefrom aws_cloudtrail_logwhere event_source = 's3.amazonaws.com' and event_name in ('GetObject', 'ListBucket')group by user_identity.arn, event_minutehaving count(*) > 100order by event_count desc;
Example Configurations
Collect logs from an S3 bucket
Collect CloudTrail logs stored in an S3 bucket that use the default log file name format.
connection "aws" "logging_account" { profile = "my-logging-account"}
partition "aws_cloudtrail_log" "my_logs" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "aws-cloudtrail-logs-bucket" }}
Collect logs from an S3 bucket with a prefix
Collect CloudTrail logs stored in an S3 bucket using a prefix.
partition "aws_cloudtrail_log" "my_logs_prefix" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "aws-cloudtrail-logs-bucket" prefix = "my/prefix/" }}
Collect logs from local files
You can also collect CloudTrail logs from local files, like the flaws.cloud public dataset.
partition "aws_cloudtrail_log" "local_logs" { source "file" { paths = ["/Users/myuser/cloudtrail_logs"] file_layout = "%{DATA}.json.gz" }}
Exclude read-only events
Use the filter argument in your partition to exclude read-only events and reduce the size of local log storage.
partition "aws_cloudtrail_log" "my_logs_write" { # Avoid saving read-only events, which can drastically reduce local log size filter = "not read_only"
source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "aws-cloudtrail-logs-bucket" }}
Collect logs for all accounts in an organization
For a specific organization, collect logs for all accounts and regions.
partition "aws_cloudtrail_log" "my_logs_org" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "cloudtrail-s3-log-bucket" file_layout = "AWSLogs/o-aa111bb222/%{NUMBER:account_id}/CloudTrail/%{DATA:region}/%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}/%{DATA}.json.gz" }}
Collect logs for a single account
For a specific account, collect logs for all regions.
partition "aws_cloudtrail_log" "my_logs_account" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "cloudtrail-s3-log-bucket" file_layout = "AWSLogs/(%{DATA:org_id}/)?123456789012/CloudTrail/%{DATA:region}/%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}/%{DATA}.json.gz" }}
Collect logs for a single region
For all accounts, collect logs from us-east-1.
partition "aws_cloudtrail_log" "my_logs_region" { source "aws_s3_bucket" { connection = connection.aws.logging_account bucket = "cloudtrail-s3-log-bucket" file_layout = "AWSLogs/(%{DATA:org_id}/)?%{NUMBER:account_id}/CloudTrail/us-east-1/%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}/%{DATA}.json.gz" }}
Collect logs for multiple regions
For all accounts, collect logs from us-east-1 and us-east-2.
partition "aws_cloudtrail_log" "my_logs_regions" { source "aws_s3_bucket" { bucket = "cloudtrail-s3-log-bucket" file_layout = "AWSLogs/(%{DATA:org_id}/)?%{NUMBER:account_id}/CloudTrail/(us-east-1|us-east-2)/%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}/%{DATA}.json.gz" }}
Source Defaults
aws_s3_bucket
This table sets the following defaults for the aws_s3_bucket source:
Argument | Default |
---|---|
file_layout | AWSLogs/(%{DATA:org_id}/)?%{NUMBER:account_id}/CloudTrail/%{DATA:region}/%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}/%{DATA}.json.gz |
Mods
This table can be used with pre-built dashboards and detections in the following mods:
Name | Description |
---|---|
Run detections and view dashboards for your AWS CloudTrail logs to monitor and analyze activity across your AWS accounts using Powerpipe and Tailpipe. |