/aws-observability

CloudTrail Operational Auditing

Using CloudTrail for operational debugging: who changed what, when. Not for security threat detection.

Contents

• Event types
• Event history
• Common operational queries
• Querying CloudTrail logs
• CloudTrail → CloudWatch integration

Event types

Event history

• 90 days of management events retained by default, no trail required
• Searchable in console by event name, resource type, user name, time range
• 200,000 event limit when downloading
• Single account, single Region only
• Cannot view data events, Insights events, or network activity events

Common lookups

# Who deleted an S3 bucket?
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=DeleteBucket \
  --start-time 2026-04-20T00:00:00Z

# Who modified a security group?
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=AuthorizeSecurityGroupIngress

# Who stopped an EC2 instance?
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=ResourceName,AttributeValue=i-1234567890abcdef0

Common operational queries

"Who deleted my resource?"

1. Check Event History (90 days) for Delete* events
2. Filter by resource name or resource type
3. Look at userIdentity.arn for the actor and sourceIPAddress for origin

"Who changed this configuration?"

1. Search for Update*, Modify*, Put* events on the resource
2. Compare requestParameters across events to see what changed

"What happened during the incident?"

1. Filter by time range of the incident
2. Look for errorCode fields (AccessDenied, ThrottlingException)
3. Correlate with CloudWatch metrics/logs for the same time window

"Who accessed my data?" (requires data events)

Data events must be explicitly enabled on the trail.

put-event-selectors REPLACES the trail's entire selector set — it does not append. Sending only the selector below to a trail that already has selectors silently drops the existing ones, which can turn off data-event capture the customer relies on for audit or security. Read first, merge, then write:

aws cloudtrail get-event-selectors --trail-name my-trail
# add your selector to the returned AdvancedEventSelectors[] list, keep the rest, then put the
# COMPLETE merged list back

Check which selector style the trail uses first. get-event-selectors returns either AdvancedEventSelectors or the basic EventSelectors, and put-event-selectors accepts only one of the two. If the trail returned basic EventSelectors, writing --advanced-event-selectors converts the trail and discards every basic selector — the same silent loss this callout is about. In that case translate the existing basic selectors into advanced form as part of the merge, and confirm the conversion with the customer before writing.

Confirm the merged set with the customer before writing. Note also that data events are billed per event and are typically far higher volume than management events, so say what the selector will match before enabling it.

# The list below must be the COMPLETE set of selectors the trail should end up with,
# i.e. existing selectors from get-event-selectors PLUS this new one.
aws cloudtrail put-event-selectors --trail-name my-trail \
  --advanced-event-selectors '[{
    "Name": "S3DataEvents",
    "FieldSelectors": [
      {"Field": "eventCategory", "Equals": ["Data"]},
      {"Field": "resources.type", "Equals": ["AWS::S3::Object"]}
    ]
  }]'

Querying CloudTrail logs

Recommended: Trail → S3 → Athena

For new setups, deliver CloudTrail logs to S3 and query with Amazon Athena:

SELECT eventTime, userIdentity.arn, sourceIPAddress, eventName
FROM cloudtrail_logs
WHERE eventName = 'DeleteBucket'
  AND eventTime > '2026-04-20'
ORDER BY eventTime DESC
LIMIT 100;

This is the long-term supported approach — works with standard SQL, scales to any volume, and integrates with existing S3-based analytics.

CloudTrail → CloudWatch integration

Alert on specific API calls

CloudTrail → Trail → CloudWatch Logs → Metric Filter → CloudWatch Alarm → SNS

1. Configure trail to deliver events to a CloudWatch Logs log group
2. Create metric filter for the event pattern (e.g., { $.eventName = "DeleteBucket" })
3. Create alarm on the metric filter
4. Configure SNS notification

Event selectors

• Basic: simple include/exclude for management and data events
• Advanced: fine-grained filtering by event source, resource type, resource ARN
• Exclude high-volume management event sources on trails: AWS KMS, RDS Data API
• Max 250 data resources across all basic event selectors per trail (does not apply to advanced event selectors)