/investigating-aws-incidents

name: investigating-aws-incidents
description: Investigate security incidents in Amazon Web Services -- reconstruct attacker activity from CloudTrail, VPC Flow Logs, and GuardDuty, anchor the investigation on the compromised principal (access key or role), trace privilege escalation and persistence through IAM API calls, detect data exfiltration and crypto-mining, and contain without destroying evidence or tipping off the attacker. Use when responding to a suspected AWS compromise, exposed access keys, anomalous CloudTrail activity, a GuardDuty finding, unexpected IAM changes, crypto-mining EC2 instances, or S3 data exfiltration.
verified: 2026-07-26

Investigating AWS Incidents

In AWS an incident is reconstructed from logs the attacker usually could not delete -- CloudTrail, VPC Flow Logs, and the control plane's own record of every API call. The investigation is therefore a log-correlation exercise anchored on the compromised principal: the access key or assumed role, and the sequence of API calls it made. Find the principal, pull every event it generated, and the rest of the intrusion falls out of the timeline.

When to Use

• Exposed or leaked AWS access keys -- committed to a repo, in a pastebin, or flagged by AWS abuse
• Anomalous CloudTrail activity -- API calls from unfamiliar regions, IPs, or user agents
• A GuardDuty finding -- credential exfiltration, tor traffic, crypto-mining, or anomalous IAM behavior
• Unexpected IAM changes -- new users, keys, roles, login profiles, or policy attachments nobody authorized
• Crypto-mining EC2 -- a spike in on-demand instances, GPU families, or an unexpected billing alert
• S3 data exfiltration -- GetObject spikes, buckets made public, or shared snapshots

When NOT to Use

• **The incident is in Microsoft 365 / Entra ID, not AWS** -- use `investigating-m365-entra`
• **The incident is in Azure resources/subscriptions** -- use `investigating-azure-incidents`
• **The incident is in Google Cloud** -- use `investigating-gcp-incidents`
• **The general IR process and host-level response** -- use `responding-to-incidents`
• **You are the attacker, not the responder** -- use `exploiting-cloud-platforms`
• **The pivot is specifically into Kubernetes on EKS** -- use `attacking-eks-gke-aks`
• **Proactive hunting with no confirmed incident** -- use `hunting-threats`

First-Hour Triage

Three actions, in order: identify the principal, pull its recent activity, preserve before you contain.

**Scope the compromised principal.** Whether the report is a leaked key, a GuardDuty finding, or a billing spike, resolve it to a single principal ARN -- an IAM user, a role, or the root account. That ARN is the anchor for everything that follows.

# What is this key/session, and does it still work?
aws sts get-caller-identity                       # if you hold the suspect creds
# Resolve an access key ID to its owner
aws iam get-access-key-last-used --access-key-id AKIA...
# Enumerate the principal's current footprint
aws iam list-access-keys --user-name <user>
aws iam list-attached-user-policies --user-name <user>
aws iam list-user-policies --user-name <user>

**Pull recent activity from the principal.** Event history (below) is the fastest first look; the S3 log bucket is the source of truth.

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=Username,AttributeValue=<user> \
  --start-time 2026-07-01T00:00:00Z --max-results 200

# By access key -- catches role sessions that lookup-by-username misses
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIA...

**Isolate without tipping off -- snapshot then contain.** An attacker who sees their key deactivated mid-operation will burn persistence you have not found yet. For anything but active, ongoing damage: preserve evidence first (snapshot volumes, export logs), map persistence, then contain everything at once. If there is live damage -- active mining, active exfil -- stop the damage and accept the trade.

CloudTrail Deep-Dive

**Event history vs. the S3 log bucket.** The console Event history is searchable but covers only 90 days of management events and drops data events. The CloudTrail S3 log bucket (or CloudTrail Lake) is the authoritative record and the only place with S3/Lambda data events -- query it, not the console, once you are past the first look. Confirm what is actually logged:

aws cloudtrail describe-trails
aws cloudtrail get-trail-status --name <trail>          # IsLogging: true?
aws cloudtrail get-event-selectors --trail-name <trail> # data events on?

**`userIdentity` types** tell you what you are looking at:

| `type` | Meaning | Investigative note | | --- | --- | --- | | `IAMUser` | Long-lived user credentials | `accessKeyId` is the pivot. Human or service account? | | `AssumedRole` | Temporary STS creds | Read `sessionContext.sessionIssuer` for the source role. | | `AWSService` | An AWS service acting | Usually benign, but check for spoofed-looking service calls. | | `Root` | Root account | Almost never legitimate for API calls. Treat as critical. | | `FederatedUser` / `WebIdentityUser` | `GetFederationToken` broker / OIDC web identity | Trace to the broker's IAM user, or the IdP session. |

For `AssumedRole`, `sessionContext.sessionIssuer.arn` names the role and `sessionContext.attributes.mfaAuthenticated` tells you whether MFA was used.

**Anomaly fields to grep:**

• `sourceIPAddress` -- external IPs on role credentials that should only run

inside the VPC (the IMDS theft signature, below). Correlate against VPC Flow Logs.

• `userAgent` -- `aws-cli/2.x`, `Boto3`, `python-requests`, or anything with

`kali` on a principal that normally shows console or SDK-from-Lambda agents.

• `awsRegion` -- calls in regions you do not operate in (attackers spin up

mining in `ap-*` / `sa-*` to dodge attention). Enumeration touches many regions fast.

• `errorCode` -- a storm of `AccessD