AWS IAM Compromise Investigation

I set out to find and undo unauthorized IAM changes on a deployment user, remove the excess admin access and the rogue credential, and add event-driven monitoring for high-risk IAM changes.

The IAM user app-deployer showed two signs of unauthorized change: the AWS-managed AdministratorAccess policy was attached to it, and it had a recently created access key that nobody could account for. My plan was to investigate the user and its credentials, remove the privileges, deactivate and delete the key, then add detection and test it with a controlled event.

1. 01
   Identified the rogue key

   I stored the account ID and found the most recently created access key on app-deployer. The key value is never written down.

2. 02
   Removed admin access

   I detached AdministratorAccess and confirmed the user had no attached policies left.

3. 03
   Disabled, then deleted the key

   I set the key to inactive first, deleted it, and confirmed the user had no access keys left.

4. 04
   Set up notifications

   I created an SNS topic, iam-change-alerts, with an email subscription and a topic policy that lets EventBridge publish to it.

5. 05
   Built the detection rule

   I created the EventBridge rule iam-high-risk-changes to match 13 high-risk IAM API calls recorded by CloudTrail, and pointed it at the SNS topic.

6. 06
   Tested it with a controlled event

   I created a test access key to trigger a CreateAccessKey event, then deleted the test key straight away.

In production I would investigate an unauthorized change before rolling it back, to tell a legitimate admin action from a malicious or accidental one.

# Find the newest access key on the user (value never printed)
ROGUE_KEY_ID=$(aws iam list-access-keys --user-name app-deployer \
  --query "sort_by(AccessKeyMetadata, &CreateDate)[-1].AccessKeyId" --output text)

# Remove admin access, then disable and delete the key
aws iam detach-user-policy --user-name app-deployer \
  --policy-arn arn:aws:iam::aws:policy/AdministratorAccess
aws iam update-access-key --user-name app-deployer --access-key-id $ROGUE_KEY_ID --status Inactive
aws iam delete-access-key --user-name app-deployer --access-key-id $ROGUE_KEY_ID

# Detection: EventBridge rule to SNS
aws events put-rule --name iam-high-risk-changes \
  --event-pattern file://iam-event-pattern.json --state ENABLED
aws events put-targets --rule iam-high-risk-changes --targets "Id=sns-iam-alerts,Arn=$SNS_ARN"

{
  "source": ["aws.iam"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["iam.amazonaws.com"],
    "eventName": [
      "AttachUserPolicy", "AttachRolePolicy", "AttachGroupPolicy",
      "PutUserPolicy", "PutRolePolicy", "PutGroupPolicy",
      "CreateUser", "CreateAccessKey", "CreateLoginProfile",
      "UpdateAssumeRolePolicy", "DeactivateMFADevice",
      "DeleteUser", "DeleteAccessKey"
    ]
  }
}

After the changes, the user had no attached policies and no access keys. The rule target was added with FailedEntryCount: 0, and the controlled CreateAccessKey call matched the rule's pattern.

1. 01Do not give application identities AdministratorAccess without a documented reason.
2. 02Alert on high-risk IAM changes: policy attachments, access key creation, trust policy edits and MFA changes.
3. 03Review access keys regularly and remove unused or unexpected ones.
4. 04Prefer roles and temporary credentials for workloads over long-lived user keys.
5. 05Investigate before reverting suspicious changes.
6. 06Test detections with controlled events instead of assuming a rule works.