/querying-aws-cloudwatch

name: querying-aws-cloudwatch
description: >-
  Runs SQL queries on CloudWatch Logs data exported as Apache Iceberg tables in S3 Tables.
  Covers VPC Flow Logs, WAF logs, CloudFront access logs, Route 53 resolver logs, Network
  Firewall logs, EKS audit logs, Verified Access logs, SES logs, VPC Lattice logs, Step
  Functions logs, NLB access logs, and 20+ other AWS vended data sources. Applies when
  analyzing network traffic, investigating security incidents, querying exported logs with
  SQL, enabling S3 Tables integration, configuring log export, correlating logs with other
  data, or running Athena queries on the aws-cloudwatch table bucket. Trigger phrases: query
  logs with SQL, analyze logs in Athena, SQL on VPC flow logs, investigate network traffic,
  run SQL on exported logs, enable S3 Tables for CloudWatch, correlate logs, historical log
  analysis, set up log querying.
version: 1
argument-hint: "[query|data-source-name|'configure'|'status']"

Query AWS CloudWatch System Tables

Overview

**Works best with** the [AWS MCP server](https://docs.aws.amazon.com/aws-mcp/) for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials.

The CloudWatch Logs S3 Tables integration exports log data as Apache Iceberg tables in the AWS-managed `aws-cloudwatch` table bucket. This enables SQL analysis via Amazon Athena and correlation of log data with non-CloudWatch data (S3 metadata, business tables, etc.). Available at no additional storage charge beyond CloudWatch ingestion pricing.

Decision Tree

| User intent | Use this skill? | Alternative | |---|---|---| | Run SQL across large volumes of log data | **Yes** | — | | Correlate logs with S3 metadata or other tables | **Yes** — join across catalogs | — | | Quick log search / pattern matching | **No** | CloudWatch Logs Insights (faster for ad-hoc) | | Real-time log streaming/tailing | **No** | CloudWatch Logs console or `logs filter-log-events` | | Set up alarms on log patterns | **No** | CloudWatch Metric Filters / Alarms | | Query historical logs before integration was enabled | **No** | CloudWatch Logs (no backfill in S3 Tables) |

Supported Data Sources

The following data sources are available through the S3 Tables integration. Each data source has a namespace pattern used in SQL queries. Not all AWS vended data sources may be available in all Regions; check the CloudWatch console Data Sources tab for current availability.

| Data Source | Namespace pattern | Common use case | |---|---|---| | VPC Flow Logs | `amazon_vpc__flow` | Network traffic analysis, rejected connections | | WAF Logs | `aws_waf__logs` | Blocked requests, rule hit analysis | | CloudFront Access Logs | `amazon_cloudfront__access` | CDN traffic patterns, error rates | | Route 53 Resolver Query Logs | `amazon_route53resolver__query` | DNS query analysis | | Network Firewall Logs | `aws_networkfirewall__logs` | Firewall rule hits, dropped traffic | | EKS Audit Logs | `amazon_eks__audit` | Kubernetes API audit trail | | Verified Access Logs | `amazon_verifiedaccess__logs` | Zero-trust access decisions | | SES Mail Logs | `amazon_ses__mail` | Email delivery/bounce tracking | | VPC Lattice Access Logs | `amazon_vpclattice__access` | Service-to-service access patterns | | Step Functions Logs | `aws_stepfunctions__logs` | Workflow execution debugging | | Global Accelerator Flow Logs | `aws_globalaccelerator__flow` | Global network traffic | | NLB Access Logs | `elastic_load_balancing__nlb_access` | Load balancer request tracing | | Shield Logs | `aws_shield__logs` | DDoS mitigation events | | Cognito Logs | `amazon_cognito__logs` | Auth/identity operations | | ElastiCache Logs | `amazon_elasticache__logs` | Redis slow log, engine log | | SageMaker Logs | `amazon_sagemaker__logs` | ML training/inference events | | WorkMail Audit Logs | `amazon_workmail__audit` | Email security/compliance | | Bedrock Agent Logs | `aws_bedrock_agent_core__logs` | AI agent invocations | | Client VPN Logs | `aws_client_vpn__connections` | VPN connection tracking | | Entity Resolution Logs | `aws_entity_resolution__logs` | Record matching operations | | MediaPackage Access Logs | `aws_elemental_mediapackage__access` | Streaming delivery metrics | | MediaTailor Logs | `aws_elemental_mediatailor__logs` | Ad insertion events | | Transfer Family Logs | `aws_transfer_family__logs` | SFTP/FTPS file transfer tracking | | Site-to-Site VPN Logs | `aws_site_to_site_vpn__logs` | VPN tunnel diagnostics |

> **Note**: This table lists the 24 most commonly queried data sources. The integration supports 43+ AWS vended data sources in total. Use `list-namespaces` on the `aws-cloudwatch` bucket to discover all available data sources in your account. Namespace patterns follow the convention `<service>__<type>`.

Common Tasks

1. Check If Configured

# Check if the aws-cloudwatch table bucket exists
aws s3tables list-table-buckets --region <REGION> \
  --query "tableBuckets[?name=='aws-cloudwatch']"

• Empty result → integration not enabled. Guide user through setup.
• Bucket exists but no namespaces → integration enabled but no log data yet (only captures events *after* association).

List available tables:

aws s3tables list-namespaces --table-bucket-arn arn:aws:s3tables:<REGION>:<ACCOUNT>:bucket/aws-cloudwatch --region <REGION>

aws s3tables list-tables --table-bucket-arn arn:aws:s3tables:<REGION>:<ACCOUNT>:bucket/aws-cloudwatch --namespace <NAMESPACE> --region <REGION>

2. Enable / Configure

**Create integration:**

aws observabilityadmin create-s3-table-integration \
  --region <REGION> \
  --encryption '{"SseAlgorithm": "aws:kms", "KmsKeyArn": "<KMS_KEY_ARN>"}' \
  --role-arn <SERVICE_ROLE_ARN>

**Associate a specific data source (recommended):**

aws logs associate-source-to-s3-table-integration \
  --region <REGION> \
  --integration-arn <INTEGRATION_ARN> \