/migrating-hive-tables-to-s3-tables

name: migrating-hive-tables-to-s3-tables
description: "Migrates a Glue Data Catalog Hive table (Parquet) into Amazon S3 Tables as managed Iceberg using the native S3 Tables migration API (StartTableMigration, SyncTableMigration, CompleteTableMigration), a managed service call, not a self-run CTAS, Glue, or EMR job. The source is a Hive table; an Iceberg source is never in scope, even when the destination is S3 Tables. Covers plan review, migrate, sync, complete, and validate. Same-account, same-region, Glue Hive Parquet source only. Use when the request asks to migrate or import a Hive table into S3 Tables, or names the native migration API. Not applicable to converting a self-managed or standard Iceberg table into S3 Tables (ingesting-into-data-lake covers that), self-run CTAS/Glue-ETL/Spark migrations (ingesting-into-data-lake), or creating empty tables (creating-data-lake-table)."
version: 1

Migrating Hive Tables to S3 Tables

Contents

• [Overview](#overview)
• [Before you migrate](#before-you-migrate)
• [When to use](#when-to-use)
• [When NOT to use (use another skill)](#when-not-to-use-use-another-skill)
• [Scope and constraints](#scope-and-constraints)
• [The 6 migration APIs](#the-6-migration-apis)
• [Migration lifecycle](#migration-lifecycle-phases)
• [Pre-migration guardrails](#pre-migration-guardrails-run-every-time-before-start)
• [Post-migration guardrails](#post-migration-guardrails-verify-success-before-complete)
• [Key API shapes](#key-api-shapes)
• [Security considerations](#security-considerations)
• [References](#references)

Overview

This skill guides an agent through migrating one Glue Hive Parquet table into an S3 table bucket with the native S3 Tables migration API. Amazon S3 Tables plans the migration, reports column and object compatibility, copies the data, and keeps the destination in sync with the source until you complete it. Your coding agent, following this skill, runs preflight checks, starts the migration, presents the plan for approval, monitors progress, completes the migration, and validates the result.

When the AWS MCP Server is available, use it to load this skill and its references (`aws___retrieve_skill`, with `file="references/<name>.md"`) and to run the AWS calls.

Before you migrate

**Agent instruction:** You MUST present the migration plan and obtain explicit user approval before any step that changes tables, data, metadata, or catalog entries, and before any IAM, KMS, or S3 bucket change (creating or editing a role or policy, editing a key policy, creating a report bucket). Show the exact JSON or command first. For a key policy, read it with `get-key-policy`, add one statement, and never replace the whole policy. The plan MUST list:

• the source table and the destination table bucket, namespace, and table name;
• the write paths that will change (ETL jobs, streaming or ingest, `INSERT`/`MERGE`, CDC);
• the query engines and downstream consumers that read the table;
• the datasets, saved queries, and permissions that reference the old Glue table or S3 prefix.

Up-front approval in the user's request (naming the source, destination, and intent to migrate) covers only read-only checks and a Start with plan review. Approving the plan, accepting any `setNull` or `drop` policy, Complete, and any delete always need explicit approval during the conversation. If the actual state differs from what was approved (a different schema, partition layout, row count, or resource), stop, explain the difference, and get approval again.

Confirm these constraints with the user as part of the plan:

• Every writer to the destination writes through an Iceberg-aware engine (for example Athena,

EMR or Glue with Iceberg). Writers that write Parquet files directly to S3 will not work.

• Every reader supports Iceberg and the S3 Tables catalog.
• Jobs and queries move from the Glue Hive table to the S3 Tables catalog reference.
• The source is not changed. Keep writers on the source until the destination is validated.
• Validate row counts and sample values ([references/validation.md](references/validation.md))

before moving production traffic.

See [Migrating tables to S3 Tables](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-migrate.html) for the full list of supported sources and limits.

> **Runtime:** works with or without the AWS MCP server; steps use AWS CLI syntax. The AWS MCP > server is recommended for sandboxed execution and audit logging. The migration commands are > part of `aws s3tables`; if an older CLI lacks them, see > [references/api-setup.md](references/api-setup.md).

When to use

Use this skill when the source is a Glue Data Catalog Hive table with Parquet data (`TableType=EXTERNAL_TABLE`, Hive Parquet SerDe) and the destination is an S3 table bucket in the same account and Region:

• Migrating that table into S3 Tables as managed Iceberg with the native migration API.
• Keeping the migrated table in sync with the active Hive source before cutover.

When NOT to use (use another skill)

The names below are other agent skills. For these requests, do not use this skill; load the named skill instead. When asked when not to use this skill, list every case:

• A self-run CTAS, Glue ETL, or EMR/Spark migration into Iceberg: use the `ingesting-into-data-lake` skill (this skill covers only the managed migration API).
• Converting a self-managed or standard Iceberg table: use the `ingesting-into-data-lake` skill.
• Loading raw files (Parquet with no Glue table, CSV, JSON) or other sources (JDBC, Snowflake, BigQuery, DynamoDB): use the `ingesting-into-data-lake` skill.
• Creating a new empty Iceberg table: use the `creating-data-lake-table` skill.
• Running or managing Athena queries: use the `querying-data-lake` skill.
• Locating tables or assets: use the `finding-data-lake-assets` skill. Auditing the catalog: use the `exploring-data-catalog` skill.
• Setting up Glue connections: use the `connecting-to-data-sou