Reorganizes table data by purging soft-deleted data, forcing a checkpoint, or upgrading for Iceberg compatibility.
REORG TABLE <table> APPLY (<mode>) [WHERE <condition>]## Overview REORG TABLE performs structural reorganization of a Delta table's data files. Unlike OPTIMIZE (which focuses on file compaction and layout), REORG addresses data-level cleanup, metadata maintenance, and format compatibility. ## PURGE Mode When a column is dropped from a Delta table via ALTER TABLE DROP COLUMN, the column data is not physically removed from existing Parquet files. The column is merely hidden by updating the schema in the transaction log. Similarly, rows deleted with deletion vectors are logically removed but remain in the physical files. REORG TABLE APPLY (PURGE) rewrites affected files to: 1. **Remove dropped column data**: Reads each file, excludes the dropped column(s), and writes a new file without that data. 2. **Materialize deletion vectors**: Filters out rows marked as deleted by deletion vectors, producing clean files with only live rows. 3. **Reclaim storage**: The resulting files are smaller because they contain only active schema columns and live rows. After purging, the old files become eligible for removal by VACUUM once the retention period expires. ## CHECKPOINT Mode Delta Lake periodically creates checkpoint files that summarize the transaction log up to a specific version. These checkpoints enable fast table loading by avoiding the need to replay every individual log entry. REORG TABLE APPLY (CHECKPOINT) forces checkpoint creation at the current table version. This is useful when: - The transaction log has accumulated many small commits and loading the table is slow. - You want to create a known-good recovery point before a large batch of operations. - The automatic checkpoint interval (every 10 commits by default) is insufficient for your workload. ## UPGRADE UNIFORM Mode UniForm (Universal Format) allows a Delta table to be read by Iceberg-compatible engines without data duplication. REORG TABLE APPLY (UPGRADE UNIFORM) modifies the table's protocol and metadata to generate Iceberg metadata alongside Delta metadata. - **V1**: Basic Iceberg V1 compatibility (append-only, schema mapping). - **V2**: Iceberg V2 compatibility (row-level deletes, schema evolution, partition evolution). After upgrading, each subsequent Delta commit also produces corresponding Iceberg metadata files. ## Result Set Returns a result set with mode-specific metrics: **PURGE mode:** | metric | value | |--------|-------| | files_rewritten | Number of files rewritten | | bytes_processed | Total bytes processed | | version | New table version | **CHECKPOINT / UPGRADE UNIFORM mode:** | metric | value | |--------|-------| | version | Table version | | changes_made | 1 if changes were applied, 0 otherwise | ## Access Control | Privilege | Object | Notes | |-----------|--------|-------| | Ownership or write | Table | Required for all REORG modes. | ## Compatibility REORG TABLE is a DeltaForge extension implementing Delta Lake table reorganization semantics. The UPGRADE UNIFORM mode implements the UniForm protocol for Iceberg interoperability.
| Name | Type | Description |
|---|---|---|
table | Specifies the name or path of the Delta table to reorganize. The table must be registered in the session. Fully qualified names (zone.schema.table) are supported. | |
mode | Specifies the reorganization mode. Valid values: - **PURGE**: Rewrites data files to permanently remove soft-deleted data. This physically eliminates column data from dropped columns and materializes deletion vectors, producing clean files with no residual deleted content. - **CHECKPOINT**: Forces creation of a Delta checkpoint at the current version. Checkpoints accelerate table loading by consolidating the transaction log into a single Parquet summary file. - **UPGRADE UNIFORM [ICEBERG_COMPAT_VERSION=1|2]**: Upgrades the table's metadata to support Iceberg reads via UniForm. Version 1 provides basic Iceberg V1 compatibility. Version 2 enables Iceberg V2 features including row-level deletes and schema evolution. Shorthand forms V1 and V2 are accepted in place of ICEBERG_COMPAT_VERSION=1 and ICEBERG_COMPAT_VERSION=2. | |
where_clause | Filter files/partitions. |
-- Purge soft-deleted data from the entire table
REORG TABLE orders APPLY (PURGE);-- Purge only a specific partition
REORG TABLE orders WHERE year = 2023 APPLY (PURGE);-- Force a checkpoint at the current version
REORG TABLE orders APPLY (CHECKPOINT);-- Upgrade table for Iceberg V2 compatibility
REORG TABLE orders APPLY (UPGRADE UNIFORM (ICEBERG_COMPAT_VERSION=2));-- Shorthand: upgrade to Iceberg V2
REORG TABLE orders APPLY (UPGRADE UNIFORM V2);-- Purge without parentheses (alternative syntax)
REORG TABLE orders APPLY PURGE;