Acts on an existing metastore mirror. Today the only supported action is REFRESH, which re-ingests the table at INCREMENTAL (default) or FULL mode.
ALTER METASTORE '<zone.schema.table>' REFRESH [MODE = INCREMENTAL | FULL]## Overview Applies an action to an existing metastore mirror. The REFRESH action re-ingests the table to advance `cache_state.cached_through_version` to the current log head. After the refresh completes, the planner serves point-in-time-correct results AS OF the new version. The ALTER form replaces the legacy `REFRESH METASTORE ...` syntax to align with standard SQL DDL verbs (CREATE / ALTER / DROP). ## When to use Run `ALTER METASTORE ... REFRESH` when the metastore has fallen behind the Delta log and you need queries to see recent commits. Specifically: - **After writes to the Delta table.** Any external commit (Spark, another DeltaForge instance, manual maintenance) advances the log head past `cached_through_version`. Until the next refresh the planner serves an older version of the table. - **On a periodic schedule.** A cron-driven refresh is the simplest way to keep a high-churn table fresh. Frequency is determined by acceptable staleness (every minute for streaming-ingest tables, hourly for ETL-batched tables). - **Before running a critical query that requires up-to-the-second data.** An inline refresh guarantees the next plan reflects the current log head; the trade-off is the latency of the refresh itself. - **During incident response.** Lag (`log_head_version - cached_through_version`) shown by SHOW METASTORE is the primary operator signal that something has stalled. ## Choosing INCREMENTAL vs FULL - **MODE = INCREMENTAL (default)** is the right answer the overwhelming majority of the time. It is cheap, idempotent, and correct. Use it for routine catch-up. - **MODE = FULL** is for the rare cases where the metastore has drifted from the source of truth: - After a destructive operation on the underlying Delta table that bypassed the metastore. - After upgrading to a metastore schema version that requires re-projection. - During development when verifying the hydrator against the file-side truth. - When SHOW METASTORE reports inconsistent state and you want to start clean without DROP + CREATE. ## Behavior - MODE = INCREMENTAL (default): reads only the JSON commits past `cached_through_version`, projects their actions, and opens Parquet footers for newly-added files. Existing rows for files unchanged in the gap are untouched. - MODE = FULL: deletes every per-table catalog row for the table, then re-ingests from version 0. Equivalent in result to DROP METASTORE + CREATE METASTORE but does not drop the registry row or per-table partitions. - One transaction per commit version, so a crash mid-ingest leaves `cached_through_version` at the last fully-projected version (idempotent on retry). ## Access Control | Privilege | Object | Notes | |-----------|--------|-------| | READ | Table | Same requirement as CREATE METASTORE. |
| Name | Type | Description |
|---|---|---|
qualified_name | Canonical 3-part name of a table already mirrored into the metastore (use CREATE METASTORE first). | |
MODE | MODE = INCREMENTAL replays only Delta commits past `cached_through_version`; idempotent and cheap. MODE = FULL wipes the table's catalog rows and re-ingests from version 0; use when the metastore has drifted from the source of truth or after a destructive change. |
-- Catch up to the current log head (typical maintenance, default INCREMENTAL).
ALTER METASTORE 'analytics.sales.orders' REFRESH;-- Explicit incremental.
ALTER METASTORE 'analytics.sales.orders' REFRESH MODE = INCREMENTAL;-- Force a full rebuild (drops prior catalog rows for this table).
ALTER METASTORE 'analytics.sales.orders' REFRESH MODE = FULL;