Postgres-backed mirror of the Delta log and Parquet footer state. Tables registered here serve planner statistics from indexed SQL instead of replaying the log per query.
## Overview The Meta Store page is the operator surface for the `df_metastore` Postgres schema, a relational mirror of every Delta action and every Parquet footer field the query planner needs for `statistics()` and `scan()`. Once a table is registered and hydrated, the planner can answer pruning, file listing, and column statistics questions with indexed SQL lookups instead of replaying the `_delta_log/` directory and reading Parquet footers on every query. This page covers the writer side of that mirror: which tables are registered, how current their cache is, and the controls to register, refresh, or unregister them. The reader side activates independently: a Delta table only routes through the mirror when its own properties carry `delta.forge.metastore.enabled = true`. That keeps opt-in explicit on a per-table basis. The metastore does not mutate the underlying Delta table. Registration is purely a row in `table_registry`, with the storage path resolved through the control plane at register and refresh time. The three-part qualified name (`zone.schema.table`) is the only identifier the metastore stores. ## Key Features - **Registry of mirrored tables.** Each row shows the fully qualified table name, the cached commit version (`cached_through_version`), the timestamp of the last hydration, and an optional description. An empty list signals that no table has been registered yet; the inline empty state explains the next step. - **Register a Delta table.** Enter a three-part qualified name and click Register. The page issues `CREATE METASTORE <qname>` through the standard query path. The metastore resolves the storage path from the control plane and performs a full initial hydration of the Delta log and Parquet footers. - **Incremental refresh.** The Refresh action replays only the commits added since the cached version. This is the everyday operation: cheap when the table is up to date, proportional to commit count otherwise. - **Full refresh.** The Full action discards the cached state and re-replays every commit from version 0. Use it after a manual repair or when the cache must be rebuilt from scratch. - **Unregister.** Drops the per-qname partitions and the `table_registry` row. The underlying Delta files are untouched; you can re-register at any time and rehydrate from the log. - **Point-in-time guarantee.** `cached_through_version` is the exact Delta version the metastore authoritatively serves. A reader either gets correct results as of that version, or transparently falls back to the log when its target version is past the cache. The mirror is never wrong, only behind by some number of versions. ## Workflow 1. Open the Meta Store page from the Catalog sidebar. 2. Review the registry table to see which tables already have a Postgres-backed mirror and how current each one is. 3. To add a table, type its three-part qualified name (`zone.schema.table`) into the Register field and press Enter or click Register. The initial hydration runs synchronously and the new row appears in the registry on completion. 4. For an existing table, click Refresh to bring the cache forward incrementally, or Full to rebuild from version 0. 5. To stop mirroring a table, click the trash icon and confirm. The underlying Delta files are not affected. 6. To activate reader-side use of the cache, set `delta.forge.metastore.enabled = true` on the Delta table itself via `ALTER TABLE ... SET TBLPROPERTIES` in the Query Explorer. The planner picks up the property automatically. ## Operational Notes - The metastore SQL commands require the executor to have been built with a configured `metastore_pool`. If the pool is not configured, the registry list returns a `not_configured` error and the inline banner explains the situation. - All four actions go through the same `delta_execute_query` path as the Query Explorer, so role permissions on `catalog.metastore.register`, `catalog.metastore.refresh`, and `catalog.metastore.unregister` apply uniformly. - Hydration is a manual operation in this version. There is no continuous tailer; commits added after a refresh remain invisible to the mirror until the next refresh.