SQL Commands

Every SQL statement DeltaForge understands, from DDL and DML through catalog, security, pipeline, and graph commands.

API Endpoint

• ALTER API ENDPOINTModifies an existing API endpoint definition. Supports merging OPTIONS, changing the URL, changing the response format, and renaming.
• CALL API ENDPOINTRaw preview. Issues one fetch through the same HTTP transport as INVOKE and returns unmodified response bodies as (_page_index INT, _raw_body...
• CREATE API ENDPOINTDefines a single fetch endpoint under a REST API source. The endpoint knows how to fetch and write raw bytes to a storage zone; format and flatten...
• DESCRIBE API ENDPOINTShows the full stored definition of an API endpoint: URL template, response format, resolved option map, pagination strategy, current watermark, and...
• DROP API ENDPOINTRemoves an API endpoint definition and its run-log entries. Landed files in the storage zone are not deleted.
• INVOKE API ENDPOINTProduction fetch, writes response bodies to the connection's storage zone under a timestamped per-run folder. Honors the stored watermark unless...
• SHOW API ENDPOINT RUNSLists invocation history for a single API endpoint, one row per INVOKE, with run status, page/byte counts, watermark progression, and error summary...
• SHOW API ENDPOINTSLists all API endpoint definitions, optionally filtered by owning rest_api connection and/or a LIKE pattern on the endpoint name.

Assertions

• ASSERTFirst-class assertion framework that validates query results against declarative conditions with configurable severity levels, supporting row...
• CONCURRENT BEGIN...ENDExecutes multiple SQL statements concurrently within a single block, providing cooperative parallelism with run-all-report-all semantics and a...

Catalog

• ALTER TAGModifies an existing tag definition in the catalog.
• CREATE SCHEMACreates a new schema namespace within a zone.
• CREATE TAGCreates a tag definition in the catalog for data governance and classification.
• CREATE ZONECreates a new zone, the top-level namespace container in the catalog.
• DESCRIBE TAGShows detailed information about a tag definition and its assignments.
• DETECT SCHEMADiscovers and saves column schema for a registered table.
• DROP FOLDERRemoves an empty directory inside a zone's storage root (local or cloud). Refuses if the folder still has any contents.
• DROP SCHEMADrops a schema (fails if it contains tables).
• DROP SCHEMA COLUMNSRemoves column metadata for a specific table or cleans up orphaned column records.
• DROP SCHEMA HISTORYRemoves schema evolution tracking data for a specific table.
• DROP TAGRemoves a tag definition and all of its assignments from the catalog.
• DROP ZONEDrops a zone (fails if it contains schemas).
• REGISTER DELTA TABLEPersistently registers an existing Delta table under a fully qualified zone.schema.table name. LOCATION must be a relative path under the zone's...
• SET TAGAssigns a tag with a value to a table, view, or column. Columns use 4-part naming: zone.schema.table.column.
• SHOW GRAPHSLists graph definitions registered in the catalog.
• SHOW SCHEMASLists schemas in the catalog, optionally filtered by zone or name pattern.
• SHOW SESSION TABLESLists all tables registered in the current compute node's session context.
• SHOW TABLESLists tables in the catalog, optionally filtered by schema or name pattern.
• SHOW TAGSLists all tag definitions in the catalog, optionally filtered by name pattern.
• SHOW ZONESLists all zones in the catalog, optionally filtered by name pattern.
• SYNC CATALOGRefreshes the compute node's permission token and catalog metadata from the control plane.
• UNREGISTER TABLERemoves a table's catalog entry without deleting the underlying data files.
• UNSET TAGRemoves a tag assignment from a table, view, or column.

Configuration

• ALTER CONNECTIONModifies an existing connection: replaces options, swaps credential references, or renames.
• ALTER CREDENTIAL STORAGEModifies one or more attributes of an existing credential-storage backend: description, config options, active state, or name.
• ALTER EXTERNAL LOCATIONUpdates an existing external location's URL, bound credential, options, owner, or comment.
• ALTER STORAGE CREDENTIALUpdates an existing storage credential's options, owner, or comment. Replaces options atomically with the new set.
• ALTER VAULT / ALTER CREDENTIALModifies a vault entry: rotates encryption-key material, replaces a credential secret, updates metadata, or renames. VAULT and CREDENTIAL are...
• CREATE CONNECTIONRegisters an external-system connection (database, object store, git host, SFTP, or REST API) used by schema discovery and federated queries.
• CREATE CREDENTIAL STORAGERegisters a cloud key-vault-style credential-storage backend (Azure Key Vault, AWS Secrets Manager, or GCP Secret Manager) used to persist vault...
• CREATE EXTERNAL LOCATIONRegisters a storage URL bound to a storage credential as a managed external location.
• CREATE STORAGE CREDENTIALCreates a named storage credential that supplies authentication material to external locations.
• CREATE VAULT / CREATE CREDENTIALCreates a vault entry, either an encryption key (material generated by DeltaForge) or a credential (caller-supplied secret or pointer to an external...
• DESCRIBE CONNECTIONEmits metadata for a single connection. Sensitive option values are masked server-side.
• DESCRIBE CREDENTIAL STORAGEEmits metadata for a single credential-storage backend. Secret config material is never included.
• DESCRIBE EXTERNAL LOCATIONReturns the URL, credential binding, and metadata for an external location.
• DESCRIBE STORAGE CREDENTIALReturns metadata about a storage credential. Secret material is never returned.
• DESCRIBE VAULT / DESCRIBE CREDENTIALEmits metadata for a single vault entry. Secret material is never included in the output.
• DROP CONNECTIONRemoves a connection registration. External tables registered through this connection must be dropped or relocated first.
• DROP CREDENTIAL STORAGERemoves a credential-storage backend. Fails if any vault entries still reference it unless those entries are linked-external (metadata-only) pointers.
• DROP EXTERNAL LOCATIONRemoves an external location registration. The underlying storage and any data it contains are not touched.
• DROP STORAGE CREDENTIALRemoves a storage credential. Grants on the credential are also removed; external locations bound to the credential are NOT removed and lose their...
• DROP VAULT / DROP CREDENTIALRemoves a vault entry and wipes its backend-stored material. VAULT and CREDENTIAL are keyword aliases.
• SHOW CONNECTIONSLists all registered external connections, optionally filtered by a LIKE substring.
• SHOW CREDENTIAL STORAGESLists all registered credential-storage backends, optionally filtered by a LIKE pattern on display_name.
• SHOW EXTERNAL LOCATIONSLists all external locations, optionally filtered by a LIKE pattern.
• SHOW STORAGE CREDENTIALSLists all storage credentials, optionally filtered by a LIKE pattern.
• SHOW VAULTS / SHOW CREDENTIALSLists vault entries, optionally filtered by entry kind and a LIKE substring on display_name. VAULTS and CREDENTIALS are keyword aliases.
• TEST CONNECTIONProbes connectivity to the external system referenced by a connection, returning success/latency/error and server version where available.
• TEST CREDENTIAL STORAGERuns the reachability probe on a credential-storage backend (the same probe executed by the GUI 'Test' button).

Data Loading

• COPY INTOLoads data from external files into a Delta table, with format inference, glob filtering, and idempotent file tracking.
• CREATE EXTERNAL TABLECreates an external table that references data files at a storage location, enabling SQL queries without copying data into a managed Delta table.
• DROP EXTERNAL TABLERemoves an external table definition from the session and catalog, optionally deleting the underlying data files from storage.
• OPEN DELTA TABLEOpens an existing Delta table from a storage location and makes it queryable under a session-scoped alias. Delta-only by design; non-Delta formats...

DDL

• ALTER TABLE ADD COLUMNAdds a single column to an existing Delta table's schema.
• ALTER TABLE ADD COLUMNSAdds multiple columns to an existing Delta table in a single atomic operation.
• ALTER TABLE ADD CONSTRAINTAdds a CHECK, NOT NULL, PRIMARY KEY, or UNIQUE constraint to an existing Delta table.
• ALTER TABLE ALTER COLUMN DROP DEFAULTRemoves the default value expression from a column. INSERTs that omit the column afterward receive NULL (or fail if the column is NOT NULL).
• ALTER TABLE ALTER COLUMN DROP NOT NULLRemoves the NOT NULL constraint from a column, allowing null values.
• ALTER TABLE ALTER COLUMN SET DEFAULTSets a default value expression for a column. The expression is stored verbatim in the column's metadata and applied to subsequent INSERTs that omit...
• ALTER TABLE ALTER COLUMN SET METADATASets or updates JSON metadata on a column for type widening tracking, custom annotations, or feature-specific hints.
• ALTER TABLE ALTER COLUMN SET NOT NULLSets a column to NOT NULL, disallowing null values in future writes.
• ALTER TABLE ALTER COLUMN TYPEChanges the data type of an existing column via type widening. Requires the delta.enableTypeWidening table property.
• ALTER TABLE CHANGE COLUMNMoves an existing column to a new position in the table schema without rewriting data files.
• ALTER TABLE CLUSTER BYConfigures or disables liquid clustering on a Delta table.
• ALTER TABLE DROP COLUMNDrops a column from a Delta table's schema using column mapping. The physical data remains in Parquet files.
• ALTER TABLE DROP CONSTRAINTDrops a named CHECK or NOT NULL constraint from a Delta table.
• ALTER TABLE RENAME COLUMNRenames an existing column in a Delta table using column mapping. The physical column name in Parquet files remains unchanged.
• ALTER TABLE RENAME TORenames a table in the catalog and DataFusion session without modifying underlying Delta files.
• ALTER TABLE REPLACE COLUMNSReplaces the entire column schema of a Delta table with a new set of columns in a single atomic operation.
• ALTER TABLE SET TBLPROPERTIESSets one or more table properties on an existing Delta table, used to enable features such as column mapping, auto-optimize, type widening, and...
• ALTER TABLE UNSET TBLPROPERTIESRemoves one or more table properties from an existing Delta table.
• CREATE DELTA TABLECreates a new Delta table with an explicit schema, optional partitioning, liquid clustering, table properties, and an optional LOCATION. Supports...
• CREATE VIEWCreates or replaces a named view backed by a SQL query.
• DROP DELTA TABLERemoves a Delta table from the session and catalog. Optionally deletes the underlying physical data files.
• DROP FEATURERemoves a Delta table feature (e.g., deletion vectors, change data feed, type widening) and optionally truncates history to allow protocol downgrade.
• DROP VIEWRemoves one or more views from the catalog and DataFusion session.

DML

• DELETEDeletes rows from a Delta table matching an optional predicate, recording file-level remove and add actions in the transaction log.
• INSERT INTOInserts new rows into a Delta table from a VALUES clause or SELECT query, with support for append, overwrite, and partition-level overwrite modes.
• MERGE INTOPerforms an atomic upsert by combining INSERT, UPDATE, and DELETE actions in a single transaction, joining a source dataset against a target Delta...
• PARALLEL CREATE TABLE ASCreates a Delta table from a SELECT query by splitting the source on a numeric key column and running multiple writers concurrently.
• PARALLEL INSERT INTOInserts SELECT results into an existing Delta table by splitting the source on a numeric key column and running multiple writers concurrently.
• TRUNCATE TABLERemoves all rows from a Delta table while preserving the table schema, properties, and transaction history.
• UPDATEUpdates column values in rows of a Delta table that match an optional predicate, rewriting affected data files and recording the changes in the...

Graph

• CREATE GRAPHCreates a named property graph definition that maps a vertex table and an edge table from the Delta lakehouse into a queryable graph structure.
• CREATE GRAPHCSRPre-builds and writes the Compressed Sparse Row (CSR) topology for a named graph to disk, enabling subsequent Cypher queries to load the graph in...
• CREATE GRAPHGPUPre-builds and persists Vulkan-derived GPU buffers (w_out, in_edge_weights) for a named graph as a compact .dgpu sidecar co-located with the .dcsr...
• DROP GRAPHDrops a named graph definition from the catalog and session registries without deleting the underlying vertex or edge Delta tables.
• DROP GRAPH CONFIGDrops per-table graph configuration entries from the session registry and catalog, removing the column mappings that define how tables participate...
• SHOW GRAPHDisplays the full configuration of one or all named graph definitions, including vertex/edge table mappings, column assignments, directionality...

Incremental

• GET INCREMENTAL FILTERQueries a target Delta table for maximum column values and generates a WHERE clause for incremental data loading.
• SET INCREMENTAL CONFIGPersists incremental loading configuration as Delta table properties for use by INCREMENTAL FILTER.
• SHOW INCREMENTAL CONFIGDisplays the stored incremental loading configuration for a Delta table.
• UNSET INCREMENTAL CONFIGRemoves all incremental loading configuration from a Delta table's properties.

Indexing

• ALTER INDEXUpdates a row-level index's mutable properties such as the auto-update flag and algorithm-specific options.
• CREATE BLOOM FILTER INDEXCreates bloom filter indexes on specified columns for efficient file-level data skipping.
• CREATE INDEXCreates a row-level sorted index (PGM or B+ tree) over a Delta table to accelerate equality and range lookups on the indexed prefix.
• DESCRIBE INDEXESLists all row-level indexes on a table along with their status, version binding, algorithm, leaf count, and on-disk size.
• DROP BLOOM FILTER INDEXRemoves bloom filter indexes from a Delta table.
• DROP INDEXDrops a row-level index from a Delta table. The child index Delta table is tombstoned at the current version.
• REBUILD INDEXRebuilds a row-level index from scratch at the parent table's current version, replacing the stale index data with a fresh full build.

Maintenance

• CLONECreates a copy of a Delta table, either as a lightweight metadata reference (shallow) or a full independent copy (deep).
• CONVERT TO DELTAConverts an existing Parquet or Iceberg directory to Delta Lake format in place by writing a fresh _delta_log alongside the existing files. No data...
• DESCRIBE AUTO OPTIMIZEShows the current auto-optimization configuration and status for a Delta table.
• DESCRIBE MAINTENANCE STATUSShows health metrics, circuit breaker state, and maintenance scheduler status for a table.
• GENERATE MANIFESTGenerates a symlink format manifest file listing all active data files in a Delta table for use by external query engines.
• INCREMENTAL VACUUMPerforms resumable, batch-based vacuum for very large tables.
• OPTIMIZECompacts small files into larger files and optionally applies Z-ordering for improved read performance.
• REORG TABLEReorganizes table data by purging soft-deleted data, forcing a checkpoint, or upgrading for Iceberg compatibility.
• RESET CIRCUIT BREAKERManually resets the circuit breaker for a table's maintenance operations, re-enabling operations that were automatically suspended after repeated...
• RESTORE TO TIMESTAMPRestores a Delta table to the state it had at a specific point in time.
• RESTORE TO VERSIONRestores a Delta table to the state it had at a specific version number.
• SET AUTO OPTIMIZEEnables or disables automatic post-write compaction and optimized writes for a Delta table.
• VACUUMRemoves old, unreferenced data files from a Delta table to reclaim storage.
• VACUUM LITEPerforms a lightweight vacuum using only the transaction log, without scanning storage.
• ZORDERApplies Z-order data layout optimization to a Delta table for efficient multi-dimensional data skipping.

Metastore

• ALTER METASTOREActs on an existing metastore mirror. Today the only supported action is REFRESH, which re-ingests the table at INCREMENTAL (default) or FULL mode.
• CREATE METASTOREMirrors a Delta table's `_delta_log/` action stream and full Parquet footer state into the Postgres-backed metastore so the query planner can serve...
• DROP METASTOREDrops a table from the metastore: removes its registry row and every per-table partition (`parquet_file`, `row_group`, `column_chunk`, typed stats...
• SHOW METASTORELists every Delta table currently registered with the metastore together with its hydration state (`cached_through_version`, `log_head_version`...

Pipeline

• DROP PIPELINERemoves a pipeline declaration from the current session's pipeline registry.
• DROP SCHEDULERemoves a schedule declaration from the current session's schedule registry.
• INCLUDE SCRIPTInlines a script's SQL at the current location for reuse across pipelines.
• PIPELINEDeclares a pipeline with metadata, scheduling, and governance attributes.
• SCHEDULEDeclares a named schedule with cron timing, retry policies, concurrency limits, and notification settings for pipeline orchestration.
• SET $paramSets a script-scoped parameter. The expression is captured as raw SQL text and re-evaluated each time the parameter is referenced.
• SHOW PIPELINESLists pipelines declared in the current session, optionally filtered by a LIKE pattern.
• SHOW SCHEDULESLists schedules declared in the current session, optionally filtered by a LIKE pattern.

Pseudonymisation

• ALTER PSEUDONYMISATION RULEEnables or disables a pseudonymisation rule, or adds and removes principals from its exempt list.
• APPLY PSEUDONYMISATIONPermanently transforms existing data (destructive -- cannot be undone).
• CREATE PSEUDONYMISATION RULECreates a runtime data transformation rule for GDPR compliance.
• DROP PSEUDONYMISATION RULERemoves a pseudonymisation rule from a table.
• SHOW PSEUDONYMISATION RULESLists active pseudonymisation rules.

Security

• ALTER GROUP ADD USERAdds one or more users to a group.
• ALTER GROUP REMOVE USERRemoves one or more users from a group.
• ALTER ROLEUpdates an existing role's parent or comment.
• ALTER USERUpdates an existing user's default role or comment.
• CREATE GROUPCreates a group for collective role assignment.
• CREATE ROLECreates a role that bundles privileges and can inherit from a parent role.
• CREATE SERVICE PRINCIPALCreates a service principal: a non-human identity for automation, jobs, integrations, and pipelines.
• CREATE USERCreates a user identity in the catalog. Authentication material is provisioned separately by the identity provider; this command registers the...
• CREATE WORKSPACECreates a workspace boundary for isolating user assets, scripts, and pipelines.
• DROP GROUPRemoves a group. The Control Plane cascades the removal: memberships are deleted and direct grants on the group are revoked.
• DROP ROLERemoves a role. The Control Plane cascades the removal: role grants and direct object grants for this role are revoked.
• DROP SERVICE PRINCIPALRemoves a service principal. The Control Plane cascades the removal: role grants and direct object grants targeting the service principal are revoked.
• DROP USERRemoves a user identity. The Control Plane cascades the removal: role grants and direct object grants targeting the user are revoked.
• DROP WORKSPACERemoves a workspace. The Control Plane cascades the removal across the workspace's grants and metadata.
• GRANT ON EXTERNAL LOCATIONGrants one or more privileges on an external location to a principal. Supports WITH GRANT OPTION for delegated administration.
• GRANT ON SCHEMAGrants a single privilege on a schema to a user or role.
• GRANT ON STORAGE CREDENTIALGrants one or more privileges on a storage credential to a principal. Supports WITH GRANT OPTION.
• GRANT ON TABLEGrants a single privilege on a table to a user or role.
• GRANT ON ZONEGrants a single privilege on a zone to a user or role.
• GRANT ROLEGrants a role to a user, group, or service principal. Idempotent: re-granting an existing assignment succeeds without error.
• REVOKE ON EXTERNAL LOCATIONRevokes one or more privileges on an external location from a principal.
• REVOKE ON SCHEMARevokes a single privilege on a schema from a user or role.
• REVOKE ON STORAGE CREDENTIALRevokes one or more privileges on a storage credential from a principal.
• REVOKE ON TABLERevokes a single privilege on a table from a user or role.
• REVOKE ON ZONERevokes a single privilege on a zone from a user or role.
• REVOKE ROLERevokes a role from a user, group, or service principal. Idempotent: revoking a non-existent assignment succeeds without error.
• SHOW GRANTS ON EXTERNAL LOCATIONLists every principal that holds a privilege on a given external location.
• SHOW GRANTS ON STORAGE CREDENTIALLists every principal that holds a privilege on a given storage credential.
• SHOW GRANTS TO PRINCIPALLists every grant assigned to a principal across object types: roles, zones, schemas, tables, external locations, and storage credentials.
• SHOW GROUPSLists groups in the catalog, optionally filtered by a LIKE pattern.
• SHOW ROLESLists roles in the catalog, optionally filtered by a LIKE pattern.
• SHOW SERVICE PRINCIPALSLists service principals in the catalog, optionally filtered by a LIKE pattern.
• SHOW USERSLists users in the catalog, optionally filtered by a LIKE pattern.
• SHOW WORKSPACESLists workspaces in the catalog, optionally filtered by a LIKE pattern.

Select Grammar

• Aggregate FunctionsFunctions that compute a single result from a set of rows.
• Common Table Expressions (WITH)Named temporary result sets that exist for the scope of a single query.
• GROUP BYGroups rows by column values for aggregate calculations.
• HAVINGFilters groups after aggregation (like WHERE for GROUP BY).
• JOIN TypesCombines rows from two or more tables based on a related column.
• LATERAL SubqueryA subquery in FROM that can reference columns from preceding tables.
• LIMIT / OFFSETPagination -- limits result count and skips rows.
• ORDER BYSorts the result set by one or more columns.
• SELECTRetrieves data from tables with filtering, grouping, and ordering.
• SubqueriesNested queries used in FROM, WHERE, or SELECT clauses.
• UNION / INTERSECT / EXCEPTCombines result sets from multiple queries.
• Window FunctionsCompute values across a set of rows related to the current row.

Statistics

• ANALYZE TABLEComputes histogram statistics and optional bloom filters for table columns to improve query optimization.
• DESCRIBE STATISTICSDisplays computed histogram statistics metadata for a table, including histogram methods, bin counts, and column coverage.
• DROP STATISTICSRemoves computed histogram statistics for a Delta table.

System

• EXECPLANInspects a SQL query without executing it. Each EXECPLAN mode returns a single focused, CLI-friendly slice of plan information; the syntax mirrors...
• FSCK REPAIR TABLEDetects and repairs metadata inconsistencies by removing transaction log entries that reference data files no longer present in the underlying...
• HELPDisplays documentation for DeltaForge SQL commands, including command listings, descriptions, syntax details, and usage examples.
• SHOW STATSRuns a SQL statement under instrumentation and returns a per-metric breakdown of what the planner intended (PLAN view) or what the runtime actually...

Table Info

• DESCRIBE CHECKPOINTLists transaction-log checkpoints for a Delta table, or returns detailed metadata for a single checkpoint version. Reports checkpoint format (V1...
• DESCRIBE DETAILReturns a comprehensive set of metadata properties for a Delta table, including format, location, partitioning, file counts, size, protocol...
• DESCRIBE HISTORYReturns the version history of a Delta table, showing each commit's version number, timestamp, operation type, operation metrics, file statistics...
• DESCRIBE TABLEReturns the schema of a Delta table, listing each column's name, data type, nullability, partition status, and optional comment.
• SHOW TABLE PROPERTIESReturns the configuration properties (tblproperties) stored in a Delta table's metadata, displayed as key-value pairs.

Utility

• COMPARE SCHEMACompares two schemas (from DDL strings or existing tables) and returns a detailed diff with generated ALTER TABLE statements for compatible changes...
• CREATE CHARTRenders an inline source query as an SVG chart and returns it in the execution result for the GUI to display.
• EXTRACT LINEAGEParses a SQL script and extracts table-level data lineage, returning inbound (read) and outbound (write) dependencies with catalog, schema, table...
• INFERTYPEExecutes a query or scans a table, then runs the type inference engine on the result columns to detect semantic data types, report confidence...
• SCRIPT TABLE AS CREATEGenerates a complete CREATE DELTA TABLE DDL script from an existing table's metadata, including column definitions, constraints, partitioning...
• SQLFORMATFormats a SQL string according to a configurable style preset, with optional overrides for indentation, keyword casing, comma placement, and line...