Demos

Runnable end-to-end demos across Delta, Iceberg, graph, healthcare, EDI, geospatial, and every supported file format.

Avro

• Avro E-Commerce Orders: Logical Types & Nullable UnionsRead 80 e-commerce orders across two Avro files with mixed codecs, exercising date/timestamp-millis logical types, nullable unions, and integer-cent...
• Avro Insurance Claims: Schema Evolution Across File VersionsRead 90 insurance claims across three Avro files where v2 adds two new fields, validating automatic NULL-filling on v1 rows, file_filter, max_rows...
• Avro IoT Sensors: Smart Building Telemetry at ScaleRead 2,500 sensor readings from 5 building-floor Avro files with schema evolution (v1→v2), mixed codecs, file_filter, max_rows, and...

CSV

• CSV Advanced Options TestbenchEight CSV external tables, each exercising one parsing option (delimiter, null_value, comment_char, skip_starting_rows, max_rows, trim_whitespace...
• Northwind Trading CompanyClassic Northwind sample database as 11 semicolon-delimited CSV external tables (830 orders, 2,155 line items) -- joins, aggregations, shipping...
• Sales Schema EvolutionSingle external table reads five quarterly CSVs (2024-Q1 to 2025-Q1) whose columns evolve over time; missing columns surface as NULL and...
• Veterinary Clinic Patient RecordsThree-branch veterinary clinic with 75 visits across branch-north/south/east subdirectories; demonstrates recursive scanning, file_filter wildcards...

Delta

• Change Data Feed -- Row-Level Change TrackingEnable Change Data Feed to emit per-row insert / update_preimage / update_postimage / delete records so downstream systems can consume only the...
• CHECK Constraints and Table Properties LifecycleDeclare CHECK constraints inside TBLPROPERTIES and prove they survive every UPDATE, DELETE, and re-INSERT -- not just the initial load.
• Delta Basics -- Create, Insert, Update, DeleteWalk through every CRUD operation on a Delta table and watch the transaction log create new versions you can time-travel back to.
• MERGE Patterns -- Slowly Changing Dimension Type 2 (SCD2)Preserve the full audit trail of dimension changes by expiring current rows with MERGE and inserting new versions in a single two-pass pattern.
• OPTIMIZE then VACUUM -- The Delta Maintenance PlaybookFix the small-files problem caused by daily micro-batch ingestion by compacting with OPTIMIZE and then purging orphaned versions with VACUUM, while...
• Partitioning and Dynamic Partition PruningPartition a Delta fact table by region and watch the engine skip entire partition directories when a JOIN or subquery filter on the dimension side...
• Schema Evolution -- Add Columns and Backfill NULLsAdd new columns to a live Delta table with zero data rewrite, rely on implicit NULL fill for historical rows, and backfill only the records that...
• Time Travel and RESTORE -- Inspect Before You RecoverRecover from an accidental bulk UPDATE by time-travelling through versions, picking the last good snapshot, and restoring the table with a single...

EDI

• Advanced JSON Extraction: On-Demand Segment Access via df_transaction_jsonExtracts deeply-nested X12 segments ad hoc using SQL JSON functions, json_typeof, json_array_length, json_extract_path_text, jsonb_pretty, and the...
• EDIFACT Basics: UNB/UNH Envelopes Across DirectoriesParses 22 UN/EDIFACT and EANCOM messages (ORDERS, ORDRSP, INVOIC, CUSCAR, BAPLIE, PAXLST, PNRGOV, DESADV, PRICAT, IFTSTA, APERAK, CONTRL) spanning...
• HIPAA Claims Financial, 837 Charges JOIN 835 RemittanceCharge-to-payment reconciliation across three 837 healthcare claims (professional, dental, institutional) and one 835 remittance advice. Two...
• Order Lifecycle: Cross-Document 850/855/856/810/861 TraceabilityTraces a purchase order through the full order-to-cash cycle: PO creation (850) to acknowledgment (855) to shipment (856/857) to invoice (810) to...
• Repeating Segments & Loops: Indexed / Concatenate / ToJson ModesThree external tables over the same 14 X12 files show how DeltaForge materializes multi-occurrence segments (N1 party loops, PO1 line items)...
• X12 Supply Chain Basics: ISA/GS/ST Envelope + Materialized FieldsParses 14 X12 EDI transactions (850/810/855/856/857/861/997/824) into two external tables, a compact view exposing the ISA/GS/ST envelope and...

Excel

• Excel Multi-Sheet ReportingExpose three sheets (Sales, Returns, Staff) from two regional workbooks as separate external tables and JOIN across them for net-revenue and...
• Excel Options TestbenchValidate each Excel connector option (sheet_name, has_header, skip_rows, max_rows, file_filter) with adversarially-shaped workbooks so misparses...
• Excel Sales Analytics: Superstore OrdersUnify four annual Superstore XLSX files (9,994 orders) into a single external table, then isolate one year with file_filter and run regional +...

FHIR

• FHIR Clinical Observations: Vital Signs & Lab ResultsFlatten HL7 FHIR R5 Observation resources, heart rate NDJSON bulk exports, vital-sign panels, lab results, and blood pressure with nested...
• FHIR Clinical Records: Conditions, Procedures & AllergiesIngest three related HL7 FHIR R5 clinical resource types (Condition, Procedure, AllergyIntolerance) from a single directory, demonstrating...
• FHIR Medications: Prescriptions & CoverageIngest HL7 FHIR R5 MedicationRequest and Coverage resources, including contained Medication definitions, dosage instructions, dispense requests, and...
• FHIR Patient Demographics: BasicsIngest HL7 FHIR R5 Patient resources from bulk NDJSON and individual JSON files, flattening nested name, telecom, and address arrays into queryable...
• FHIR XML Clinical ResourcesIngest native HL7 FHIR R4 XML resources (Patient, Observation), handling the default FHIR namespace, @value-attribute pattern, repeating elements...

Graph

• Global Shipping Routes: Weighted Shortest Path and MSTRoute optimization on 25 world ports and 55 weighted shipping lanes: Dijkstra shortest path, BFS depth layers, minimum spanning tree, and PageRank...
• GPU Banking Fraud Analytics, 10M Accounts, 48M TransactionsGPU-accelerated fraud and influence analytics on a 10M-account banking network with 48M transactions: PageRank, Louvain, betweenness, triangle...
• Hospital Referral Network: Graph DML (INSERT, UPDATE, DELETE)Mutate a live graph via standard DML on its backing Delta tables, verify changes with Cypher, and re-run algorithms on the modified topology.
• Karate Club: Graph Basics, Ingestion, and Verified AlgorithmsLoad Zachary's Karate Club into Delta tables, build an undirected graph, and run Cypher patterns plus 15 algorithms with NetworkX-verified golden...
• Research Collaboration Network: Advanced Cypher PatternsUse advanced Cypher on a university collaboration graph (40 researchers, 170 directed edges, 4 relationship types): negative patterns, aggregations...
• Sales Territory Optimization: SQL + Cypher InteropMix Cypher and SQL in a single query: run graph algorithms via the cypher() table function, persist results into Delta, then JOIN with relational...

HL7

• HL7 Clinical Workflows - Documents, Scheduling & Edge CasesParse MDM clinical documents, SIU appointment bookings, and a parser-torture ADT with all six HL7 escape sequences in a single unified table.
• HL7 Lab Orders & Results - ORM/ORU WorkflowModel the end-to-end lab workflow with HL7 v2 ORM orders and ORU results, including abnormal flags, value types, and the 14-analyte comprehensive...
• HL7 Patient Administration - ADT LifecycleUnify HL7 v2 ADT admit/update/discharge messages from six EHRs across v2.3 through v2.6 into queryable patient tables.

Iceberg

• Iceberg V1: Warehouse Inventory SnapshotsRegister a native Apache Iceberg v1 table (489 retail SKUs across 3 warehouses) and run inventory analytics with zero conversion to Delta.
• Iceberg V2: Fleet Telemetry AnalyticsRead a native Iceberg v2 table (450 fleet GPS pings) and validate enhanced per-column manifest stats with safety, fuel, and speeding analytics, no...
• Iceberg V3: Deletion Vectors (Puffin)Read a native Iceberg v3 table whose 36 faulty-scanner rows are retracted via a Puffin-encoded deletion vector: DeltaForge applies the Roaring...

JSON

• JSON Country Factbook - Deep Nesting & Schema EvolutionExtract deeply nested (4+ level) JSON fields across 10 heterogeneous country files, with exclude_paths and schema-evolution NULL filling.
• JSON Customers - BasicsRead a flat JSON array file into an external table with include_paths, type inference, and file_metadata.
• JSON Music Catalog - Nested Arrays & ExplodeExplode nested track arrays into per-track rows while also producing a per-album summary that keeps the vendor subtree as a JSON blob.
• JSON Subtree Capture - json_pathsCapture complex nested subtrees as serialized JSON string columns with json_paths, and contrast against full flattening.

ORC

• ORC Clinical Trials: NULL Handling & CASE LogicExercise ORC's NULL bitmap on a heavily-nullable patient dataset using COALESCE, NULLIF, CASE, and string predicates.
• ORC Energy Meters: Advanced Aggregation at ScaleAggregate 1,500 smart-meter readings across 3 monthly ORC files using HAVING, COUNT DISTINCT, FILTER clauses, and CTE + window ranking.
• ORC Insurance Claims: Cross-Table JOINs & SubqueriesJoin two ORC-backed external tables (policies, claims) and prove INNER/LEFT/anti-joins, EXISTS, and IN-subquery semantics across ORC data.
• ORC Server Logs: Schema Evolution & Glob FilteringRead 5 HTTP access-log ORC files with two schema versions and prove schema evolution, LOCATION glob filtering, and file_metadata traceability.
• ORC Warehouse Inventory: Window Functions on Mixed NumericsApply ROW_NUMBER, RANK, LAG/LEAD, NTILE, and running totals to 100 rows of warehouse inventory stored across 2 ORC files with int32, float64, and...

Parquet

• Parquet Flight Delays: Airline Delay AnalysisRead three quarterly Parquet files with schema evolution (NULL-filled columns), plus file_filter and file_metadata for per-file drill-down.
• Parquet Supply Chain: Order AnalyticsRecursively scan 14 quarterly Parquet files across year-based subdirectories with file_filter, max_rows, row_group_filter, and file_metadata for...

Protobuf

• Protobuf Address Book - Contact DirectoryRead proto3 AddressBook binaries, flatten nested PhoneNumber messages, decode PhoneType enums, and handle sparse/repeated fields across 3 team files...
• Protobuf Freight Shipping - Multi-Carrier ManifestRead multi-carrier proto3 ShippingManifest binaries to exercise bool/int64/float scalars, 3-level nesting (Shipment -> Package -> Dimensions), two...
• Protobuf IoT Sensor Network - Manufacturing Floor SensorsRead proto3 SensorNetwork binaries across two factory floors and a warehouse (20 sensors, 82 readings) to practice exploding repeated readings...

Pseudonymisation

• Pseudonymisation Apply: Clinical Trial De-identificationPharma de-identification workflow: register query-time rules on a clinical-trial table, verify transforms with SHOW and SELECT, and aggregate...
• Pseudonymisation Healthcare: HL7, FHIR, and EDI De-identificationExercise all seven transform types and all three linkability scopes across three healthcare formats: HL7 v2 ADT, FHIR R4 Patient, EDI HIPAA X12...
• Pseudonymisation Lifecycle: Insurance ClaimsInsurance-claims walkthrough of the full pseudonymisation rule lifecycle: CREATE five rules with PRIORITY, SHOW them, SELECT through them, DROP a...
• Pseudonymisation Quickstart: Banking KYCRegister four query-time pseudonymisation rules on a retail-bank KYC table, redact SSN, mask phone, keyed-hash last name, generalize date of birth...

Spatial

• GIS Emergency Response Network: Multi-Step Spatial AnalyticsNYC dispatch scenario combining st_distance nearest-neighbor ranking, multi-algorithm distance comparison, navigation bearings, and WKT polygon...
• GIS Maritime Shipping: PostGIS-Compatible Geospatial FunctionsExercises all 18 PostGIS-compatible st_* functions against 5 cargo vessels navigating 10 world ports with 40 GPS position reports, distances...
• H3 + GIS Delivery Optimization: Cross-Function Spatial AnalyticsCombines H3 hexagonal indexing with GIS distance/bearing functions across 3 warehouses and 15 stores to rank delivery routes and detect suboptimal...
• H3 GPS Fleet Tracker, 10K Pings, 21 H3 Functions10,000 deterministic GPS pings across 5 world cities, indexed with H3 resolution-9 cells and joined against polyfilled city polygons for O(1)...
• H3 Point-in-Polygon, 1M Row Geofencing PerformanceRide-share geofencing across 12 pricing zones and 1,000,000 driver GPS positions in 8 world cities, converts O(n × m) polygon intersection into O(1)...

Subsurface

• 2D Survey CDP Navigation IndexA processing centre delivers a reprocessed 2D seismic survey one line at a time, and each line is indexed into a curated Delta table so an...
• Aerial Survey Raster CatalogueAn environmental baseline survey delivers orthophoto and elevation tiles for a licence area, and before anything is processed the GIS team has to...
• Depth Surface Handover and Gross Rock VolumeA geophysicist depth-converts a seismic interpretation and hands the result to the mapping team as ZMAP+ grids, who load them to compute gross rock...
• Drilling Survey Anti-CollisionA directional driller hands over the definitive survey at the end of each bit run, as WITSML, and the well engineer loads it to run anti-collision...
• Induced Seismicity MonitoringProduced water from oil and gas production is disposed of by injecting it back underground, and in Oklahoma and the Permian Basin that injection is...
• Licence Divestment Data RoomA buyer's subsurface team has three weeks with the data room for Dutch block P/12 before the bid is due, and the log package in it was written...
• Monthly Production AllocationAn operator files monthly produced volumes per facility with the regulator as PRODML, one document per facility per reporting year, and the...
• North Sea Demo Field, Five Formats JoinedFive upstream energy formats over one field, read in place with no export step, and then joined: a SEG-Y seismic volume (120 traces), two LAS well...
• Offshore Lease FootprintEvery active oil and gas lease on the United States Outer Continental Shelf, as the Bureau of Ocean Energy Management publishes it, loaded against...
• Reservoir Model Handover AuditA partner delivers a static reservoir model for a unitisation study as a RESQML package, and before the asset team loads it into the modelling...
• Reservoir Simulation History Match ReviewA reservoir engineer history matches a field before a development plan goes in. Each iteration is a full simulation run, and the simulator writes a...
• Seismic Crew Field Record QCA land seismic crew drops one folder of SEG-D field records per acquisition day, and a scheduled loader reads each day's drop in place and appends...
• Static Model Deck IngestionA static-model archive receives ECLIPSE GRDECL decks from two places and the simulation team loads both to check the property distributions before...
• Survey Navigation Shot Point DatabaseA survey contractor delivers 2D navigation as UKOOA P1/90, one file per line, and the client builds a shot-point database from it incrementally. The...
• Volve Composite Log IngestionReal Volve DLIS composite logs, ingested incrementally into an analysis-ready Delta table. Two deliveries: LWD composites for 15/9-F-9 (1204 frames)...
• Well Log Library ConsolidationAn operator takes a position in Dutch acreage and receives the regulator's public log package with it: seven wells logged between 1958 and 1990, in...
• Well Log Tape Archive RecoveryAn operator's data-management team brings a LIS-79 well-log tape archive into the lake without a vendor conversion, one digitisation batch at a...

XML

• XML Books: Schema EvolutionUnion 5 yearly XML catalog exports (2000-2004) into one external table, with added/dropped elements becoming NULL in files that lack them.
• XML E-Commerce: Order Line ExplosionExplode 3+ level order XML into per-line-item rows with CDATA descriptions, exclude internal audit blocks, and build a per-order summary with repeat...
• XML NYT News: RSS Feed AnalysisRead 7 regional NYT RSS XML feeds (231 items, 4 namespaces) into a joined-categories article table and an exploded category table for keyword...
• XML Subtree Capture, xml_paths & nested_output_formatCapture complex XML subtrees (specifications, supplier) as single string columns in JSON or raw XML form instead of flattening them into dozens of...