An operator's data-management team brings a LIS-79 well-log tape archive into the lake without a vendor conversion, one digitisation batch at a time. The 11 March batch is the real Volve 15/9-F-4 composite as OSDU redistributes it: 12 curves, 21505 frames, wrapped in Tape Image Format. The 12 March batch is a 26-curve reprocessed composite whose specification record does not fit in one 1024-byte physical record and so arrives split with the successor and predecessor bits set. Three things have to be right for the frame arithmetic to close with nothing left over: the tape framing is stripped, the curve list is found in logical record type 64 rather than 34, and the split specification is rejoined. The real tape has no null convention, so a curve the tool was not running reads as a number, and 1114 of its density-correction readings are below minus one gram per cubic centimetre with the worst at minus 57338. The loader applies one physical validity rule per curve and keys on the source file, so re-running a batch is a no-op. Recovers the well's casing programme from the bit-size curve and finds 1096 net pay frames between 2772 and 3423 m. See ATTRIBUTION.md.
-- ============================================================================
-- LIS Tape Archive Recovery - Setup Script
-- ============================================================================
-- An operator's data-management team is bringing a well-log tape archive into
-- the lake. The tapes are LIS-79, the format that preceded DLIS, and they are
-- wrapped in Tape Image Format: twelve bytes of tape framing around every
-- physical record. Nothing modern opens them without a vendor conversion
-- step, which is exactly what the team is trying to avoid paying for a second
-- time.
--
-- The archive is digitised in batches, and the loader runs per batch:
--
-- 11 March The Volve 15/9-F-4 composite. 12 curves, 21505 frames.
-- 12 March A reprocessed composite for 15/9-F-4 A. 26 curves, 640 frames.
--
-- The first tape is real. It is Equinor's Volve 15/9-F-4 composite log as the
-- OSDU Forum redistributes it, byte for byte; see ATTRIBUTION.md in the
-- parent folder. Its awkwardness is real too, and the load has to deal with
-- it: the tape has no null convention, so a curve the tool was not recording
-- reads as a number indistinguishable from zero, and one of its density
-- correction readings is minus fifty-seven thousand grams per cubic
-- centimetre.
--
-- The second tape is written by generate_data.py, because no open corpus
-- publishes a second LIS tape. It carries the shape the real one does not: a
-- 26-curve specification is 1049 bytes against the 1024-byte physical record
-- cap, so it arrives split across two physical records with the successor and
-- predecessor bits set.
--
-- 1. volve_composite external, DISCOVER over the 11 March batch
-- 2. reprocessed_composite external, DISCOVER over the 12 March batch
-- 3. tape_archive DELTA, the curated target
-- ============================================================================
-- ----------------------------------------------------------------------------
-- STEP 1: Zone and schema
-- ----------------------------------------------------------------------------
CREATE ZONE IF NOT EXISTS {{zone_name}} TYPE EXTERNAL
COMMENT 'External tables - demo datasets and file-backed data';
CREATE SCHEMA IF NOT EXISTS {{zone_name}}.log_archive
COMMENT 'Legacy LIS-79 well-log tapes read in place, curated into Delta';
-- ----------------------------------------------------------------------------
-- STEP 2: Register both batches with DISCOVER
-- ----------------------------------------------------------------------------
-- LIS is identified from the physical record structure underneath the tape
-- framing, which means detection has to strip the framing first. A detector
-- that reads the raw head sees twelve bytes of little-endian tape offsets and
-- concludes nothing.
--
-- The two batches are registered separately because their curve sets differ.
-- Keeping them apart is what lets the loader project each onto the core the
-- curated table keeps, rather than widening the table for every reprocessing
-- product a vendor happened to ship.
-- ----------------------------------------------------------------------------
DROP EXTERNAL TABLE IF EXISTS {{zone_name}}.log_archive.volve_composite;
DISCOVER {{zone_name}}.log_archive.volve_composite
PATH '{{data_subdir}}/landing/volve'
WITH (FILE_METADATA = true);
DROP EXTERNAL TABLE IF EXISTS {{zone_name}}.log_archive.reprocessed_composite;
DISCOVER {{zone_name}}.log_archive.reprocessed_composite
PATH '{{data_subdir}}/landing/reprocessed'
WITH (FILE_METADATA = true);
-- (excerpt; run the demo for the full script)## What this demo does An operator's data-management team brings a LIS-79 well-log tape archive into the lake without a vendor conversion, one digitisation batch at a time. The 11 March batch is the real Volve 15/9-F-4 composite as OSDU redistributes it: 12 curves, 21505 frames, wrapped in Tape Image Format. The 12 March batch is a 26-curve reprocessed composite whose specification record does not fit in one 1024-byte physical record and so arrives split with the successor and predecessor bits set. Three things have to be right for the frame arithmetic to close with nothing left over: the tape framing is stripped, the curve list is found in logical record type 64 rather than 34, and the split specification is rejoined. The real tape has no null convention, so a curve the tool was not running reads as a number, and 1114 of its density-correction readings are below minus one gram per cubic centimetre with the worst at minus 57338. The loader applies one physical validity rule per curve and keys on the source file, so re-running a batch is a no-op. Recovers the well's casing programme from the bit-size curve and finds 1096 net pay frames between 2772 and 3423 m. See ATTRIBUTION.md. ## What it creates - `{{zone_name}}.log_archive.volve_composite` (external table) - `{{zone_name}}.log_archive.reprocessed_composite` (external table) - `{{zone_name}}.log_archive.tape_archive` (delta table) ## Running it The demo ships `setup.sql`, `queries.sql` and `cleanup.sql` under `demos/subsurface/lis-tape-archive-recovery`. The queries carry their own `ASSERT`s, so a run that completes has verified its own numbers. Data: 2 files, 1.1 MB.