Data Lakehouse & Columnar Storage (Parquet)

Over the past three decades, enterprise analytical data management has cycled through multiple architectural revolutions. Early traditional data warehouses delivered ACID transactions and fast query performance but were crippled by high proprietary hardware costs and rigid schema-on-write paradigms. The subsequent rise of big data lakes (Hadoop and cloud object storage) unlocked virtually infinite storage for semi-structured and unstructured data, but lakes quickly deteriorated into chaotic 'data swamps' plagued by corrupted files, dirty reads, missing transactional guarantees, and sluggish query execution.

The contemporary Data Lakehouse architecture reconciles these two paradigms. By layering open transactional metadata protocols (Delta Lake, Apache Iceberg, Apache Hudi) and high-performance columnar storage formats (Apache Parquet) directly on top of low-cost commodity cloud object storage, the Lakehouse delivers the reliability, ACID semantics, and governance of an enterprise data warehouse with the flexibility, open formats, and elasticity of a data lake.

The Evolution of Analytical Storage Architectures#

The architectural trajectory from warehouse to lake to lakehouse is rooted in changing storage-compute economics and the diversity of analytical workloads.

+---------------------------------------------------------------------------------------------------+
|                                  Analytical Architecture Evolution                                |
+-----------------------------------+-----------------------------------+---------------------------+
| Traditional Data Warehouse        | Cloud Data Lake                   | The Modern Data Lakehouse |
| (e.g., Teradata, Oracle Exadata)  | (e.g., Hadoop HDFS, Raw AWS S3)   | (Delta Lake, Iceberg)     |
+-----------------------------------+-----------------------------------+---------------------------+
| - Tightly coupled compute/storage | - Decoupled compute & S3 storage  | - Decoupled compute/S3    |
| - Rigid Schema-on-Write           | - Schema-on-Read (No governance)  | - Enforced Schema & ACID  |
| - High proprietary hardware cost  | - Low commodity object cost       | - Low commodity storage   |
| - Full ACID transactions          | - No ACID (Dirty reads, append-only)| - Full ACID & MVCC      |
| - SQL BI analytics only           | - SQL, Spark, Python ML, Streaming| - Unified SQL, ML, & BI   |
| - Proprietary file formats        | - Raw CSV, JSON, ORC, Parquet     | - Open Parquet + Metadata |
+-----------------------------------+-----------------------------------+---------------------------+

Why Data Lakes Became Data Swamps#

Early cloud data lakes relied on plain directories of files on cloud object storage (e.g., s3://bucket/events/*.json). While inexpensive, cloud object stores lack POSIX filesystem semantics:

1. Lack of Atomic File Swaps: Object storage lacks atomic multi-file directory rename operations. If a Spark job writing 100 files fails at file 87, the partial dataset remains permanently in the directory. Downstream queries read corrupted, half-written data.
2. No Multi-Version Concurrency Control (MVCC): If an ETL pipeline writes new records to a partition while a data scientist runs an analytical query against the same partition, the reader crashes with missing file exceptions (FileNotFoundException) or reads duplicate records.
3. No In-Place Mutation: Modifying or deleting records (e.g., for GDPR "Right to be Forgotten" compliance) required reading the entire multi-terabyte dataset, rewriting the modified files to a new directory, and deleting the old directory.

The Data Lakehouse & Open Table Formats#

The modern Data Lakehouse resolves these structural failures by introducing a transactional metadata layer between the query engine and the underlying object storage files. Rather than treating an object storage directory as a table, the table is defined by a formal, versioned transaction log.

+---------------------------------------------------------------------------------------+
|                              The Lakehouse Storage Stack                              |
+---------------------------------------------------------------------------------------+
|  Analytical Query Engines: Spark, Trino, Presto, DuckDB, Snowflake, Databricks        |
+---------------------------------------------------------------------------------------+
|  Transactional Metadata Protocol: Delta Lake / Apache Iceberg / Apache Hudi           |
|  - ACID Transaction Log (JSON / Avro manifests)                                       |
|  - Multi-Version Concurrency Control (MVCC) & Optimistic Concurrency Control (OCC)    |
|  - Schema Enforcement, Evolution, and Time-Travel Snapshot History                    |
+---------------------------------------------------------------------------------------+
|  Columnar Physical File Storage: Apache Parquet / ORC (Snappy / ZSTD compressed)      |
+---------------------------------------------------------------------------------------+
|  Cloud Object Storage: AWS S3, Google Cloud Storage, Azure Data Lake Storage Gen2    |
+---------------------------------------------------------------------------------------+

1. Delta Lake Architecture (Databricks)#

Delta Lake maintains an ordered, append-only transaction log located inside the _delta_log/ directory:

• Commit Log (`.json`): Every state-changing operation (write, update, delete, schema change) writes a new sequential JSON file (e.g., 000000.json, 000001.json). The commit log lists atomic actions: add (registering new Parquet data files) and remove (tombstoning deleted files).
• Checkpointing (`.checkpoint.parquet`): Every 10 commits, Delta Lake automatically aggregates preceding JSON files into a consolidated Parquet checkpoint file. Query engines read the latest checkpoint plus subsequent JSON files to compute the current table state in milliseconds without scanning millions of historical commits.
• ACID Guarantees via OCC: When two writers concurrently attempt to commit changes, Delta Lake uses Optimistic Concurrency Control (OCC):

1. Both writers record the starting version V_start.

2. The first writer commits version V_start + 1.

3. The second writer checks whether the files it modified overlap with the files committed by the first writer. If there is no conflict, it commits as V_start + 2; if a conflict exists, it aborts and retries.

2. Apache Iceberg (Netflix)#

Apache Iceberg utilizes a hierarchical snapshot tree architecture:

• Iceberg Catalog: Stores the current pointer to the active metadata file (v1.metadata.json).
• Metadata File: Contains table schema, partition specs, and a list of all historical snapshots.
• Manifest List: For each snapshot, contains pointers to manifest files along with partition summaries and file counts.
• Manifest Files (Avro): Contains pointers to individual physical Parquet data files along with detailed column-level metrics (min/max values, null counts, NaN counts).
• Hidden Partitioning: Unlike Hive (where partition columns must be explicitly stated in queries, e.g., WHERE date = '2026-10-30'), Iceberg abstracts partitioning. Users query raw timestamps (WHERE event_time > '2026-10-30 08:00:00'), and Iceberg automatically prunes daily or hourly partitions under the hood without query modifications.

Columnar Storage Mechanics: Apache Parquet#

Analytical queries (OLAP) exhibit fundamentally different access patterns than transactional queries (OLTP):

• OLTP (Row-Oriented: PostgreSQL, MySQL): Inserts, updates, and retrieves complete rows across all 50 columns for a single specific user (SELECT * FROM users WHERE id = 4820).
• OLAP (Column-Oriented: Parquet, ClickHouse, Snowflake): Aggregates a handful of columns across billions of records (SELECT AVG(amount), SUM(tax) FROM transactions WHERE year = 2026).

Reading row-oriented files (CSV, JSON, Avro) for analytical aggregation requires loading 100% of all byte data from disk into memory, discarding 95% of unqueried columns. Columnar formats read exclusively the queried columns.

Row-Oriented Storage (CSV / JSON):
[ ID, Name, Age, Country ] [ ID, Name, Age, Country ] [ ID, Name, Age, Country ]
===> Must scan all bytes to read 'Age'!

Column-Oriented Storage (Apache Parquet):
[ ID_1, ID_2, ID_3 ] [ Name_1, Name_2, Name_3 ] [ Age_1, Age_2, Age_3 ] [ Country_1, Country_2, Country_3 ]
===> Scans ONLY the 'Age' column block from disk!

Physical Anatomy of a Parquet File#

An Apache Parquet file is a structured binary container formatted into a hierarchical layout:

+---------------------------------------------------------------------------------------+
|                               Apache Parquet File Layout                              |
+---------------------------------------------------------------------------------------+
|  Magic Bytes: "PAR1"                                                                  |
+---------------------------------------------------------------------------------------+
|  Row Group 1 (typically 128 MB to 512 MB in size)                                     |
|  +---------------------------------------------------------------------------------+  |
|  | Column Chunk 1 (e.g., 'user_id')                                                |  |
|  |   - Data Page 1 (Dictionary encoded & Snappy compressed)                        |  |
|  |   - Data Page 2 (RLE bit-packed)                                                |  |
|  +---------------------------------------------------------------------------------+  |
|  | Column Chunk 2 (e.g., 'transaction_amount')                                     |  |
|  |   - Data Page 1 (Plain float64 array, compressed)                               |  |
|  +---------------------------------------------------------------------------------+  |
|  | Column Chunk 3 (e.g., 'country_code')                                           |  |
|  |   - Dictionary Page (Unique country string values)                              |  |
|  |   - Data Page 1 (Integer dictionary indices)                                    |  |
|  +---------------------------------------------------------------------------------+  |
+---------------------------------------------------------------------------------------+
|  Row Group 2 (contains next batch of N rows)                                          |
|  ...                                                                                  |
+---------------------------------------------------------------------------------------+
|  File Footer (Metadata)                                                               |
|  - File Schema (Column types, nested structures)                                      |
|  - Row Group Metadata (Offsets, byte lengths)                                         |
|  - Column Chunk Metadata & Statistics:                                                |
|      Column 'age': min = 18, max = 89, null_count = 0                                 |
|      Column 'amount': min = 1.50, max = 9480.00, null_count = 12                     |
+---------------------------------------------------------------------------------------+
|  Footer Length: 4-byte integer                                                        |
+---------------------------------------------------------------------------------------+
|  Magic Bytes: "PAR1"                                                                  |
+---------------------------------------------------------------------------------------+

1. Projection Pushdown#

Because column chunks are physically contiguous in the file, query engines read only the exact byte ranges corresponding to queried columns.

• In a table with 100 columns where a machine learning feature extraction query requires 5 features, projection pushdown immediately eliminates 95% of disk I/O and network bandwidth.

2. Predicate Pushdown (Row Group Skipping)#

When a SQL query contains a filter clause (WHERE transaction_amount > 10000.00):

1. The query engine reads only the small File Footer at the very end of the Parquet file.
2. It inspects the min and max statistics for the transaction_amount column chunk in each Row Group.
3. If Row Group 1 has max = 9480.00, the engine mathematically proves that zero records matching > 10000.00 can possibly exist in Row Group 1.
4. Row Group 1 is completely skipped: zero bytes of data from Row Group 1 are read from disk.

Columnar Compression & Encoding Algorithms#

Because a single column chunk contains values of the identical data type (often with high repetition or sorted ordering), columnar data compresses with extraordinary efficiency (typically achieving 70% to 90% space reduction over raw CSV/JSON).

1. Dictionary Encoding#

When a column contains repeating strings or categories with low-to-medium cardinality (e.g., US state codes):

• Parquet constructs a Dictionary Page storing unique values: {0: "CA", 1: "NY", 2: "TX", 3: "WA"}.
• The data page replaces long string bytes with tiny integer indices (0, 0, 1, 2, 0, 3).

2. Run-Length Encoding (RLE) & Bit-Packing#

For sorted or clustered data where identical values repeat consecutively:

• Sequence: [CA, CA, CA, CA, CA, CA, CA, CA] (Indices: [0, 0, 0, 0, 0, 0, 0, 0])
• RLE encodes this entire 8-element sequence as two bytes: (count = 8, value = 0).
• Bit-Packing: If a dictionary contains only 4 unique states, indices 0, 1, 2, 3 require only 2 bits per integer (ceil(log2(4)) = 2 bits), packing 4 distinct record coordinates into a single 8-bit byte.

3. General-Purpose Block Compression#

After encoding, data pages are compressed using high-throughput block compression codecs:

• Snappy: Optimized for blazing decompression speed (> 250 MB/s per core) with moderate compression ratios (~50%). Default standard in real-time query engines.
• Zstandard (ZSTD): Developed by Meta; achieves superior compression ratios (often 2x to 3x better than Snappy) with high decompression speeds. Modern industry standard for long-term Lakehouse storage.

Compaction & Multidimensional Clustering (Z-Ordering)#

The Small Files Problem#

In streaming ingest pipelines, micro-batches write small Parquet files every 10 seconds. Over months, a table accumulates millions of tiny 500 KB Parquet files.

• The Pathology: Cloud object storage (S3) charges per API request (LIST, GET). Query engines spend 90% of their execution time listing millions of S3 objects and downloading headers, rather than reading data.
• Compaction (Bin-Packing): Periodic background maintenance jobs read thousands of tiny files and coalesce them into optimal, contiguous 128 MB to 512 MB Parquet files:

```sql

OPTIMIZE orders;

```

Multidimensional Clustering: Z-Ordering#

Standard table partitioning works well for a single low-cardinality dimension (e.g., PARTITIONED BY (order_date)). However, real-world queries filter across multiple independent high-cardinality dimensions (e.g., WHERE customer_id = 1042 AND product_category = 'ELECTRONICS').

Z-Ordering (Morton space-filling curves) maps multi-dimensional feature space onto a 1D space-filling curve while preserving spatial locality:

+---------------------------------------------------------------------------------------+
|                            Z-Order Curve Multi-Column Clustering                      |
+---------------------------------------------------------------------------------------+
|  Product Category                                                                     |
|       ^                                                                               |
|     3 |   (12)-----(13)     (14)-----(15)                                             |
|       |     |   /    |        |   /    |                                              |
|     2 |   ( 8)-----( 9)     (10)-----(11)                                             |
|       |        /                  /                                                   |
|     1 |   ( 4)-----( 5)     ( 6)-----( 7)                                             |
|       |     |   /    |        |   /    |                                              |
|     0 |   ( 0)-----( 1)     ( 2)-----( 3)                                             |
|       +-------------------------------------> Customer ID                             |
|             0        1        2        3                                              |
+---------------------------------------------------------------------------------------+

By interleaving the binary bits of customer_id and product_category, data points with similar coordinates in both dimensions are colocated within the same physical Parquet row groups. When a query filters on either or both columns, predicate pushdown skips up to 90% of row groups, achieving orders-of-magnitude query acceleration.

OPTIMIZE orders ZORDER BY (customer_id, product_category);

Schema Evolution & Non-Breaking Contract Changes#

In continuous enterprise data pipelines, upstream production applications constantly evolve: new columns are added, columns are deprecated, and field data types are widened (e.g., converting an integer user_id to a 64-bit bigint).

The Lakehouse Schema Governance Matrix#

Open table formats (Delta Lake, Apache Iceberg) enforce strict Schema Validation by default:

• Schema Enforcement (Schema Validation): If an incoming DataFrame contains extra columns not declared in the table schema, or if data types conflict, the write operation is rejected atomically with an exception. This prevents silent corruptions where rogue columns enter production tables unnoticed.
• Schema Evolution (`mergeSchema = true`): When an intentional schema update occurs, the table format updates the table metadata without rewriting existing historical Parquet data files:

```python

# Delta Lake Schema Evolution

df_new.write \

.format("delta") \

.mode("append") \

.option("mergeSchema", "true") \

.save("s3://lakehouse/gold/users")

```

Column Mapping & In-Place Renaming#

Historically, Parquet identified columns by physical name strings in the file schema. Renaming a column required rewriting every historical Parquet file on S3.

• Modern table formats decouple physical column storage from logical metadata using Column Mapping:

* Each column is assigned an immutable internal integer ID (column_id = 1042).

* The Parquet file chunks store the internal ID.

* When a column is renamed from phone_number to mobile_phone, the table format merely updates the logical metadata mapping: 1042 -> "mobile_phone".

Zero data files are rewritten*, executing instantaneous table renames over petabyte-scale datasets.

Data Lakehouse Governance & Fine-Grained Security#

In enterprise deployments, data lakes cannot operate as open file systems accessible to all engineers. Compliance frameworks (GDPR, HIPAA, SOC 2, PCI-DSS) mandate strict access control, audit logging, and sensitive data masking.

+---------------------------------------------------------------------------------------+
|                       Enterprise Lakehouse Governance Layer                           |
+---------------------------------------------------------------------------------------+
|  Users / Data Scientists / BI Dashboards                                              |
|                           |                                                           |
|                           v                                                           |
|  [ Governance Layer: Unity Catalog / Apache Polaris / AWS Lake Formation ]            |
|  - Centralized Identity & Access Management (SSO / RBAC / ABAC)                       |
|  - Dynamic Column Masking: SHA-256 hash on SSN, email                                 |
|  - Dynamic Row-Level Security: Filter 'WHERE region = user.assigned_region'          |
|                           |                                                           |
|                           v Grants Temporary Scoped S3 Tokens                         |
|  [ Object Storage Data: s3://enterprise-lakehouse/data/ ]                             |
+---------------------------------------------------------------------------------------+

Dynamic Column Masking & Row-Level Filtering#

Centralized catalog services (e.g., Unity Catalog, Apache Ranger, AWS Lake Formation) intercept SQL queries prior to execution planning:

1. Dynamic Column Masking: A data engineer queries SELECT email, ssn, purchase_amount FROM transactions. If the engineer lacks the PII_ACCESS role, the governance engine transparently rewrites the query plan:

```sql

SELECT

CONCAT('***@', SPLIT_PART(email, '@', 2)) AS email,

'XXX-XX-' || SUBSTRING(ssn, 8, 4) AS ssn,

purchase_amount

FROM transactions;

```

1. Row-Level Security (RLS): German regional analysts querying global orders automatically receive a transparent predicate injection (WHERE country_code = 'DE'), guaranteeing regional isolation without requiring duplicate physical tables.

High-Dimensional Vector Embeddings in Parquet#

With the explosion of Generative AI, Retrieval-Augmented Generation (RAG), and neural semantic search, data lakehouses increasingly store billions of high-dimensional vector embeddings alongside traditional tabular attributes.

• Apache Arrow & Fixed-Size List Extensions: Parquet natively supports fixed-size lists (FixedSizeListArray), enabling dense 768-dimensional or 1536-dimensional float32 vectors to be stored contiguously in memory.
• Vector Indexing Engines (e.g., LanceDB, DuckDB-VSS): By colocating dense vector chunks inside columnar Parquet files, lakehouses allow unified hybrid queries combining relational SQL filters with cosine similarity searches:

```sql

SELECT document_id, text, cosine_distance(embedding, [0.042, -0.891, ...]) AS score

FROM enterprise_knowledge_base

WHERE department = 'LEGAL' AND classification = 'CONFIDENTIAL'

ORDER BY score ASC

LIMIT 5;

```

This eliminates the need to maintain disconnected vector database infrastructure, consolidating generative AI embeddings directly within the enterprise Data Lakehouse.

Checkpoint Practice Problems#

Problem 1 (Columnar Projection Pushdown Bandwidth Reduction): An e-commerce clickstream analytical table has C = 80 columns. The average uncompressed row size is 2,000 bytes. A daily batch of N = 50,000,000 rows is stored in an object storage data lake.

• Scenario A (Row-oriented CSV storage): The entire dataset is stored in raw CSV format.
• Scenario B (Parquet Columnar storage): A feature engineering job runs an aggregation query selecting only k = 4 columns (user_id, timestamp, page_type, duration). These 4 columns account for a combined 120 bytes per row (uncompressed). Parquet block compression achieves a 3.0x compression ratio (compressed size = uncompressed / 3.0) for both scenarios.

Calculate:

1. The total compressed data volume transferred over the network from S3 under Scenario A.
2. The total compressed data volume transferred under Scenario B leveraging Parquet projection pushdown.
3. The percentage reduction in network I/O bandwidth achieved by Scenario B over Scenario A (1 GB = 1,000,000,000 bytes).

Solution:

1. Scenario A (Row-oriented CSV):

* Uncompressed volume:

50,000,000 rows · 2,000 bytes/row = 100,000,000,000 bytes = 100.0 GB

* Compressed volume transferred:

100.0 GB / 3.0 ≈ 33.333 GB

1. Scenario B (Parquet Columnar with Projection Pushdown):

* Because of projection pushdown, only the 4 queried columns (120 bytes/row) are read from disk:

50,000,000 rows · 120 bytes/row = 6,000,000,000 bytes = 6.0 GB uncompressed

* Compressed volume transferred:

6.0 GB / 3.0 = 2.000 GB

1. Bandwidth Reduction:

```

Reduction = [ (33.333 GB - 2.000 GB) / 33.333 GB ] · 100%

= [ 31.333 / 33.333 ] · 100% = 94.0%

```

Projection pushdown slashes data transfer bandwidth by 94.0%, reducing download payload from 33.3 GB down to 2.0 GB.

Problem 2 (Parquet Predicate Pushdown Row Group Skipping): A Parquet table contains 20 independent Row Groups, each holding 1,000,000 records. A query executes:

SELECT COUNT(*) FROM sensor_telemetry WHERE temperature > 85.0;

The file footer metadata records the following temperature statistics across the 20 row groups:

• Row Groups 1 to 14: min = 15.0, max = 82.5
• Row Groups 15 to 17: min = 20.0, max = 84.8
• Row Group 18: min = 40.0, max = 89.2
• Row Group 19: min = 70.0, max = 95.0
• Row Group 20: min = 86.0, max = 110.0

1. Identify which row groups are eligible for complete elimination via predicate pushdown without reading their data pages.
2. Identify which row groups must be physically decompressed and scanned.
3. Compute the percentage of row groups pruned from the scan.

Solution:

1. Evaluate predicate condition temperature > 85.0:

A row group can be safely skipped if and only if its maximum value satisfies max ≤ 85.0.

Row Groups 1 to 14: `max = 82.5 ≤ 85.0` -> Pruned / Skipped (14 row groups)*.

Row Groups 15 to 17: `max = 84.8 ≤ 85.0` -> Pruned / Skipped (3 row groups)*.

Total pruned row groups = 14 + 3 = 17 row groups*.

1. Scanned row groups:

* Row Group 18 (max = 89.2 > 85.0): Must scan (some records may match).

* Row Group 19 (max = 95.0 > 85.0): Must scan.

* Row Group 20 (max = 110.0 > 85.0): Must scan.

Total scanned = 3 row groups.

1. Percentage pruned:

```

Pruning Percentage = (17 / 20) · 100% = 85.0%

```

Predicate pushdown prunes 85.0% of data blocks without reading a single data page byte.

Problem 3 (Small Files Compaction & API Cost Elimination): A streaming ingestion engine writes 6 small Parquet files per minute into an S3 data lake.

• Daily files produced: 6 files/min · 1,440 min = 8,640 files per day.
• Average file size: 500 KB. Total daily volume = 4,320,000 KB = 4.32 GB.
• Daily analytical queries: Q = 200 queries per day, each performing a table scan.
• AWS S3 pricing: 0.0004 per 1,000 GET requests.

1. Compute the daily count of S3 GET requests and monthly S3 API cost (30 days) incurred by analytical queries scanning the uncompacted files from a single month (accumulating 259,200 files).
2. If a daily compaction job merges each day's 8,640 files into 9 optimal 480 MB Parquet files (accumulating 270 files per month), compute the new monthly S3 API cost.
3. Compute the absolute monthly API cost savings.

Solution:

1. Uncompacted Scenario (30-day cumulative pool = 259,200 files):

* Each query issues 1 GET request per file: 259,200 GET requests per query.

* Daily queries = 200 queries:

200 · 259,200 = 51,840,000 GET requests per day.

* Monthly GET requests (30 days):

30 · 51,840,000 = 1,555,200,000 GET requests per month.

* Monthly API cost:

(1,555,200,000 / 1,000) · 0.0004 = 1,555,200 · 0.0004 = 622.08 per month.

1. Compacted Scenario (30-day cumulative pool = 270 files):

* Each query issues 270 GET requests:

200 queries · 270 files = 54,000 GET requests per day.

* Monthly GET requests:

30 · 54,000 = 1,620,000 GET requests per month.

* Monthly API cost:

(1,620,000 / 1,000) · 0.0004 = 1,620 · 0.0004 = 0.648 ≈ 0.65 per month.

1. Cost Savings:

```

Monthly Savings = 622.08 - 0.65 = 621.43 per month (99.9% reduction)

```

Compaction eliminates file overhead, reducing monthly S3 request costs from 622.08 to 65 cents while dramatically improving query execution speed.

Problem 4 (Run-Length Encoding Compression Ratio): A sorted integer column chunk in a Parquet row group contains N = 10,000 consecutive values of a sensor status code:

• Values 1 to 6,000: Status code 0
• Values 6,001 to 9,500: Status code 1
• Values 9,501 to 10,000: Status code 2

1. Under raw uncompressed storage (using 32-bit / 4-byte integers), compute the total byte size required.
2. Under Run-Length Encoding (RLE), where each consecutive run of identical integers is encoded as a 2-tuple (count: uint32, value: uint32) requiring 8 bytes total per run, compute the encoded byte size.
3. Compute the exact compression ratio achieved by RLE for this column chunk.

Solution:

1. Uncompressed Size:

```

Size_raw = 10,000 integers · 4 bytes/integer = 40,000 bytes (40.0 KB)

```

1. RLE Encoded Size:

There are exactly three contiguous runs:

* Run 1: (count = 6000, value = 0) -> 8 bytes

* Run 2: (count = 3500, value = 1) -> 8 bytes

* Run 3: (count = 500, value = 2) -> 8 bytes

```

Total Runs = 3

Size_RLE = 3 runs · 8 bytes/run = 24 bytes

```

1. Compression Ratio:

```

Compression Ratio = Size_raw / Size_RLE = 40,000 bytes / 24 bytes ≈ 1,666.67 to 1

```

Run-Length Encoding compresses the 40 KB column block down to just 24 bytes, achieving a 1,667x compression ratio.