Apache Iceberg
ELI5 - The Vibe Check
Imagine a giant folder of Parquet files that used to be pure chaos, nobody knew which files were current, nobody could add a column safely, and two people writing at once could corrupt everything. Iceberg is the librarian who shows up and adds a proper table of contents, version history, and safe locks, all sitting on top of files you already have in cheap object storage.
Real Talk
Apache Iceberg is an open table format that adds ACID transactions, schema evolution, hidden partitioning, and time travel on top of data files (typically Parquet) stored in object storage like S3. It gives engines like Spark, Trino, Snowflake, and Flink a shared, versioned view of the same tables without locking you into one compute engine.
Show Me The Code
SELECT * FROM orders FOR VERSION AS OF 1234567890;
-- Iceberg time travel: query the table exactly as it looked at a past snapshot
When You'll Hear This
"We migrated our data lake to Iceberg so Snowflake and Spark can query the same tables." / "The Iceberg snapshot lets us time-travel back to yesterday's data."
Related Terms
Apache Parquet
Parquet is a file format that stores your data by column instead of by row, which sounds boring until you realize most analytics queries only touch a handf...
Data Lake
A data lake is a massive storage dump where you throw every piece of data in its raw format. CSV files, JSON, images, logs, whatever.
Delta Lake
Delta Lake is Iceberg's cousin from the Databricks side of the family, same basic idea: bolt ACID transactions and version history onto a pile of Parquet f...
Lakehouse
For years you had to pick a side: a data warehouse, structured, fast, expensive, or a data lake, cheap, flexible, and a mess nobody fully trusted.
Schema Evolution
Your data's shape never actually stays still: new fields show up, old ones get renamed, types change, and yet you've got old data sitting around in the old...