[{"data":1,"prerenderedAt":106},["ShallowReactive",2],{"term-s\u002Fschema-evolution":3,"related-s\u002Fschema-evolution":90},{"id":4,"title":5,"acronym":6,"body":7,"category":71,"description":72,"difficulty":73,"extension":74,"letter":75,"meta":76,"navigation":77,"path":78,"related":79,"seo":84,"sitemap":85,"stem":88,"subcategory":6,"__hash__":89},"terms\u002Fterms\u002Fs\u002Fschema-evolution.md","Schema Evolution",null,{"type":8,"value":9,"toc":65},"minimark",[10,15,19,23,26,30,54,58,61],[11,12,14],"h2",{"id":13},"eli5-the-vibe-check","ELI5 - The Vibe Check",[16,17,18],"p",{},"Your data's shape never actually stays still: new fields show up, old ones get renamed, types change, and yet you've got old data sitting around in the old shape while new data arrives in the new shape. Schema evolution is the set of rules and tooling that lets both versions coexist peacefully instead of every change becoming a breaking migration that takes the whole pipeline down.",[11,20,22],{"id":21},"real-talk","Real Talk",[16,24,25],{},"Schema evolution is the ability to change a data schema, adding, removing, or modifying fields, while remaining compatible with data written under previous versions of that schema. Formats like Avro and Protocol Buffers, and table formats like Iceberg, support this natively through compatibility rules (backward, forward, or full compatibility), so producers and consumers don't need to be upgraded in lockstep.",[11,27,29],{"id":28},"show-me-the-code","Show Me The Code",[31,32,37],"pre",{"className":33,"code":34,"language":35,"meta":36,"style":36},"language-sql shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","-- Backward-compatible schema change: add a nullable column with a default\nALTER TABLE orders ADD COLUMN discount_code TEXT DEFAULT NULL;\n","sql","",[38,39,40,48],"code",{"__ignoreMap":36},[41,42,45],"span",{"class":43,"line":44},"line",1,[41,46,47],{},"-- Backward-compatible schema change: add a nullable column with a default\n",[41,49,51],{"class":43,"line":50},2,[41,52,53],{},"ALTER TABLE orders ADD COLUMN discount_code TEXT DEFAULT NULL;\n",[11,55,57],{"id":56},"when-youll-hear-this","When You'll Hear This",[16,59,60],{},"\"The schema registry blocked the deploy because the change wasn't backward compatible.\" \u002F \"Iceberg's schema evolution let us add a column without rewriting every existing file.\"",[62,63,64],"style",{},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":36,"searchDepth":50,"depth":50,"links":66},[67,68,69,70],{"id":13,"depth":50,"text":14},{"id":21,"depth":50,"text":22},{"id":28,"depth":50,"text":29},{"id":56,"depth":50,"text":57},"database","Your data's shape never actually stays still: new fields show up, old ones get renamed, types change, and yet you've got old data sitting around in the old...","intermediate","md","s",{},true,"\u002Fterms\u002Fs\u002Fschema-evolution",[80,81,82,83],"Avro","Apache Iceberg","Schema Registry","Migration",{"title":5,"description":72},{"changefreq":86,"priority":87},"weekly",0.7,"terms\u002Fs\u002Fschema-evolution","qx-dbU6TXyBby4I6uOX8rddfTF1lamQhHWBCa09A5UM",[91,95,99,103],{"title":81,"path":92,"acronym":6,"category":71,"difficulty":93,"description":94},"\u002Fterms\u002Fa\u002Fapache-iceberg","advanced","Imagine a giant folder of Parquet files that used to be pure chaos, nobody knew which files were current, nobody could add a column safely, and two people...",{"title":80,"path":96,"acronym":6,"category":97,"difficulty":73,"description":98},"\u002Fterms\u002Fa\u002Favro","backend","Avro is a data format built for a world where the shape of your data keeps changing but old and new versions still need to talk to each other.",{"title":83,"path":100,"acronym":6,"category":71,"difficulty":101,"description":102},"\u002Fterms\u002Fm\u002Fmigration","beginner","A migration is a versioned script that modifies your database schema — adding a column, creating a table, changing a type.",{"title":82,"path":104,"acronym":6,"category":97,"difficulty":73,"description":105},"\u002Fterms\u002Fs\u002Fschema-registry","Think of it as the dictionary police for your data. Every message that flows through your system has to be registered in the Schema Registry first, like ge",1792528121695]