Apache Iceberg

Apache Data Lakehouse Weekly: November 3 to 11, 2026

The Iceberg Catalog Federation Map, November 2026: Which Platforms Read and Write Whose Catalog

Which platforms read and write whose Iceberg catalog in November 2026, with auth method, write support, release stage and a dated source per edge....

Iceberg v4's _last_updated_time Column: "Rows Changed Since T" Without Rewriting Manifests

Iceberg v4's _last_updated_time column records when each row last changed, inherited from the snapshot timestamp with no manifest rewrite....

Should Iceberg v4 Stop Writing ORC and Avro Data Files? The Feature-Gap Case and the Read Path That Stays

Iceberg v4 may stop writing ORC and Avro data files while keeping existing files readable, plus how to audit an estate and plan a migration....

Iceberg v4 Stats-Only Fields: How Tables Store Statistics for Partition Expressions, and Why tight_stats Replaced tight_bounds

Iceberg v4 stats-only fields give derived values a field ID in content_stats, and the file-level tight_stats flag says when statistics are exact....

Apache Data Lakehouse Weekly: October 28 to November 4, 2026

Apache Data Lakehouse Weekly: October 21–28, 2026

Primary Keys Come to Apache Iceberg V4, and Most of Them Start Out Unenforced

Apache Iceberg V4 adds primary key, unique and check constraints, an enforced flag and per-snapshot status. What the spec says and how to prepare tabl...

Per-Developer Sandboxes on Apache Iceberg Branches: Lifecycle, Isolation Limits, and Cleanup

Give every developer and pull request a zero-copy Iceberg sandbox on branches: lifecycle, what branches do not isolate, and cleanup....

Inside the Iceberg REST Catalog's Multi-Table Commit Endpoint

How POST /v1/{prefix}/transactions/commit works: request shape, requirements, atomicity, error codes, idempotency, and limits....

Storing Quantized Embeddings in Apache Iceberg: Scalar 8-Bit Codes vs Product Quantization

How SQ8 and product quantization shrink Iceberg embedding columns 4x to 64x, how to store codes and codebooks, and what breaks....

Apache Data Lakehouse Weekly: October 14–21, 2026

Iceberg Is Getting a Vector Type, and Parquet Decides What It Looks Like

Iceberg's proposed vector type enforces fixed dimension, non-null, and finite values. What it promises, how Parquet shapes it, and what to do now....

What Vortex Has to Prove Before Iceberg Adds a Fourth File Format

The Vortex proposal split into criteria and candidate. What field IDs, statistics, governance, and ecosystem support a new Iceberg file format must pr...

How to Choose Between Partitioning, Sort Orders, and Z-Order in Apache Iceberg

Partitioning, sort orders, and Z-order each cluster Iceberg data differently. Compare the costs, the failure modes, and how to measure layout with dat...

Choosing Self-Hosted Object Storage for the Lakehouse After MinIO

MinIO's community edition is archived. Compare RustFS, SeaweedFS, Ceph, and Garage against what Iceberg actually needs, then test and migrate....

Apache Data Lakehouse Weekly: October 6–14, 2026

What Apache Polaris 1.9.0 Brings to Low-Egress Multi-Cloud Catalogs

What Apache Polaris 1.9.0 adds for multi-cloud lakehouses: Cloudflare R2 credential vending, a DynamoDB persistence backend, and the Iceberg 1.12 clie...

Upgrading to Apache Iceberg 1.12.0 Without Breaking Your Engines

A platform engineer's upgrade guide to Apache Iceberg 1.12.0: variant and geospatial changes, REST catalog behavior, removed APIs, and rollback....

Apache Data Lakehouse Weekly: September 29 to October 7, 2026

Apache Iceberg 1.12.0 shipped this week, and the release set off a chain of follow-up work across the open lakehouse stack....

How to Build a Golden-Question Benchmark for Your Data Agents

Build a golden-question benchmark for data agents: frozen Iceberg tags, execution-based scoring with Arrow, and CI gates....

When a Data Agent Gives a Wrong Number: Incident Response for Agentic Analytics

A plan for when a data agent gives a wrong number: severity levels, containment, Iceberg time travel reproduction, and blast radius queries....

Semantic Caching for Data Agents Without Serving Stale or Leaked Numbers

Cache data agent results safely with canonical query keys, Iceberg snapshot freshness checks, and permission-aware cache keys....

Keeping Audit Snapshots Alive While Iceberg Snapshot Expiration Runs Every Night

How Iceberg snapshot tags keep audit snapshots alive through nightly expiration: retention calendars, RETAIN semantics, compaction cost, and erasure c...

Fast Classification Models, LLMs, and the Apache Iceberg Lakehouse

Open the bill for any team that has put a large language model into a data pipeline and look at what the calls are doing....

Why the Iceberg DataFusion Integration Is Moving to Apache DataFusion

Why the Iceberg DataFusion integration moved to the DataFusion project, and what the split means for users, Comet, and iceberg-rust contributors....

What Iceberg v4's Proposed FILE Type Means for Multimodal Tables

Iceberg v4's proposed FILE type brings first-class media references to tables, via Parquet's FILE logical type, ranges, checksums, and pre-signed URLs...

Fast Classification Models, LLMs, and the Apache Iceberg Lakehouse

How fast classification models like Jev alongside open alternatives such as GLiClass compare with LLMs, and how to run both together inside an Apache ...

CVE-2026-73334 and the Trust Boundary Inside an Encrypted Parquet File

CVE-2026-73334 lets a tampered Parquet footer route a reader's KMS token to an attacker. Here's the fix, Iceberg's safe path, and how to audit your la...

Parquet Page Indexes and the Last Mile of Pruning in Apache Iceberg

Parquet page indexes can cut selective Iceberg scans by an order of magnitude on sorted data. How they work, what they cost, and how to lay out tables...

How Apache Polaris Plans to Share Iceberg Tables Across Organizations

Apache Polaris's Open Sharing proposal adds first-class shares, external consumers, and listings so any Iceberg REST engine can read shared tables....

Vector Search Directly Over Iceberg Tables, and When You Still Need a Vector Database

Embeddings are just columns. When exact vector search over Iceberg scans beats a vector database, when it does not, and how to lay out tables....

Migrating Into Iceberg Without Moving Data

add_files, snapshot, and migrate compared: the three in-place paths into Iceberg, the reconciliation each requires, the layout traps, and the rollback...

What Iceberg Table Maintenance Actually Costs

A cost model for compaction, snapshot expiry, orphan cleanup, and manifest rewriting: what each operation spends, on which meter, and how to set a sch...

Running an Iceberg Lakehouse on Kubernetes

Catalog, maintenance, and compaction as Kubernetes workloads: scheduling classes, job structure, credential flow, and the failures that come from the ...

Partition Statistics Files in Apache Iceberg

The underused Iceberg metadata for planning: what the partition statistics file holds, what the spec guarantees, how to write one, and when it earns i...

What to Assert When You Test an Iceberg Pipeline

Fixtures, in-memory catalogs, and golden metadata: the assertions that catch wrong rows, unsafe reruns, schema drift, and concurrent-write corruption ...

The 2026 Iceberg REST Catalog Compatibility Report

A repeatable test for what an Iceberg REST catalog actually serves, a scoring scheme that separates design from breakage, and the 2026 evidence across...

Serving Iceberg Tables From Two Regions

Three multi-region topologies that work and one that mostly does not, what an Iceberg commit costs across regions, and where the catalog has to live....

How Iceberg Catalogs Hand Engines Storage Access

Credential vending end to end: the wire protocol, scoped access on each cloud, remote signing, credential lifetime on long jobs, and failures that loo...

Kafka Connect to Iceberg: How the Commit Actually Works

Exactly-once semantics in the Iceberg sink connector: the coordinator, the control topic, offsets stored inside Iceberg snapshots, and where duplicate...

The Open Lakehouse Explained, Then Built on Your Laptop with Dremio and RustFS or Ceph

The five layers of the open lakehouse explained, then a lab: Parquet, Iceberg, Polaris, Arrow, and Ossie running in two containers on your own machine...

dbt on Iceberg: Incremental Models on Open Tables

How dbt incremental materializations map to Iceberg operations, and the configuration, predicates, and maintenance that keep them healthy....

Disaster Recovery for Iceberg Tables: Replication, Backup, and Restore

Disaster recovery for Iceberg across four tiers: snapshots, object versioning, catalog backup, and cross-region replication....

Deleting User Data From an Immutable Lakehouse: GDPR Hard Deletes on Iceberg

How to turn a logical delete on immutable Iceberg into a physical erasure across snapshots, versions, replicas, and downstream copies....

Geospatial Data in Apache Iceberg: Geometry, Geography, and GeoParquet

How Iceberg v3 geometry and geography types, bounding boxes, and native Parquet types give spatial data first-class standing....

Default Column Values and Field IDs: How Iceberg Schema Evolution Works at the Spec Level

How field IDs and initial and write defaults let Iceberg change schemas on large tables without rewriting data, at the spec level....

The Iceberg Table Properties That Actually Matter

The Iceberg table properties that decide file count, pruning, write amplification, retention, and metadata growth, by workload....