Latest Articles: page 4
692 posts from Alex's blogs and newsletters, newest first. Thoughts on tech, data, policy, and philosophy.
Postgres Meets the Lakehouse: pg_lake, pg_duckdb, and When Postgres Is Enough
What pg_lake, pg_duckdb, and pg_mooncake do at the Iceberg level, and honest thresholds for when Postgres is enough....
Schema Registries and Event Schemas: Avro, Protobuf, and JSON Schema on the Way Into the Lakehouse
How Avro, Protobuf, and JSON Schema evolve through a registry, and how that maps to the schema evolution rules of Iceberg....
Storage-Partitioned Joins and the Bucket Transform
How the spec-defined bucket transform lets engines skip the shuffle in joins, and how to set it up and keep it engaged in Spark....
Synthetic Data in the Lakehouse: Generation, Governance, and Testing
What synthetic data in a lakehouse is for, the generation methods, how to preserve fidelity, and where synthetic tables belong....
Will AI Replace Data Engineers?
What the evidence shows about whether AI replaces data engineers, which parts of the job compress, and which parts do not....
Geospatial Data in Apache Iceberg: Geometry, Geography, and GeoParquet
A logistics team stores 40 million delivery stops in an Apache Iceberg table....
Open Standards for Agentic Harnesses
Every team that gets serious about AI agents hits the same wall, usually around month three....
Apache Data Lakehouse Weekly: August 19 to 26, 2026
The lakehouse projects spent this week arguing about boundaries....
The Great Despecialization: Why AI Changes the Shape of Jobs Instead of Deleting Them
I run a handful of personal websites....
AI Weekly: Qwen4 Preview, Hot Chips, and Agent Tools Go GA
Week of August 19 to 26, 2026...
Your Agent Should Answer the Phone: A Field Guide to AI Gateways on Slack, Discord, Telegram, Signal, and Teams
The most useful thing my terminal agent ever did happened while I was nowhere near a terminal....
The Minimum Wage Debate Done Honestly
TL;DR The minimum wage debate has been dominated by two oversimplified positions: the conservative view that any wage floor above the market rate must...
Graphs in AI Engineering Have Solved Three Problems. The Fourth Is the Plan.
Ask an agent to ship a feature and watch what it does....
Agent-Driven Storage Tiering for Apache Iceberg: Moving Cold Data Without Breaking Queries
A background agent can move cold Iceberg partitions to cheaper tiers without breaking live queries. Heatmaps, path-safe moves, and restore paths....
Securing the Agentic Lakehouse Gateway: Preventing Prompt Injection and Data Exfiltration
Agentic lakehouse gateways face prompt injection and exfiltration through query results. A threat model and defenses for the layer in front of data....
Apache Ossie and Apache Polaris: Putting Semantic Models in the Open Catalog
Apache Ossie and Polaris put metric definitions in the open catalog. What the spec covers, what Polaris stores, and what is still unfinished....
Arrow Flight SQL and ADBC: Why the Database Driver Is the Slowest Part of Your Query
JDBC and ODBC often dominate large-result time. Flight SQL and ADBC keep data columnar from server to client, with Python, Go, and Rust examples....
DataFusion Comet 1.0 and What Native Rust Scans Change for Spark on Iceberg
DataFusion Comet 1.0 replaces Spark Iceberg scans with native Rust. What speeds up, what still falls back to the JVM, and how to deploy it....
FSST and ALP: The Two Encodings Fixing Parquet's Weakest Compression Cases
ALP and FSST target Parquet's worst cases: floats and high-cardinality strings. How they work and what they change for Iceberg tables....
Governance-as-Code for the Lakehouse: Managing REST Catalog RBAC and Masking in Git
Put REST catalog RBAC and masking in Git. How to review grants, apply them safely, and keep lakehouse access from drifting....
Metric Contracts in Code: Testing, Versioning, and Serving Business Logic to Multi-Agent Systems
Metric contracts in code let teams test, version, and serve business logic to multi-agent systems without each agent inventing its own SQL....
High-Throughput Branch Merging: Automating Concurrency and Conflict Resolution in Multi-Branch Iceberg Pipelines
High-throughput Iceberg branch merges need conflict detection and automation. How to reconcile concurrent writes without stalling pipelines....
Multi-Cloud REST Catalog Topologies: Running Apache Polaris Across AWS, Azure, and GCP
Polaris can catalog Iceberg tables across AWS, Azure, and GCP. Four topologies, credential vending, and the tradeoffs of each design....
Parquet-Only Manifests in Iceberg v4: Why the Metadata Layer Is Going Columnar
Iceberg v4 is moving manifests from Avro to Parquet so planners can read only the stats they need. Why the metadata layer is going columnar....