Every data leader wants open data. Fewer actually have it. Many cross into a new kind of lock-in at the very moment they think they have escaped the old one.

Over the past decade, the industry has chased interoperability through three waves. Each wave solved a real problem. Each also created the next control point. Understanding that pattern is the difference between a lakehouse strategy that can travel with you and one that only looks portable on a slide.

Three eras of lock-in

In the first era, the lock-in was the engine.

Hadoop promised an escape from proprietary warehouses, but many enterprises became dependent on specific managed distributions, operational tooling, and platform expertise. Cloud data warehouses then transformed analytics with speed, elasticity, and simplicity. They improved time-to-insight, but they also tightly integrated storage, compute, metadata, optimization, and governance. That integration was valuable. It also made migration a major undertaking. 

In the second era, the lock-in shifted to the format.

Storage and compute decoupled. Object storage became the durable system of record. Parquet won as the dominant analytical file format. Then Apache Iceberg emerged as a table format: a metadata layer that gives files in object storage database-like behavior, including ACID transactions, schema evolution, partition evolution, time travel, and safer multi-engine access.

That was a real breakthrough. The promise that “the data is yours, in your bucket, in an open format” became credible.

We are now firmly in the third era. The lock-in has moved up the stack, into the catalog and the operational layer around it. That is where the conversation gets uncomfortable.

What Iceberg actually solved

Credit where it is due: Apache Iceberg is the real deal, and it made the lakehouse real.

Parquet defines how data is stored in files. Iceberg defines how those files behave as a table. A catalog tells engines where the current table metadata lives and, increasingly, serves as the foundation for access control, credential vending, discovery, auditability, and governance.

Iceberg gave data teams a versioned, transactional table abstraction on object storage that multiple engines can read and write, at least in principle. That matters because enterprises rarely run one engine. One team may use Spark for pipelines, another Trino for interactive SQL, another Flink for streaming, and another managed warehouse or lakehouse platform for BI and machine learning.

For a Chief Data Officer (CDO), the implication is direct: architecture review cannot treat engine support as static. It changes release by release, and your team needs a living matrix where engine capabilities and workload requirements are evaluated together.

That is not a failure of Iceberg. It is the reality of an ecosystem moving fast. It does mean that interoperability must be evaluated at the feature level, not the logo level.

If your workload is analytical, large-scale, append-heavy, multi-engine, and benefits from time travel or open storage, Iceberg is often close to the right answer. That sentence is doing a lot of work, and many adoption decisions skip past it.

The fine print no one puts on the slide

Not every workload benefits.

Small datasets, sub-second serving, simple single-engine pipelines, and high-throughput transactional patterns may be worse off on Iceberg than on a purpose-built system. The honest question before any migration is not “Can this run on Iceberg?” It is “What materially improves if this runs on Iceberg?”

Engine support is also uneven and asymmetric. A platform may support Iceberg reads but not writes. It may support v2 tables but not newer v3 features. It may handle schema evolution well but behave differently on deletes, merges, branching, complex types, or metadata tables. Older readers may not understand newer table features. That makes compatibility a workload-level question, not a checkbox.

Beyond engine compatibility, open table formats carry real operational costs. When self-managed, tables need compaction, snapshot expirations, and metadata cleanup. Without that work, performance quietly degrades, and storage costs grow. Open formats reduce one kind of dependency. They do not remove the need for ongoing table operations. 

And then there is the catalog.

The catalog is where lock-in lives now (for now)

The Iceberg REST Catalog specification is one of the most important developments in open lakehouse interoperability. It gives engines a common API for interacting with catalogs, instead of requiring every engine to implement every catalog-specific integration.

That is meaningful progress. Apache Polaris has graduated to a top-level Apache project. Nessie, Gravitino, and other open catalog efforts provide credible self-managed paths. Managed platforms are also exposing Iceberg REST endpoints or Iceberg-compatible catalog services. The direction of travel is clear: the catalog API layer is converging.

But speaking the same API is not the same as being interchangeable.

Each catalog has opinions. 

• Who owns the files? 
• Who can create tables? 
• Who vends credentials? 
• Where are permissions enforced? 
• How are masking rules represented? 
• How is lineage captured? 
• Who manages deletes, compaction, retention, and orphan-file cleanup? 
• What happens when a table moves? 
• What happens when the catalog changes?

These are not small details. They are the operating model.

That is the question worth asking in every catalog evaluation: not “Does it speak the API?” but “What decisions does it make for us, and can we reverse them?”

Some catalogs expect to own the file lifecycle. Others sit alongside storage you manage yourself. Some integrate deeply with a platform’s governance layer. Others focus more narrowly on table discovery and metadata. Some expose open APIs while keeping advanced governance capabilities inside the platform boundary.

If your access controls, compliance rules, and data masking cannot move with the data, you are still locked in – regardless of how open the underlying files are.

That is the deepest lock-in vector in the modern lakehouse stack, and it is still under-discussed.

What true interoperability would require

If we use the word seriously, true interoperability means more than open files.

• It means read and write parity across the engines you actually use, for the features your workloads actually depend on.
• It means catalog portability: the ability to move table metadata between catalog implementations without rebuilding the entire operational layer.
• It means governance that travels: access policies, masking rules, lineage, and audit controls that are portable, not re-implemented platform by platform.
• It means operational ownership without operational burden: open enough to leave, but manageable enough to stay.

Today, the industry delivers the first requirement reasonably well and the second unevenly. The third is improving. The fourth remains the hardest.

That is why “we use Iceberg” is not the same as “we are interoperable.”

The pragmatic path

The honest reframing for data leaders is this: the right question is not “Are we open?” Rather, it is “Which lock-ins are we choosing, and are they the ones we would choose on purpose?”

There is no architecture with zero lock-in. Self-managing Iceberg on object storage with your own catalog gives you maximum optionality, but it also creates real engineering responsibility. You own table maintenance, compaction strategy, metadata hygiene, engine-version compatibility, governance integration, and operational troubleshooting.

A managed Iceberg service makes a different trade-off. It may reduce operational burden, improve performance, simplify governance, and help teams move faster. In return, you may accept platform-specific behavior in catalog management, security, optimization, and lifecycle operations.

That is not automatically bad. Sometimes “buy” beats “build” because velocity matters. In fact, managed Iceberg is often a better trade than the warehouse lock-in of a decade ago. Your data can still live in an open table format. Other engines may still be able to read it. Migration may be hard, but it does not have to be catastrophic.

The goal is not to avoid every dependency. The goal is to know which dependencies are strategic, which are acceptable, and which would hurt if the business needed to change direction.

A closing observation

The encouraging truth is that the industry is moving in the right direction. Open table formats are real. REST catalog convergence is real. Managed Iceberg services are reducing operational burden. Vendors have also learned that credible openness is now a product feature, not a philosophical extra.

The discouraging truth is that “open” has become a marketing surface area. The hardest interoperability work is less glamorous: governance portability, catalog substitutability, consistent engine behavior, metadata scalability, and operational controls that travel with the data.

The teams that benefit most over the next five years will not be the ones that simply ask, “Are we open?”

They will be the ones that ask, “Which lock-ins are we choosing, which ones are we reducing, and which ones would hurt us if we had to leave?”

CDO strategy checklist: Questions for architecture review

Before approving further investments in lakehouse interoperability, challenge your data architecture teams with these four key questions:

• Governance Portability: If we migrate our primary data platform five years from now, do our security policies and compliance audit trails travel seamlessly with the storage layer, or will they require a complete ground-up rebuild?
• Operational TCO: Do we have the engineering depth to manage open-source table optimization, or will the hidden operational costs offset the vendor savings we expect?
• Workload Asymmetry: Which specific workloads genuinely require multi-engine write capabilities, and where are we introducing unnecessary architectural complexity for limited practical benefit?
• Intentional Dependencies: Where are we consciously accepting platform-specific capabilities to accelerate time-to-market, and do we have a documented plan for what happens if that vendor’s roadmap diverges from our long-term goals?