Materialized Lake Views Meet Kimball: Where Incremental Refresh Falls Apart

I was sitting in front of my Microsoft Fabric workspace, watching a Materialized Lake View (MLV) refresh complete in just 47 seconds. The source table had more than two billion rows. Power BI reports that previously took tens of seconds now loaded almost instantly.

Pre-computing expensive joins and aggregations over Delta Lake clearly worked. I remember thinking: this could become the default acceleration layer for every Kimball warehouse I build.

Then the business called.

That customer you marked as Sydney for January? They were actually living in Canberra until April. Can you fix it?

“Sure,” I said. “That’s just a Type 2 dimension correction.”

That seemingly ordinary request ended up exposing one of the most interesting architectural tensions I’ve encountered in Microsoft Fabric. Can a forty-year-old dimensional modelling methodology coexist with a modern incremental query acceleration engine?

After several weeks of experimenting with Materialized Lake Views, I came away with an unexpected conclusion. The biggest optimisation wasn’t Materialized Lake Views — it was reducing the amount of data that needed to change in the first place.

Two Great Ideas That Pull in Different Directions

Kimball dimensional modelling and Materialized Lake Views both solve real problems. Kimball gives us historical accuracy. Materialized Lake Views give us speed. Individually, they’re fantastic. Together, they expose a subtle trade-off that only becomes obvious once your warehouse starts receiving late-arriving corrections.

A Quick Refresher on Type 2 Dimensions

If you’ve worked in enterprise data warehousing, you’ve almost certainly implemented Slowly Changing Dimensions (SCD Type 2). The concept is straightforward: instead of overwriting history, we preserve it. When a customer moves from Sydney to Brisbane, we don’t modify the existing record — we close it and create a new one.

Every fact row continues to resolve against the version of the customer that existed when the transaction occurred. Historical reports remain historically correct. This pattern has been the backbone of dimensional modelling for decades.

Why Materialized Lake Views Are So Exciting

Without an MLV, every analytical query repeatedly performs the same expensive work: join large fact tables, resolve surrogate keys, apply temporal predicates, aggregate billions of rows. Every dashboard repeats exactly the same computation.

Materialized Lake Views let Fabric perform that work once and persist the result as Delta data. Here’s a simplified example:

CREATE OR REPLACE MATERIALIZED LAKE VIEW gold.product_sales_summary
(
    CONSTRAINT valid_quantity CHECK (quantity > 0) ON MISMATCH DROP
)
COMMENT "Aggregated product sales"
AS

SELECT
    ...

Combined with Change Data Feed (CDF), Fabric can often maintain these materialised results incrementally as new data arrives. For append-only workloads, this is incredibly powerful. After seeing refreshes complete in under a minute over multi-billion-row datasets, I started wondering whether Materialized Lake Views could become the default optimisation layer for dimensional models.

Then I introduced a late-arriving dimension.

Where Incremental Maintenance Starts to Break Down

Type 2 itself isn’t the difficult part. Late-arriving dimension changes are.

Imagine the business discovers months later that the customer wasn’t actually living in Sydney between January and March. They were in Canberra. The original timeline is wrong. Kimball tells us to preserve history by correcting it. That typically means closing existing validity ranges, inserting corrected versions, and re-resolving affected fact rows.

The resulting timeline becomes:

For a Kimball warehouse, that’s completely normal. For an engine optimised around incremental maintenance, it’s a much harder problem.

Microsoft’s documentation makes this behaviour explicit:

Incremental refresh applies only when source data remains append-only during the refresh cycle. If updates or deletes are detected, Materialized Lake Views fall back to a full refresh — even when Change Data Feed is enabled.

That sentence completely changed how I thought about the architecture. The problem wasn’t query performance anymore. It was the cost of rebuilding history.

My First Attempt: Reducing the Scope of Change

Like most engineers, my instinct wasn’t to rebuild everything. It was to ask: how little can I rebuild?

If only customer C001 changed, and only between January and March, why would I ever recompute an entire Gold table containing billions of rows? I partitioned the Gold layer by month. When a historical correction arrived, I identified the affected business keys and rebuilt only the impacted partitions.

Late-arriving correction? Only rebuild the affected months — everything else untouched. This dramatically reduced the amount of work required to rebuild the warehouse.

Unfortunately, it didn’t change how Materialized Lake Views behaved. From the MLV engine’s perspective, rebuilding part of the source table is still a delete-and-insert operation in the Delta logs – that’s how replaceWhere works under the hood, even when you\’re only touching specific partitions. That still trips the full refresh fallback.

I had reduced the cost of rebuilding the Gold layer. I hadn’t reduced the cost of maintaining the Materialized Lake View. That distinction turned out to be incredibly important.

The Real Optimisation Isn’t Faster Refreshes

At this point I realised I had been asking the wrong question. I was trying to make refreshes faster. The better question was: can I make refreshes smaller?

That’s a completely different optimisation. Instead of thinking in tables, start thinking in business impact. Which customer changed? Which dates are affected? Which Gold partitions reference those dates? Which downstream models need rebuilding?

Once you can answer those questions precisely, the amount of work drops dramatically.

Building Around the Limitation

This led me to a metadata-driven architecture that separates change detection from recomputation:

Figure: The metadata-driven architecture that separates change detection from recomputation. Impact analysis identifies only the affected slices before Materialized Lake Views enter the picture.

Notice what this architecture is optimising. It isn’t Materialized Lake Views — it’s the warehouse itself. Instead of rebuilding billions of rows because one customer changed, the pipeline rebuilds only the small business slice that is genuinely affected. The optimisation happens before Materialized Lake Views even enter the picture.

Does This Solve Materialized Lake Views?

Not entirely. Today’s Materialized Lake Views are exceptionally good at maintaining pre-computed query results when source tables evolve through append-only ingestion. Historical corrections are different — replacing portions of a source table still triggers a full refresh. That isn’t a criticism of MLVs. It’s simply a consequence of optimising for a different workload.

Fortunately, Microsoft has already indicated broader update support is on the roadmap. As those capabilities mature, this limitation may disappear. Until then, I’d treat Materialized Lake Views as an optimisation layer rather than the foundation of a mutable dimensional architecture.

What I Learned

I started this experiment wanting to understand whether Materialized Lake Views could accelerate a Kimball warehouse. I finished with a different conclusion.

The biggest performance gains didn’t come from Materialized Lake Views. They came from reducing the scope of change. Once I stopped thinking in terms of entire tables and started thinking in terms of affected business keys, date ranges and partitions, the architecture became simpler, more predictable and dramatically cheaper to operate.

Materialized Lake Views are an impressive capability, especially for analytical workloads that evolve primarily through new data arriving over time. Kimball dimensions remain one of the best ways to preserve historical accuracy. The challenge is recognising that these two ideas optimise for different things. One optimises for correctness. The other optimises for incremental maintenance.

As Fabric continues to evolve, I expect that gap to narrow. Until then, understanding where the tension exists is just as important as understanding the feature itself. After all, in enterprise data warehousing, the hardest problems rarely come from today’s data. They come from yesterday’s mistakes.