Data Engineering: Systems That Stay Up

Most Medallion explainers say what each layer does, never why it exists. The answer: Silver is the modern Operational Data Store.

Is the 40-year-old Kimball dimensional model compatible with modern Materialized Lake Views? One Fabric engineer’s deep dive into late-arriving dimension corrections, incremental refresh limits, and why the real optimisation isn’t faster refreshes — it’s smaller ones.

We’ve all been there: it starts as a simple proof-of-concept. You open up a new Notebook in Microsoft Fabric, write...

A few years ago, I was testing an Azure Function that pulled secrets from Key Vault. It worked flawlessly on...

A few months ago, I was minding my own business—tuning data pipelines, optimizing our queries, and doing the usual data...

Data pipelines are the unsung heroes of the analytics world. When they work, insights flow; when they falter, the entire...

The Permission Paradox Picture this: I’m sitting in yet another emergency meeting because someone can’t access the resources they need...

Let me tell you a story about a data architecture that was slowly turning into a money pit. You know...