I build systems, then pull them apart
I build data systems for a living. Then I go home and build more systems for fun.
This site is where I write about what I discover after asking, “There has to be a better way to do this.”
Most posts begin the same way: I build something, break something, question an assumption, and eventually find a simpler way forward. What survives becomes a field note.
What is the database doing when that query slows down?
What did the cloud service abstract away?
Why did this architecture become difficult to change?
I’ve always wanted to know what’s underneath
I’ve never been comfortable treating technology as magic. That curiosity took me from hardware and networking into software, databases, cloud platforms, and eventually data architecture.
It also explains the Proxmox server, containers, firewalls, and self-hosted services at home, even when paying someone else ten dollars a month would be considerably easier.
Sometimes understanding how something works is more interesting than taking the easiest path.
The rabbit holes
A surprising amount of my career can be traced back to one innocent question. The destination is rarely where I expected.
Why is this SQL database costing so much?
SQL Server → object storage → Parquet → Delta Lake → Fabric
What stuck: a cost problem can reveal an architecture that is fighting its own growth. Design for change, not just today’s bill.
Surely a notebook doesn’t need software engineering practices?
Helper functions → business logic → tests → dev containers → AI agents
What stuck: experiments quietly become systems. Recognising that transition early makes them much less frightening to change.
Why do I need all these moving parts?
Credentials → linked services → storage → compute → pipelines → permissions
What stuck: plumbing has an operating cost. Sometimes removing a component is more valuable than adding a clever one.
Could I just host this myself?
One service → Docker → Proxmox → DNS → firewalls → backups
What stuck: self-hosting leaves nowhere for complexity to hide. When it breaks, there is no managed-service team underneath it. There is just you.
Can AI actually make me more productive?
Code generation → context → tools → tests → guardrails → autonomy
What stuck: generating code became easy. Deciding what context an agent needs, what it may touch, and how its work is verified is the real engineering problem.
A few things I believe
These are the ideas I keep returning to after enough migrations, production incidents, and architecture diagrams.
Simple beats clever
A brilliant architecture nobody understands has a short honeymoon. Complexity should earn its place because somebody eventually has to operate it.
Architecture is trade-offs
There is rarely one universally correct platform or pattern. Understanding the context and trade-offs matters more than memorising best practices.
Look under the abstraction
I happily use managed services. I also want to know what they manage for me, and where the hidden complexity will surface when something goes wrong.
Prototype early
Ugly working prototypes expose more truth than weeks of theoretical design. Gaps are inevitable; the useful part is making them visible early.
Technology is often the easy part
Projects usually struggle because assumptions were wrong, ownership was unclear, or interfaces were misunderstood. It is rarely because of one Spark setting.
AI still needs systems thinking
AI can generate the wrong code extraordinarily quickly. The advantage is knowing what to build, which constraints matter, and how to verify the result.
I write about what survived contact with reality
I don’t want this to become another site that rewrites product documentation. There is plenty of documentation already.
Most posts start with something I met while building a real system: an architecture that did not scale, a migration assumption that failed, a feature I wanted to test properly, or an AI workflow that worked better in theory than in practice.
- Build the smallest useful version
- Break it somewhere safe
- Measure what actually happened
- Change my mind when the evidence says to
- Write down the part worth carrying forward
Away from the keyboard
Sydney · home lab · football · family
These days I’m also learning that a young child may be the ultimate distributed system: unpredictable workloads, limited observability, frequent alerts, and absolutely no SLA.
Somehow, I wouldn’t have it any other way.
Welcome to the rabbit hole
If you’re interested in data engineering, architecture, Microsoft Fabric, cloud platforms, AI-assisted development, or simply why systems behave the way they do, you’ll probably find something here.
I don’t pretend to have all the answers. I enjoy pulling systems apart until I understand them a little better, then write down what I find.
Read the field notes