James Dzidek

I'm focused on data platform architecture and pipeline orchestration at enterprise scale.

My background spans two decades across consulting, enterprise architecture, and data engineering. I spent nearly five years as a Digital Architect and Advisor at Microsoft Consulting Services, driving cloud transformations for some of Canada's largest organisations across financial services, manufacturing, and retail. Before that, I served as Lead Data Architect at the Ontario Cannabis Store, where I built their data analytics platform from the ground up on Azure. Earlier in my career I worked as an Enterprise Architect at DNB, Norway's largest financial institution, and as a consultant at IBM.

I hold a PhD in Empirical Software Engineering from the University of Oslo, with my main research published in IEEE Transactions on Software Engineering — the field's most prestigious journal.

Today my work centers on Dagster and Databricks implementations inside large enterprises — typically on Azure, with Unity Catalog, ADLS, and medallion architectures. I also work with LLMs, Databricks, and Kubernetes, helping teams operationalise AI workloads and containerised data infrastructure at scale. I write here about orchestration maturity, asset-centric pipeline design, and the practical realities of building data platforms that engineering teams can actually trust.

If something here resonates with a problem you're working through, I'd be glad to talk. Get in touch.