Martin's Blog

Building a framework for orchestration in Azure Data Factory: Framework components

Before we dive into the details of the Data Factory pipelines, it is worth explaining the conceptual structure of my framework and its components. How it all fits together is important, and after reading the post on the metadata and this one the pieces of the puzzle will hopefully start falling into place…

Building a framework for orchestration in Azure Data Factory: Metadata

The metadata that drives the execution within a framework is probably the most critical part. Going back to our analogy of building a house, the metadata would be the foundation. It is here where you are going to make some architectural decisions outside of which the framework cannot operate…

Building a framework for orchestration in Azure Data Factory: Orchestration

There is no shortage of opinions around “best practices” when it comes to the development of ETL processes.

I have chosen the path of orchestration, and this blog post describes what that means and why I think it’s a good choice for cloud-based ETL.