DBT with Fabric
This is a step by step data warehousing tutorial for Fabric Data Warehouse with DBT lets get started with creation of Python Virtual Env python -m venv venv #activate the venv .\venv\Scripts\A
Latest articles
All DSA Problems
Find the element with the highest repeats in an array # Online Python compiler (interpreter) to run Python online maxFre = 0 maxEle = 0 arr = [5, 6, 5, 6, 9, 6] map = {} # iterate through arr and populate map {} tp track how many times something wa...
Simple trick to Bulk Insert in Slow Changing Dimension Task in SSIS
I am sure my readers have a very good understanding of how SSIS Data Flow Task “Slow Changing Dimension” works. The point of this blog is not to introduce you to slow changing dimension task rather to point out something interesting you can do with i...
Azure DevOps Automated CICD pipeline with SQL Server Git Integration
Recently I realised working with quite a few data teams across UK and US that many of these data teams still using VM hosted SQL Server and manually merging into main. To be honest, I helped the teams to get started with simple azure CICD build and r...
Don't hire DevOps Engineers for your cloud data pipelines.
Many DevOps engineers will hate me for saying these, but I believe, that having a background in data and from the experience of working with DevOps engineers trying to be data engineers, I definitely qualify to say what I am going to say. Data is the...
DBT for SQL Server? Yes possible.
Installing DBT using SQL server is extremely simple. I had many clients coming to me and asking if they could use DBT with SQL Server on-prem or managed instances. They look surprised when I answered them, "Yes, why not". Now you might ask me, what i...
Become the best user of Power BI : Regex
By the time you finish this tutorial, you will become one of the best POWER BI Wizards. You'll find yourself becoming quite the adept POWER BI RegEx user. Not only will you handle RegEx tasks with ease, but you'll also seamlessly integrate JavaScript...
Azure Project ETL Workshop: File Structure Validation, Slow Changing Dimension and Data Loading
Challenges: Building Dynamic Data Pipeline: Processing multiple files with different column names; Need a dynamic pipeline that can handle varying file structures. Maintaining file structure integrity: Need to validate file structures against a dat...
Get started with Azure Data Factory
Creating a resource group in Azure: Select "Create" or "Create Resource" and search for "resource group". Specify a name for the resource group (e.g., "project-ip"). Select a location (e.g., "UK West"). Optionally, add tags for the resource group...