Microsoft Fabric Data Engineer
Experience: 5-10 Years (2-4 years of hands-on Microsoft Fabric experience preferred)
Job Summary
We are looking for an experienced Microsoft Fabric Data Engineer to design, develop, implement, and maintain scalable data engineering solutions using Microsoft Fabric. The ideal candidate should have strong expertise in data integration, ETL/ELT pipelines, Lakehouse architecture, Data Warehouse development, and Azure data services.
The candidate will be responsible for building robust data pipelines, transforming large datasets, optimising data processing, and enabling analytics and reporting solutions.
Key Responsibilities
-
Design, develop, and maintain end-to-end data pipelines using Microsoft Fabric.
-
Build and manage Lakehouse and Data Warehouse solutions.
-
Develop ETL/ELT processes using Fabric Data Factory.
-
Work extensively with Fabric Notebooks using PySpark and Python.
-
Implement data transformations using SQL, Python, and Spark.
-
Work with OneLake for centralised data storage and management.
-
Develop and optimise Delta Lake tables.
-
Implement data ingestion from multiple sources, including APIs, databases, and cloud platforms.
-
Work with Fabric Dataflows Gen2 and Pipelines.
-
Implement medallion architecture (Bronze, Silver, Gold).
-
Perform data modelling, data cleansing, and data quality checks.
-
Optimise performance, scalability, and cost of data solutions.
-
Implement security, governance, and access controls.
-
Collaborate with BI developers, data architects, and business stakeholders.
Mandatory Technical Skills
Skill
Required expertise
Microsoft Fabric
Lakehouse, OneLake, Data Factory
PySpark
Data transformation and processing
Python
Data engineering and automation
SQL
Complex queries, stored procedures
Fabric Pipelines
ETL/ELT development
Delta Lake
Delta tables and optimisation
Dataflows Gen2
Data ingestion and transformation
Data Modelling
Dimensional modelling
Azure
Azure Data Lake, Azure SQL, ADF
Medallion Architecture
Bronze, Silver, Gold layers
Good-to-Have Skills
-
Power BI and semantic models.
-
Microsoft Fabric Real Time Intelligence.
-
Eventhouse and Eventstreams.
-
Azure Databricks.
-
Azure Synapse Analytics.
-
Microsoft Purview and data governance.
-
CI/CD using Azure DevOps or GitHub.
-
Spark performance tuning.
-
Experience with enterprise-scale data platforms.