Data Engineer

If you are an experienced Data Engineer with excellent communication skills and a proven track record of working on large scale data enablement projects, we have a new contract we would like to discuss with you.

Please note this role requires onsite attendance once a week and has been deemed inside IR35.

Requirement:

• Experience in Azure Synapse, ETL, Pyspark, SQL , data Modelling and Data Bricks.
• Design and develop Azure Pipelines including data transformation and data cleansing
• Document source-to-target mappings
• Re-engineer manual data flows to enable scaling and repeatable use
• Build accessible datasets for distribution and analysis
• Development of Azure Pipelines for transforming data
• A scalable meta-data driven ingestion and transformation framework
• Aligning transformation pipelines and datasets with Purview
• Experience of working with Azure Data Lakes, Data warehousing, and pipelines and Storage accounts
• Experience in building and managing Pipelines including: building data interfaces to source systems, combining and transforming data into appropriate storage formats
• Experience identifying and resolving issues in databases, data processes, data products and services.
• Proficiency in T-SQL and Python to develop automation