Data Engineer
- Location
- Sheffield, England, United Kingdom
Responsibilities: Configure data ingestion processes and solutions. Lead the execution of Microsoft Fabric migration activities. Lead and guide migration engineers and QA resources. Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks. Create data integration test cases and support testing activities. Implement and amend data … file formats and partitioning, including Parquet, JSON, XML and CSV. Experience with Azure DevOps/GitHub and CI/CD. Strong programming experience in Python, with knowledge of other programming languages beneficial. Experience with both synchronous and asynchronous interface approaches. Knowledge of data modelling, including 3NF, dimensional modelling and Data ...