1 of 1 Contract Data Transformation Jobs in Yorkshire

Job Title: Data Engineer (w. Prophecy) In this role you will: Design, develop, and optimize data pipelines using Azure Databricks, PySpark, and Prophesy. Implement and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Apache Airflow for orchestration. Develop and optimize complex SQL queries and … Python-based data transformation logic. Work with version control systems(GitHub, Azure DevOps) to manage code and deployment processes. Automate deployment of data pipelines using CI/CD practices in Azure DevOps. Ensure data quality, security, and compliance with best practices. Monitor and troubleshoot performance issues … in data pipelines. To be successful in this role you should meet the following requirements: Must have experience with Delta Lake and Lakehouse architecture. Proven experience in data engineering, working with Azure Databricks, PySpark, and SQL. Hands-on experience with Prophesy for data pipeline development. Proficiency in More ❯