Data Engineering Consultant - Pyspark
- Location
- City Of London, England, United Kingdom
works. Experience with cloud platforms (e.g., AWS, Azure, GCP) and data lake architectures. Strong experience and programming skills in languages such as Python and SQL and the ability to write complex SQL queries. Use of GitHub and CI/CD practices. Support development of the Azure Databricks Lakehouse platform, shaping … frameworks and solutions that other engineering teams will adopt in future data projects. Build, optimise, and maintain data processing frameworks using Python, ensuring performance, scalability, and maintainability. Support DBT integration and best practices for transformation pipelines within Databricks. Apply software engineering principles including: Full development lifecycle management Source control, automated ...