3 of 3 PySpark Jobs in the North East

with large-scale datasets using Azure Data Factory (ADF) and Databricks. Strong proficiency in SQL (T-SQL, Spark SQL) for data extraction, transformation, and optimisation. Proficiency in Azure Databricks (PySpark, Delta Lake, Spark SQL) for big data processing. Knowledge of data warehousing concepts and relational database design, particularly with Azure Synapse Analytics. Experience working with Delta Lake for schema … evolution, ACID transactions, and time travel in Databricks. Strong Python (PySpark) skills for big data processing and automation. Experience with Scala (optional but preferred for advanced Spark applications). Experience working with Databricks Workflows & Jobs for data orchestration. Strong knowledge of feature engineering and feature stores, particularly in Databricks Feature store for ML training and inference. Experience with data More ❯

solutions and enjoys working with modern tech in a collaborative environment. What You’ll Do: Design, build, and maintain scalable data pipelines Optimize and automate data workflows Work with PySpark, Python, and SQL to process and manage large datasets Collaborate in a cloud-based environment to deliver efficient and reliable data solutions What We’re Looking For: Proven experience … with Python, PySpark, and SQL Strong understanding of data engineering principles and cloud infrastructure Ability to work collaboratively and communicate technical concepts clearly A passion for clean, efficient, and scalable code Why Join Us? Supportive team environment with a strong focus on innovation Opportunities for career growth and development Flexible working arrangements Interested? Please send us your CV and More ❯

scientific discipline, backed by minimum A A A grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with Azure and Data Bricks You're collaborative with excellent More ❯