4 of 4 Contract Apache Spark Jobs in the North West

data-based insights, collaborating closely with stakeholders. Passionately discover hidden solutions in large datasets to enhance business outcomes. Design, develop, and maintain data processing pipelines using Cloudera technologies, including Apache Hadoop, Apache Spark, Apache Hive, and Python. Collaborate with data engineers and scientists to translate data requirements into technical specifications. Develop and maintain frameworks for efficient More ❯

focused data team responsible for building and optimising scalable, production-grade data pipelines and infrastructure. Key Responsibilities: Design and implement robust, scalable ETL/ELT pipelines using Databricks and Apache Spark Ingest, transform, and manage large volumes of data from diverse sources Collaborate with analysts, data scientists, and business stakeholders to deliver clean, accessible datasets Ensure high performance … practices Work with cloud-native tools and services (preferably Azure ) Required Skills & Experience: Proven experience as a Data Engineer on cloud-based projects Strong hands-on skills with Databricks , Apache Spark , and Python or Scala Proficient in SQL and working with large-scale data environments Experience with Delta Lake , Azure Data Lake , or similar technologies Familiarity with version More ❯

to support data needs Optimise data storage and retrieval for performance Work with batch and real-time processing frameworks Implement and manage ETL processes Use tools like Python, SQL, Spark, Airflow, Kafka, dbt, Snowflake, Redshift, BigQuery Requirements Bachelors degree in Computer Science, Engineering, or a related field 1+ year in a Data Engineer or similar role Proficiency in SQL More ❯

on Databricks Data Platform & Optimization Design and optimize Databricks clusters for performance, cost management, and resource utilization Implement medallion architecture (Bronze, Silver, Gold) for data processing and transformation Optimize Spark jobs and queries for maximum performance and cost efficiency Monitor and troubleshoot data pipeline performance issues Collaboration & Best Practices Work closely with data scientists, analysts, and business stakeholders to … Airflow, or similar) Understanding of Azure security best practices, RBAC, and data encryption Streaming & Real-Time Processing Knowledge of real-time data processing using Azure Event Hubs, Kafka, and Spark Streaming Experience with event-driven architectures and streaming data pipelines Data Governance Understanding of data governance frameworks, Unity Catalog, and data quality practices Experience with data lineage, cataloguing, and More ❯