5 of 5 Apache Spark Jobs in the West Midlands

/CD platform (Github Actions & Admin), Password/Secret management (hashicorp vault). Strong Data related programming skills SQL/Python/Spark/Scala. Database technologies in relation to Data Warehousing/Data Lake/Lake housing patterns and relevant experience when handling structured and non-structured data ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Gen2). Optimize data storage formats (Delta, Parquet, Avro, ORC) for performance and cost. Maintain and optimize Synapse SQL Pools, Dedicated SQL Pools, and Spark Pools. Implement and monitor data quality rules, validation frameworks, and error handling. Ensure compliance with data governance, security, RBAC, and privacy standards. Work with … years of professional experience in data Engineer or a related field, with a proven track record of delivering impactful solutions. SQL Pools, serverless SQL, Spark Pools Strong SQL, performance tuning, query optimization Data modeling & warehouse concepts (Kimball/Inmon) Pipelines, triggers, linked services, integration runtime Data flows & orchestration ...

engineering with stakeholder engagement and solution design. Key Responsibilities Build and optimise data pipelines, ETL processes, and data platforms Develop solutions using Python, Spark, Kafka, Hadoop or similar Wo click apply for full job details ...

engineering with stakeholder engagement and solution design. Key Responsibilities Build and optimise data pipelines, ETL processes, and data platforms Develop solutions using Python, Spark, Kafka, Hadoop or similar Work in Agile teams to deliver production-ready systems Translate business requirements into technical data solutions Engage with stakeholders and communicate …/CD Skills & Experience Experience as a Data Engineer/Big Data Engineer Strong coding in Python, Scala or Java Hands-on with Spark, Kafka, ETL/data pipelines Knowledge of cloud platforms (AWS, Azure or GCP) Familiar with Agile and software engineering best practices Strong communication/stakeholder ...