1 to 25 of 43 Permanent Apache Spark Jobs in the UK excluding London

Data Engineer Salary up to £60,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via chelsea.hackett@searchability.com ABOUT THE CLIENT Due to continued growth, we’re seeking a skilled Data … optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering ...

looking for: · 3+ years of professional Python development experience as a Data Platform Engineer, using AWS. · Experience developing and deploying Serverless technologies, provisioning Apache Airflow, and using infrastructure as code such as AWS CloudFormation or Terraform. · Takes a collaborative approach to problem solving and breaking down complex problems, whilst … with software engineering requirements and specifications. · Good interpersonal skills, positive can-do attitude and willing to help other members of the team. · Understanding of Apache Spark, Apache Trino or other big data and machine learning systems. · Exposure to using AI tools to enhance productivity and quality ...

NoSQL technologies Hands-on experience with at least one major cloud platform (Azure, AWS, or GCP) Experience with big data technologies such as Spark, Hadoop, or Kafka Proficiency with ETL/ELT tools and frameworks (e.g. Airflow, dbt, Informatica) Strong programming skills in Python, Java, Scala, or similar Experience ...

with Red Hat OpenShift and Ceph storage platforms Programming experience (e.g. Python, JavaScript) Experience working with event streaming and data pipeline technologies (e.g. Kafka, Apache NiFi, Apache Spark) Experience delivering multi-system integration projects within defence, government, or highly regulated environments Experience implementing and configuring training simulation ...

with Red Hat OpenShift and Ceph storage platforms Programming experience (e.g. Python, JavaScript) Experience working with event streaming and data pipeline technologies (e.g. Kafka, Apache NiFi, Apache Spark) Experience delivering multi-system integration projects within defence, government, or highly regulated environments Experience implementing and configuring training simulation ...

computer science, IT or a related field, or in bioinformatics with a demonstrated IT expertise Experience in big data technologies and frameworks, such as Apache Spark, Hadoop or similar platforms Hands-on experience with CI/CD (GitLab CI/GitHub Actions) Familiarity with Java Familiarity with Google ...

Strong programming experience in Java (preferred) or Python Experience building ETL/ELT or streaming data pipelines Hands-on experience with Kafka, Flink, or Spark (streaming experience preferred) Good understanding of stream-processing concepts (event time, state, backpressure) Experience with cloud platforms (AWS preferred; Azure or GCP also considered ...

data estate. Deep expertise in Microsoft Azure cloud services - ADF, ADLS, Synapse, Purview. Databricks - Delta Lake architecture, optimisation and advanced data processing. Apache Spark for large-scale distributed computing and performance tuning. Microsoft Fabric - OneLake and DirectLake integration. Azure Synapse Analytics for enterprise-scale data warehousing. Infrastructure ...

experience in data engineering or large-scale data platforms Strong programming skills in Java (preferred) or Python Hands-on experience with Kafka, Flink, or Spark, with a preference for streaming technologies Solid understanding of real-time data processing concepts such as event time, state handling, and backpressure Experience building ...

experience working with data warehouses. Solid understanding of ETL/ELT design and data pipeline optimisation. Experience with big data frameworks such as Apache Spark, and exposure to streaming technologies like Kafka. Strong problem-solving skills and the ability to collaborate effectively across teams. What's on offer ...

with cross-functional teams to deliver data-driven solutions Support data modelling, governance, and quality initiatives What were looking for Experience with Python, SQL, Spark (and ideally Scala) Hands-on delivery of ML solutions in production Exposure to cloud platforms (AWS, Azure, or GCP) Understanding of DevOps, containers ...

focus on production-quality code • Solid understanding of ETL processes and best practices • Experience with SQL/NoSQL databases and big data tools (e.g. Spark, Pulsar) • Hands-on experience with cloud platforms (AWS, Azure, or GCP) • Experience developing and deploying data services end-to-end • Strong stakeholder communication ...

focus on production-quality code • Solid understanding of ETL processes and best practices • Experience with SQL/NoSQL databases and big data tools (e.g. Spark, Pulsar) • Hands-on experience with cloud platforms (AWS, Azure, or GCP) • Experience developing and deploying data services end-to-end • Strong stakeholder communication ...

code. LLMOps & AI Model Management: Familiarity with tools like MLFlow, LangChain, Hugging Face, Kubeflow, or similar platforms. Data Processing: Working knowledge of Databricks/Spark or comparable large-scale data processing tools. SQL: Solid capabilities in data querying and preparation. Data Architectures: Understanding of modern data infrastructure (lakehouses, data ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

scalable cloud-based data solutions Experience working with complex, multi-system integrations Experience in regulated industries (e.g. healthcare) is a plus Desirable Databricks/Apache Spark Azure DevOps & CI/CD pipelines Microsoft Fabric/Synapse CRM integrations (e.g. Eclipse, CareLineLive) Power Platform (Power Apps/Automate) Data ...

followed Required Skills & Experience: Strong experience with Azure data platform (e.g., Data Factory, Data Lake, Synapse) Hands-on expertise with Databricks (PySpark or Spark) Solid SQL development skills Experience working with Kafka for streaming data pipelines Good understanding of lakehouse architecture, including bronze/silver/gold layering concepts ...

levels. Knowledge of industry‐standard data models and frameworks such as Lambda, Medallion, and Data Mesh. Experience with real‐time data processing (Kafka, Spark) and BI tools (Power BI, Tableau). Strong analytical and problem‐solving skills, translating business requirements into effective data solutions. Equal Opportunities & Reasonable Adjustments ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

translate business needs into technical roadmaps What were looking for Strong experience in data architecture & large-scale systems design Expertise across Python, SQL, Spark, Scala Deep knowledge of AI/ML frameworks and modern data platforms Experience with cloud-native architecture and DevOps (Kubernetes, Terraform, CI/CD) Proven ...

Ensure high standards of data quality, governance, security and accessibility across data platforms Develop scalable data transformation logic using Python, SQL and Spark-based frameworks Monitor pipeline performance and troubleshoot data workflows to ensure reliability and efficiency Work closely with data scientists, analysts and engineering teams to support advanced … data services (such as S3, Glue, Redshift, EMR, Athena, Lambda or similar) Strong programming experience with Python and SQL Hands-on experience with Spark or distributed data processing frameworks Proven experience building and maintaining data pipelines, data lakes and data warehouses Strong understanding of data modelling, metadata management ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...