1 to 25 of 151 Apache Jobs in England

native solutions that support analytics, automation, and digital transformation. Data Engineering, Full Stack & Platform Development Design and implement scalable data pipelines using tools like Apache Spark, Airflow, or dbt.Build and maintain data lakes, warehouses, and real-time streaming solutions. Develop APIs and microservices to expose data securely and efficiently. ...

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms ...

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms ...

Python, Java or Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms ...

Python, Java or Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms ...

Python, Java or Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms ...

Qualifications 10–15 years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks. Expert in Apache Spark, PySpark, Databricks SQL, Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture, BI ecosystems, and modern data platforms. Cloud expertise in Azure ...

detail \n A proactive approach with the ability to own and deliver tasks \n Desirable: \n \n Experience with Azure Data Factory, Databricks or Apache Spark \n Familiarity with tools such as Fivetran \n Understanding of modern data architectures (e.g. medallion architecture) \n Exposure to CI/CD, Azure ...

strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. Essential qualifications • Extensive experience with Python and PySpark/Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

forums, architecture review boards, design authority meetings, and client stakeholder workshops. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. AWS Secrets Manager ...

principles Experience working within an SDLC with an integrated development team using CI/CD tooling including Git Desirable Skills Hands-on experience with Apache NiFi for data pipeline development and orchestration AWS experience including familiarity with EMR, S3 and SQS Awareness of domain-driven design approaches and architecture ...

evolution of the enterprise data estate, leveraging modern data platforms such as Databricks , Snowflake , Azure Synapse , and BigQuery . Oversee data engineering using Apache Airflow , dbt , and Prefect , ensuring data pipelines are performant, governed, and aligned with enterprise metadata standards ( Collibra , Alation , Microsoft Purview ). Drive the adoption ...

Lead Backend Software Engineer – Data Services Federated Data Services | Distributed Systems | Apache Spark | DataFusion | Python | Kubernetes Location: London (2 days per week onsite) Rate: £700–£800 per day (Outside IR35) Duration: Initial 6-month contract with likely extension Overview We're looking for an experienced Lead Backend Software Engineer … data services capable of securely querying and processing data across multiple environments. Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks. Design and implement Kubernetes-based services and containerised workloads. Build secure APIs and backend services ...

developing others. Desirable to have skills or experience in: - SAS Viya for data analytics. - Job Scheduling tooling (e.g. Berlin/JS7 Job Scheduler, TWS, Apache Airflow). - Denodo for data virtualisation. - Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). - ALM Tooling (Jira, Confluence). ...

Core Java Spring Boot frameworks Proven experience building distributed backend systems Hands-on experience with Kafka (or similar streaming platforms) Experience with Scala and Apache Spark for data processing Integration withbackend services such as: PostgreSQL Redis S3 (or equivalent object storage) Expertise in modern engineering practices, including: Continuous delivery ...

ecosystem •Infrastructure as Code using Terraform, Ansible, or CloudFormation •GitOps tooling such as Argo CD •Scripting (Python, Shell) •Container orchestration tools (e.g., Docker Swarm, Apache Mesos) •Service Mesh technologies (e.g., Istio, Linkerd) •Monitoring and observability tools (Prometheus, Grafana, ELK) •Applying infrastructure security, compliance, and governance practices •Working within agile ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/CD pipelines ...

Infra security, compliance and governance • Familiarity with CNCF ecosystem and building platforms on top of Kubernetes • Container orchestration tools, e.g. Docker Swarm/Apache Mesos • Knowledge of Service Mesh technologies, e.g. Istio/Linkerd • Experience of monitoring tools, e.g. Prometheus, Grafana, ELK • Rounded problem-solving skills - troubleshoot and resolve ...

tools such as Terraform Familiarity building and integrating with NoSQL database (familiarity with MongoDB is preferred) or SQL databases such as Postgres Familiarity. With Apache NiFI Familiarity with web standards, accessibility, and development best practice Bonus: Building microservices in C# using .Net Experience building solutions that integrate with LLMs ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly to stakeholders ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly to stakeholders ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly to stakeholders ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...