126 to 150 of 304 Apache Jobs in England

integration, and production support. Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing. Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE. Experience with data formats such as JSON, CSV, Avro, and Parquet. Working knowledge of Git, automated testing … observability, and release management. Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency. Preferred Skills Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks. Exposure to Terraform or another infrastructure-as-code tool. Knowledge of Dataplex ...

model: Hybrid 3 days on site each week Rate: up to £530 per day --- Responsibilities Design, implement, and maintain high-throughput data pipelines using Apache Spark and cloud-native tools Develop ETL/ELT workflows for managing structured and unstructured data in AWS cloud environments Collaborate with AI/… analytics enablement --- Requirements (Must Have Skills) 5+ years of data engineering experience building high-performing and large-scale data solutions Hands-on expertise in Apache Spark development using Java or Scala (Python experience considered) Strong proficiency in AWS cloud services (S3, EMR, Glue, IAM, etc.) and cloud-native architecture ...

services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great ...

including target modelling, cutover approach, reconciliation strategy and technical risk management. Design and build the real‐time data architecture using Amazon MSK/Apache Kafka and PySpark Structured Streaming (evaluating Apache Flink where it is the better fit), including event schema governance, exactly‐once semantics and stream … large‐scale batch and streaming processing, and strong command of data modelling and data warehousing concepts. Production experience designing and operating streaming architectures with Apache Kafka or Amazon MSK, including stream processing with Spark Structured Streaming. Hands‐on experience with open table formats, such as Apache Iceberg ...

unify data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits Databricks strives to provide comprehensive benefits and perks. For specific details in your region, refer to the regional ...

infrastructure. Well versed and proven experience in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast‐paced environment. Excellent communication and collaboration skills. Longstanding experience ...

hybrid) across large structured or semi-structured datasets. Hands-on experience with data lake, warehouse or lakehouse architectures at scale: Delta Lake or Apache Iceberg, schema evolution patterns, partition pruning, and the trade-offs between query performance and storage cost. Infrastructure fluency on AWS with Kubernetes or ECS. ...

infrastructure. Well versed and proven experience in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem‐solving skills and ability to work in a fast‐paced environment. Excellent communication and collaboration skills. Longstanding experience ...

learning models using frameworks such as PyTorch or TensorFlow.Experience with statistical analysis, experimental design, or building data pipelines using platforms such as Databricks or Apache Spark.Experience using AI-assisted software development tools, such as Claude Code, GitHub Copilot, Databricks AI Assistant, or comparable tools.Experience building agentic workflows, multi-agent ...

learning models using frameworks such as PyTorch or TensorFlow.Experience with statistical analysis, experimental design, or building data pipelines using platforms such as Databricks or Apache Spark.Experience using AI-assisted software development tools, such as Claude Code, GitHub Copilot, Databricks AI Assistant, or comparable tools.Experience building agentic workflows, multi-agent ...

automated testing, deployment pipelines, and version control in a data engineering contextKnowledge of data modelling concepts and pipeline orchestration patterns beyond Databricks Workflows (e.g. Apache Airflow)Prior experience working in large-scale Hadoop to cloud migration programmes or legacy platform modernisation projectsFamiliarity with streaming data processing (e.g. Apache ...

product managers, to understand data requirements and deliver high-quality solutions Architect and implement data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery Mentor and guide junior data … Engineering, or a related field 7+ years of experience in data engineering, with at least 3 years in a leadership role Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools Experience with cloud platforms such as Azure, AWS, or GCP Solid understanding of data modeling ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based pipelines … production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys & Apache Airflow. Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with AWS S3 for large-scale data storage. ...

13.x, REST API design* Data: MySQL (schema design, indexing, query optimization, replication/backup strategies)* Platform: Linux (Ubuntu), Docker, web server tuning (Nginx/Apache)* Cloud: AWS EC2, S3, Secrets Manager (plus IAM, VPC, security groups)* CI/CD: Jenkins, GitHub Actions* Testing: Phpunit, unit/integration/ ...

Glass UK seeks a hands-on Technical Delivery Lead to drive engineering strategy for a scalable multi-tenant data integration platform built on Apache Airflow and Google Cloud. You will lead the delivery of production-grade ETL/ELT pipelines and cloud solutions, collaborating with cross-functional teams ...

TOAD. AWS Data Stack: Strong working knowledge of the AWS cloud data ecosystem, specifically requiring experience with S3, AWS Glue, PySpark, EMR, and Apache Iceberg. Architecture & Design : Ability to translate complex business requirements into high-level and low-level technical designs (HLD/LLD), ensuring data quality and governance. ...

TOAD. AWS Data Stack: Strong working knowledge of the AWS cloud data ecosystem, specifically requiring experience with S3, AWS Glue, PySpark, EMR, and Apache Iceberg. Architecture & Design : Ability to translate complex business requirements into high-level and low-level technical designs (HLD/LLD), ensuring data quality and governance. ...

Experience Strong hands-on experience administering Linux environments, ideally Ubuntu. Good knowledge of web technologies and LAMP/LEMP architectures. Experience configuring and supporting Apache or Nginx web servers. Strong understanding of PHP and MySQL environments. Knowledge of web technologies and authoring BASH/Shell scripts to run unattended ...

Experience • Strong hands-on experience administering Linux environments, ideally Ubuntu. • Good knowledge of web technologies and LAMP/LEMP architectures. • Experience configuring and supporting Apache or Nginx web servers. • Strong understanding of PHP and MySQL environments. • Knowledge of web technologies and authoring BASH/Shell scripts to run unattended ...

proactive mindset and passion for improving reliability, resilience and operational efficiency Experience with Java and Spring Boot or .NET application support. Knowledge of Apache Artemis, Red Hat AMQ, IBM MQ or similar messaging technologies. Experience with Ansible, StackStorm or other configuration and workflow automation tools. Experience with infrastructure ...

proactive mindset and passion for improving reliability, resilience and operational efficiency Experience with Java and Spring Boot or .NET application support. Knowledge of Apache Artemis, Red Hat AMQ, IBM MQ or similar messaging technologies. Experience with Ansible, StackStorm or other configuration and workflow automation tools. Experience with infrastructure ...

required, including 5 years' UK residency; successfully obtaining security clearance is a condition of employment These will help you succeed Strong working knowledge of Apache Iceberg and open table formats, with the ability to architect interoperable strategies that avoid vendor lock-in Practical experience designing or transitioning organisations towards ...

accuracy, reliability, and performance of data pipelines and integrations across our data platform. The role will focus on testing Snowflake, SQL-based data transformations, Apache Flink pipelines, and end-to-end data integrations. The successful candidate will work closely with Data Engineers, Software Engineers, and other stakeholders to design ...