1 to 25 of 155 Apache Jobs in England

latency, high-throughput services for personalisationDeveloping Java/Spring Boot microservices with REST/gRPC APIsWorking with Kafka and event-driven architecturesBuilding pipelines using Apache Flink and Apache BeamIntegrating ML models and feature storesWorking with NoSQL databases and Redis cachingUsing Apache Camel for integration where requiredCollaborating with … event-driven systemsFamiliarity with stream processing conceptsKnowledge of distributed systemsStrong communication and teamwork skillsInterest in data-driven personalisation systems Nice to haveExperience with Apache Flink and Apache BeamExperience with KafkaFamiliarity with Apache CamelExperience with NoSQL databases (Cassandra, DynamoDB) and RedisCloud experience (GCP or AWS)Docker and KubernetesML ...

supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred)Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred)Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting ...

using Python, PySpark, and dbtBuild and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

data quality and consistency across platforms while complying with governance policies.What You Bring:Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs.Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB, Cosmos ...

development and automation capabilities, to improve the value realized by automationRequired qualifications, skills and capabilitiesExperience with building data pipelines on Databricks and/or Apache Spark in production.Strong coding skills in Python (PySpark) and SQL (Scala a plus).Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema ...

technologies as our stack evolves.Nice to HaveExperience with stream processing frameworks such as Flink or Spark Streaming.Experience with task orchestration tools such as Apache Airflow or similar systems.Familiarity with additional technologies: GraphQL, React, HTML5, JavaScript, CSS, Postgres, Gradle, BERT.Experience working with and designing infrastructures for large-scale data processing ...

expected to be up to 10% but may increase over time as the business evolves.Desired QualificationsExperience working with distributed data processing frameworks, including Apache Spark.Exposure to advanced analytics or AI‐adjacent data use cases, including preparing data for machine learning or intelligent applications.Familiarity with additional Azure services such ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. ...

Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark.• Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics).• Strong KDB experience ...

testing, and deployment.Deep understanding of Quantexa components such as:Entity Resolution (ER)Scoring FrameworkContextual Network GenerationUI ConfigurationStrong background in Big Data technologies, including:Scala, Apache Spark, Hadoop, ElasticSearchExperience with data modelling and design in the context of large-scale analytics platforms.Strong stakeholder management and communication skills – capable of interfacing ...

maintain scalable and robust data pipelines using SQL and Python, and leveraging any of the following: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark.Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms.Implement continuous integration and continuous deployment ...

design approaches.You understand where traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices ...

design data architectures that facilitate model training and inference.Data Governance Tools: Familiarity with data governance or data cataloguing tools (such as Collibra, Alation, or Apache Atlas) and lineage-tracking systems. Hands-on experience setting up or maintaining a data catalogue, documenting data definitions, or automating data lineage capture ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern data ...

QualificationsExperience in Data Platforms, Data Engineering, Cloud Engineering, or ML Platform Engineering roles, with exposure to Azure Strong hands-on experience with Python and Apache Spark Experience with Azure data platform technologies such as Azure Databricks, ADLS Gen2, and Unity Catalog Working knowledge of security and access management (RBAC ...

Learning (ML):• Deep understanding of machine learning principles, algorithms, and techniques.• Experience with popular ML frameworks and libraries like TensorFlow, PyTorch, scikit-learn, or Apache Spark.• Proficiency in data preprocessing, feature engineering, and model evaluation.• Knowledge of ML model deployment and serving strategies, including containerization and microservices.• Familiarity with ...

platformsAdvanced Python and SQL skills, with experience in modern engineering practices such as CI/CD and TDDExperience with big data frameworks such as Apache Spark or similar technologiesSolid understanding of data modelling, data warehousing, OLTP and data lake architecturesExperience working with cloud ecosystems such as AWS, Azure ...

pipelines.QualificationsHands-on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases.Expert-level proficiency in Python, SQL, and Apache Spark for data pipeline development.Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks.Solid understanding of data modeling, dimensional design, and schema normalization ...

Terraform and Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery.Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines.Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

expertise in data pipeline development, ETL/ELT processes, and data warehousing solutions.Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc.Experience with relational and NoSQL databases.Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering teams ...

Auto ML, Tensorflow, Natural Language, Vision AIPA’s common technology components:Google CloudCloud RunCloud FunctionsKubernetes Engine, Compute Engine, VPCsCloud Spanner, Cloud SQL and BigQueryDataflow, Apache BeamVertex AI, DialogFlow, Auto MLArchitectureMicroservices patternsEvent-driven architectures and message queuesRelational and non-relational databasesLanguages and frameworksJava 17+, JavaScript, Python, SQLCommon frameworks and APIs ...

platform and lakehouse transformation journeys What We're Looking For Strong commercial experience in Data Engineering Hands-on experience with Databricks and/or Apache Spark Strong PySpark and SQL development skills Experience designing, building and supporting data pipelines Experience working with Azure or AWS Good understanding of modern ...

market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...