1 to 25 of 158 Apache Jobs in England

detect threats, analyze patterns, and respond to incidents in real-time. You'll be designing and developing highly scalable streaming services using technologies like Apache Kafka, Apache Flink, Apache Storm and Apache Spark in Java, Python and Go. You'll design and implement data pipelines that … Design, develop, and maintain streaming microservices in Java and Go that provide real-time data processing capabilities to internal teams Build and optimize Apache Flink and Apache Spark based data pipelines running on Kubernetes infrastructure Design event-driven systems leveraging Kafka for high-throughput, low-latency message streaming ...

native solutions that support analytics, automation, and digital transformation. Data Engineering, Full Stack & Platform Development Design and implement scalable data pipelines using tools like Apache Spark, Airflow, or dbt.Build and maintain data lakes, warehouses, and real-time streaming solutions. Develop APIs and microservices to expose data securely and efficiently. ...

understanding of Quantexa components such as: Entity Resolution (ER), Scoring Framework, Contextual Network Generation, UI Configuration. Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch. Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable ...

understanding of Quantexa components such as: Entity Resolution (ER) Scoring Framework Contextual Network Generation UI Configuration Strong background in Big Data technologies, including:Scala, Apache Spark, Hadoop, ElasticSearch Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable ...

code. Modern Data & Cloud Practices : Experience with modern data engineering tooling and cloud platforms (ideally AWS and Databricks), and familiarity with distributed processing (e.g. Apache Spark). Exposure to infrastructure as code (such as Terraform and Databricks Asset Bundles) is a plus. Engineering Mindset : Understanding of testing, version control ...

Data Cleansing & Transformation \n \n Real Time Data Processing \n\n \nInfrastructure & Platforms\n \n \n \n \n Linux (Red Hat preferred) \n \n Apache/Nginx \n \n Elasticsearch \n \n Grafana \n \n Prometheus \n \n Cosmos DB \n \n Kafka or Azure Event Hubs ...

strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. Essential qualifications • Extensive experience with Python and PySpark/Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures ...

engineers and support code reviews, design reviews, and engineering best practices. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. Experience with Spark ...

market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

Experience with Infrastructure-as-Code tools such as Terraform Knowledge of DevOps, CI/CD, platform automation, and cloud‐native operational practices Familiarity with Apache Spark, dbt, and modern data integration technologies Delivery Experience Experience leading architecture for enterprise-wide Databricks implementations and cloud data modernization initiatives Proven success ...

based solutions. Experience working with Cosmos DB SDKs in technologies such as .NET, Java, Python, or Node.js. Strong understanding of distributed computing concepts and Apache Spark architecture. Hands-on experience developing data pipelines using PySpark. Experience with Pytest and automated testing frameworks. Strong understanding of partitioning, indexing strategies ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working with ...

ownership Technical Application ownership Qualifications Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , Harness Strong experience of developing on Apache Spark/Hadoop Strong experience with AWS cloud services including AWS Glue, S3, Step Strong experience building data pipelines and data lake Experience ...

ecosystem •Infrastructure as Code using Terraform, Ansible, or CloudFormation •GitOps tooling such as Argo CD •Scripting (Python, Shell) •Container orchestration tools (e.g., Docker Swarm, Apache Mesos) •Service Mesh technologies (e.g., Istio, Linkerd) •Monitoring and observability tools (Prometheus, Grafana, ELK) •Applying infrastructure security, compliance, and governance practices •Working within agile ...

with the ability to deliver incremental data modelling solutions in collaboration with engineering teams. Technical Environment (Desirable) Microsoft Fabric Azure Data Factory Python PySpark Apache Spark Desirable Skills Azure DevOps Contract Details Sector: Banking/Financial Services Location: London – Office based 4 days per week Contract Type: TBC Start ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

with dbt or Airbyte is preferred. Experience developing production Spark or Hadoop applications Gained experience with a workload scheduling and orchestration tool such as Apache Airflow/Cloud Composer Familiarity with modern DataOps (e.g. quality eng, testing, pipeline deployments etc) practices and IaC (e.g. Terraform) Familiarity with the constellation ...

ability to help refine problem statements and propose solutions taking effort-impact-scalability tradeoff into account. Some skills that will make you stand out: Apache Spark, Iceberg, Kafka, dbt Scikit-Learn, XGBoost, PyTorch, MLFlow,, GraphFrames, Ray AWS (S3, EMR, SageMaker, Lakeformation), Terraform, Docker, GitHub CI/CD Knowledge Graphs ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/CD pipelines ...

Infra security, compliance and governance • Familiarity with CNCF ecosystem and building platforms on top of Kubernetes • Container orchestration tools, e.g. Docker Swarm/Apache Mesos • Knowledge of Service Mesh technologies, e.g. Istio/Linkerd • Experience of monitoring tools, e.g. Prometheus, Grafana, ELK • Rounded problem-solving skills - troubleshoot and resolve ...

reviews and continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices ...

innovation-led working environment Key Experience Proven experience as a Data Architect, Databricks Architect, Lead Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data ...

statistics skills: distributions, statistical testing, regression analysis Cloud management and deployment – A strong background in different cloud platforms like Amazon AWS, Microsoft Azure, OpenStack Apache, and Google Cloud Platform Databases – A familiarity of working with different databases such as Postgres, MySQL, NoSQL (e.g., MongoDB), Redis is recommended. Web technologies ...