76 to 100 of 257 Apache Jobs in London

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

Security In-depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse ...

stakeholders, and business leaders.Strong presentation, demo, and storytelling skills, with the ability to explain complex technical concepts clearly and credibly.Familiarity with SQL, Python, R, Apache Spark, or similar technologies is a plus.Experience with Alteryx or similar analytics platforms is highly desirable.Ability to work independently in a fast-paced environment ...

preferred knowledge of AI infrastructure. Proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

asubject, preferred knowledge of AI infrastructure. Proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

taking customers from EDW to the full capabilities of the Data Intelligence Platform. Demonstrated competence in the Lakehouse architecture including hands-on experience with Apache Spark, Python and SQL Excellent communication skills; both written and verbal Public Cloud Architecting: Hands-on experience designing and implementing big data architectures within ...

complex pipelines and data models in a cloud environment Strong proficiency in SQL; solid Python and/or PySpark capabilityDemonstrable experience with Databricks, Apache Spark, and Delta Lake architectureProven ability to design and own enterprise-grade canonical data models across multiple upstream source systemsExperience integrating data from APIs ...

/CD, Application Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns.Deep lakehouse ...

technical solutions.Tech StackDevelopment: TypeScript, Angular/React, Python, SQL, OpenAPICloud: AWS (Lambda, API Gateway, S3, SQS, SNS, Aurora, and others as required)Databases: PostgreSQL, Apache Iceberg, DynamoDB, VectorDB (e.g. ChromaDB)IaC and CI/CD: Terraform, AWS CLI, GitLabAI Development: Agentic AI tooling (e.g. GitHub Copilot, Claude Code)Experience ...

platforms:AWS,Azureand GCP Modern data platforms:Databricks, Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation … quality:dbt, Great Expectations and equivalent tooling Orchestration:Apache Airflow and cloud-native orchestration services Event streaming:Kafka and cloud-native event-streaming technologies Data modelling:Dimensional,analyticaland domain-oriented modelling Data governance:Catalogue, metadata, lineage, quality, security,privacyand access controls Data products:Reusable, discoverable and well-governed dataproductsand ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

owned infrastructure. You’ll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark. Key Responsibilities Build and maintain new and existing data pipelines Develop data solutions using Python, Azure and Kubernetes Work … with Apache Airflow and Apache Spark Support the ongoing development and improvement of the data platform Ensure data remains securely hosted on MoD-owned infrastructure Collaborate with wider technical and delivery teams Troubleshoot and resolve technical issues across existing data pipelines Required Experience Strong commercial experience ...

cloud development. Exposure in hosting and setting up container-based applications. Experience is Api Gateway Management services Exposure to Caching Framework like (Redis, Apache Ignite etc) Exposure to Log Monitoring Tools like (Elastic Search & Splunk etc ) The ability to understand and support, modify, and maintain systems and code developed ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement Why Apply ...

data engineeringStrong proficiency in Python and RustHands-on experience with AWS cloud, Linux, and Docker.Expertise in distributed computing frameworks (Ray, Dask, or Apache Arrow).Strong analytical skills, with the ability to handle large-scale, high-frequency data.Solid understanding of ETL pipelines, data modelling, and database optimisation.Experience working with real ...

Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python ...

well as present complex findings clearly to both technical and non-technical audiencesProficient in using Tech Stack such as Amazon S3, Lambda function, Kafka, Apache Airflow or Bloomberg Teck Stack in production environmentExceptional problem-solving skills, numerical proficiency and high attention to detailAbility to work independently as well ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

modern frameworksWorking knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least oneDeep experience with distributed computing with Apache Spark and knowledge of Spark runtime internalsFamiliarity with CI/CD for production deploymentsWorking knowledge of MLOps, ML/AI models and AI APIsDesign ...

modern frameworksWorking knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least oneDeep experience with distributed computing with Apache Spark and knowledge of Spark runtime internalsFamiliarity with CI/CD for production deploymentsWorking knowledge of MLOps, ML/AI models and AI APIsDesign ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

knowledge of AI infrastructure. Good proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

preferred knowledge of AI infrastructure. Good proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts across financial domains ...