126 to 150 of 422 Apache Jobs in the UK

/CD, Application Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse ...

platforms:AWS,Azureand GCP Modern data platforms:Databricks, Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation … quality:dbt, Great Expectations and equivalent tooling Orchestration:Apache Airflow and cloud-native orchestration services Event streaming:Kafka and cloud-native event-streaming technologies Data modelling:Dimensional,analyticaland domain-oriented modelling Data governance:Catalogue, metadata, lineage, quality, security,privacyand access controls Data products:Reusable, discoverable and well-governed dataproductsand ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

owned infrastructure. You’ll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark. Key Responsibilities Build and maintain new and existing data pipelines Develop data solutions using Python, Azure and Kubernetes Work … with Apache Airflow and Apache Spark Support the ongoing development and improvement of the data platform Ensure data remains securely hosted on MoD-owned infrastructure Collaborate with wider technical and delivery teams Troubleshoot and resolve technical issues across existing data pipelines Required Experience Strong commercial experience ...

file processing , data transformation, and batch processing Strong SQL and database skills with focus on: Query optimization Performance tuning Data modeling Experience with Apache Flink , ETL frameworks, or equivalent data processing technologies Exposure to REST APIs, messaging systems, and event‐driven architectures Understanding of CI/CD pipelines ...

cloud development. Exposure in hosting and setting up container-based applications. Experience is Api Gateway Management services Exposure to Caching Framework like (Redis, Apache Ignite etc) Exposure to Log Monitoring Tools like (Elastic Search & Splunk etc ) The ability to understand and support, modify, and maintain systems and code developed ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor peers ...

possible, on-site in Gloucester when required. You will design and manage data pipelines, ETL processes, and data integration for National Security clients using Apache tools including NiFi, Kafka, and SQL/NoSQL databases like MongoDB. Proficiency in Groovy, Python or Java for ETL, data modeling, and data governance ...

Jenkins, Bamboo, Concourse etc. Monitoring utilising products such as: Prometheus, Grafana, ELK, filebeat etc. Observability - SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Edge technologies e.g. NGINX, HAProxy etc. Excellent knowledge of YAML or similar languages It is essential for this role ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement Why Apply ...

data engineeringStrong proficiency in Python and RustHands-on experience with AWS cloud, Linux, and Docker. Expertise in distributed computing frameworks (Ray, Dask, or Apache Arrow).Strong analytical skills, with the ability to handle large-scale, high-frequency data. Solid understanding of ETL pipelines, data modelling, and database optimisation. Experience ...

production, providing subject matter expertise on the .Net stack and contributing to technical design discussions. You'll use a range of technology including Apache Flink with Java for large scale data processing and will be able to assess and recommend new and emerging technologies, using the best tool ...

Code, Cursor, or AI-assisted coding CLI workflows) to accelerate code generation, refactoring, and daily execution. Message-Driven Architecture: Hands-on experience with Kafka, Apache Flink, or equivalent real-time event-streaming platforms. Containers & Data: Proficiency with Docker, Kubernetes, and relational databases (MS SQL, Sybase). Delivery Automation & Monitoring ...

Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python ...

Monitor and maintain - Monitor data systems for performance issues and make any necessary updates. What You’ll Bring ETL processing languages such as Python Apache Spark/NiFi/Kafka Relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) AWS and data-related services Palantir Foundry ...

/CD paradigms and systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Observability – SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable: Jupyter Hub Awareness RabbitMQ ...

well as present complex findings clearly to both technical and non-technical audiencesProficient in using Tech Stack such as Amazon S3, Lambda function, Kafka, Apache Airflow or Bloomberg Teck Stack in production environmentExceptional problem-solving skills, numerical proficiency and high attention to detailAbility to work independently as well ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

Experience of working with MOD Accreditors; and Knowledge of Onsite/project tech stack includes but is not limited to Azure, AWS, Docker, Kubernetes, Apache (NiFi, Kafka), NodeJS, Typescript, MongoDB, AI, Machine Learning etc. Just as important are curiosity, strong problem solving ability, and the motivation to continuously improve ...

modern frameworksWorking knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least oneDeep experience with distributed computing with Apache Spark and knowledge of Spark runtime internalsFamiliarity with CI/CD for production deploymentsWorking knowledge of MLOps, ML/AI models and AI APIsDesign ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints Familiarity with AI evaluation and observability practices for LLM workloads Understanding ...

knowledge of AI infrastructure. Good proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

preferred knowledge of AI infrastructure. Good proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

Databricks or SnowflakeHands-on expertise with Spark/PySpark and big data processing technologiesKnowledge of open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpointsFamiliarity with AI evaluation and observability practices for LLM workloadsUnderstanding of agentic design ...