101 to 125 of 413 Apache Jobs in the UK

covering ingestion, streaming, storage, processing, and analytics Architect and guide event-driven and streaming data solutions using Kafka (Confluent Platform/Confluent Cloud); Apache Flink experience is a plus Define and maintain infrastructure-as-code using Terraform, including cloud resources, networking, and platform services Contribute directly to platform components ...

data technologies (e.g. Azure Cosmos DB, Azure SQL Database, Azure Data Factory) Python or another object‐oriented programming language Azure Databricks and/or Apache Spark Reporting or visualisation tools such as Power BI, MicroStrategy, or similar platforms CI/CD workflows, including automated build, test, and deployment pipelines ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering (ETL/ ...

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested code. ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering (ETL/ ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

business leaders. Strong presentation, demo, and storytelling skills, with the ability to explain complex technical concepts clearly and credibly. Familiarity with SQL, Python, R, Apache Spark, or similar technologies is a plus. Experience with Alteryx or similar analytics platforms is highly desirable. Ability to work independently in a fast ...

preferred knowledge of AI infrastructure. Proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

asubject, preferred knowledge of AI infrastructure. Proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

taking customers from EDW to the full capabilities of the Data Intelligence Platform. Demonstrated competence in the Lakehouse architecture including hands-on experience with Apache Spark, Python and SQL Excellent communication skills; both written and verbal Public Cloud Architecting: Hands-on experience designing and implementing big data architectures within ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability – Experience ...

Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based and kernel methods (gradient boosting, random forest, SVM) and anomaly-detection ...

complex pipelines and data models in a cloud environment Strong proficiency in SQL; solid Python and/or PySpark capabilityDemonstrable experience with Databricks, Apache Spark, and Delta Lake architectureProven ability to design and own enterprise-grade canonical data models across multiple upstream source systemsExperience integrating data from APIs ...

SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with AI evaluation and observability practices: evals ...

/CD, Application Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse ...

platforms:AWS,Azureand GCP Modern data platforms:Databricks, Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation … quality:dbt, Great Expectations and equivalent tooling Orchestration:Apache Airflow and cloud-native orchestration services Event streaming:Kafka and cloud-native event-streaming technologies Data modelling:Dimensional,analyticaland domain-oriented modelling Data governance:Catalogue, metadata, lineage, quality, security,privacyand access controls Data products:Reusable, discoverable and well-governed dataproductsand ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

owned infrastructure. You’ll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark. Key Responsibilities Build and maintain new and existing data pipelines Develop data solutions using Python, Azure and Kubernetes Work … with Apache Airflow and Apache Spark Support the ongoing development and improvement of the data platform Ensure data remains securely hosted on MoD-owned infrastructure Collaborate with wider technical and delivery teams Troubleshoot and resolve technical issues across existing data pipelines Required Experience Strong commercial experience ...

file processing , data transformation, and batch processing Strong SQL and database skills with focus on: Query optimization Performance tuning Data modeling Experience with Apache Flink , ETL frameworks, or equivalent data processing technologies Exposure to REST APIs, messaging systems, and event‐driven architectures Understanding of CI/CD pipelines ...

cloud development. Exposure in hosting and setting up container-based applications. Experience is Api Gateway Management services Exposure to Caching Framework like (Redis, Apache Ignite etc) Exposure to Log Monitoring Tools like (Elastic Search & Splunk etc ) The ability to understand and support, modify, and maintain systems and code developed ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor peers ...

possible, on-site in Gloucester when required. You will design and manage data pipelines, ETL processes, and data integration for National Security clients using Apache tools including NiFi, Kafka, and SQL/NoSQL databases like MongoDB. Proficiency in Groovy, Python or Java for ETL, data modeling, and data governance ...