151 to 175 of 422 Apache Jobs in the UK

Application Resiliency, and Security. Preferred Qualifications, Capabilities, And Skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Experience with evaluation frameworks and approaches (golden datasets, LLM-as-judge, human-in-the-loop review, red teaming). Solid grounding in data ...

Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts across financial domains ...

solutions. Tech StackDevelopment: TypeScript, Angular/React, Python, SQL, OpenAPI.Cloud: AWS (Lambda, API Gateway, S3, SQS, SNS, Aurora, and others as required).Databases: PostgreSQL, Apache Iceberg, DynamoDB, VectorDB (e.g. ChromaDB).IaC and CI/CD: Terraform, AWS CLI, GitLab. AI Development: Agentic AI tooling (e.g. GitHub Copilot, Claude Code ...

environments. Technical Skills: Experience building and operating ETL/ELT pipelines. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with some of these or similar technologies ...

Linux operating systems, including command-line administration and troubleshooting. Solid experience with SQL or data pipeline technologies, Experience with data processing frameworks such as Apache Airflow, Apache Flink or Google Dataflow, with a practical understanding of ETL pipeline design and the ability to diagnose issues across data transformation …/IP networking principles, including the ability to diagnose connectivity issues and interpret network traffic. Familiarity with event streaming platforms such as Apache Kafka, including an understanding of producer/consumer patterns and how message queues are used to move data reliably between systems at scale. Familiarity with infrastructure ...

Spring Boot, REST APIs and microservices Experience with a relational database like MS SQL Server, PostgreSQL, Sybase ASE Experience with Streaming technologies like Kafka, Apache Flink Experience with cloud technologies, such as Microsoft Azure Experience with Agile process Experience with DevOps and tools like Azure DevOps Our benefits ...

support data pipelines, ensuring they are production-ready and meet data quality standards. You will work with technologies such as Python, SQL, and Apache Spark, helping drive analytics and operational decision-making. Ideal candidates possess strong programming skills, a relevant degree, and a collaborative spirit, as you will partner ...

reusability 🌳 Working alongside engineers who care about doing it right — strong standards, real peer review, no shortcuts 🌳 Tech stack: Terraform, Python, AWS (Lambda, S3, Apache Iceberg), GitLab CI/CD 🌳 Must-have: Active SC Clearance + solid Python experience This is a genuine chance to shape core platform architecture ...

applied across the team 🌳 Mentoring Data Engineers, running design reviews, and enforcing data engineering standards across the board 🌳 Tech stack: AWS Glue, S3, MWAA, Apache Iceberg, Python, PySpark, Terraform 🌳 Must-have: Active SC Clearance + proven leadership in data pipeline design This isn't a "build what ...

applied across the team 🌳 Mentoring Data Engineers, running design reviews, and enforcing data engineering standards across the board 🌳 Tech stack: AWS Glue, S3, MWAA, Apache Iceberg, Python, PySpark, Terraform 🌳 Must-have: Active SC Clearance + proven leadership in data pipeline design This isn't a "build what ...

applied across the team 🌳 Mentoring Data Engineers, running design reviews, and enforcing data engineering standards across the board 🌳 Tech stack: AWS Glue, S3, MWAA, Apache Iceberg, Python, PySpark, Terraform 🌳 Must-have: Active SC Clearance + proven leadership in data pipeline design This isn't a "build what ...

models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline development. ...

MongoDB. And work with PostgreSQL too. The Senior Data Engineer will be well versed in Kafka. The Senior Data Engineer will know all about Apache Spark, Airflow and Iceberg. The Senior Data Engineer will have solid cloud infrastructure knowledge e.g. Google Cloud (GCP). The Senior Data Engineer will ...

stakeholdersWillingness to pick up and learn new technologies and frameworksNice to have: Knowledge of databases, SQLFamiliarity with Boost ASIOFamiliarity with data serialization formats such Apache Arrow/Parquet, Google Protocol Buffers, FlatbuffersExperience with gRPC, and Websocket protocolsExperience with Google Cloud/AWS and/or containerization in Docker/ ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...

warehousing architecturesExpopsure to data governance frameworks, metadata management, and regulatory compliancIt would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability toolingExperience leading architecture workshops with both technical and non-technical stakeholdersA background providing ...

help evolve platform standards and documentation What we’re looking for 3–5 years of data engineering experience Strong Python & SQL skills Experience with Apache Spark/PySpark Ability to monitor, troubleshoot, and resolve pipeline failures Familiarity with lakehouse concepts Experience with Git, Databricks, streaming/CDC, AWS, Great ...

years' experience delivering enterprise data platforms or analytics solutions. Strong commercial experience delivering Databricks solutions. Consulting or customer-facing delivery experience. Strong understanding of Apache Spark, modern Lakehouse architecture and Unity Catalog. Experience with Azure, AWS or GCP. Experience implementing CI/CD and deployment automation. Excellent communication ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints Familiarity with AI evaluation and observability practices for LLM workloads Understanding ...

deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

blocking framework (Vert.x, SpringWebFlux, or equivalent) and asynchronous, event-driven service design. Deep experience with stream processing and event-driven architecture - Kafka Streams, Apache Flink, or equivalent - including stateful streaming, topology design, and managing local state stores (e.g.RocksDB) at scale. Proventrack recordbuilding high-throughput, low-latency systems where tail ...

operating ETL/ELT pipelines and large-scale data architectures. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with similar technologies is essential).Ability to work ...

should have specialist experience in one or more of the following technologies**Azure Databricks*** Design and build high-performance data pipelines: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.* Experience of Databricks ML and Azure … develop predictive models and drive business insights.* Proven expertise in Databricks, Apache Spark, and data pipeline development and strong understanding of data warehousing concepts and practices.* Experience with Microsoft Azure cloud platform, including Azure Data Lake Storage, Databricks and Azure Data Factory.* Azure Data Engineer Associate and Databricks Certified ...

help design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … opportunity to grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data ...