451 to 475 of 480 Apache Airflow Jobs in the UK

Senior Data Engineer to design and deliver scalable data pipelines using Python, PySpark and SQL in a cloud-based environment. The role focuses on Airflow orchestration, Kubernetes deployment on Azure, data modeling and building a modern data warehouse to support analytics and product insights. You will mentor juniors, collaborate ...

requiring strong data engineering expertise, experience working with financial data, and the ability to operate within a fast-paced investment environment. Key Skills Snowflake Airflow DBT Bloomberg Integrations Responsibilities Design, build and maintain scalable data pipelines and workflows. Develop and enhance data solutions using Snowflake, Airflow and DBT. … Drive best practices across data engineering, automation and platform support. Required Experience Experience within Investment Management, Asset Management or Financial Services Expertise with Snowflake, Airflow and DBT. Experience integrating Bloomberg data and services into enterprise platforms. Strong SQL and data modelling skills. Experience working with large-scale financial ...

Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment/config: Ansible (physical), Terraform/Argo CD/Helm (Kubernetes) Development environment: MacBook … Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code Degree in Computer Science or numerical degree Competitive salary ...

We are Foundational. Foundational is an early-stage, mission-driven startup solving one of the hardest engineering challenges of the decade: creating an exquisite, real-time understanding of Earth and orbit using ground-based lasers. ...

Why Sony Interactive Entertainment? Sony Interactive Entertainment isn't just the Best Place to Play — it's also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

driven user experience. Hybrid role with three on-site days in City of London. Responsibilities include building scalable data pipelines (BigQuery, Dataflow/Apache Beam, Airflow), ensuring data quality and observability (Looker, Monte Carlo), and collaborating with product engineering and data science teams to plan data tracking ...

value, and/or FICC options trading Experience with Linux-based, concurrent, high-throughput, low-latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced ...

value, and/or FICC options trading Experience with Linux‐based, concurrent, high‐throughput, low‐latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced ...

store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree ...

Designing and implementing data classification and governance-by-design Maintaining and improving a GCP/BigQuery lakehouse environment Building and supporting pipelines using Python, Airflow/Cloud Composer and dbt Identifying security vulnerabilities and supporting remediation Implementing robust CI/CD, testing and Terraform-based infrastructure Working across integrations … Deep knowledge of RBAC/IAM within cloud data environments Strong understanding of data security, governance and access controls Hands-on experience with Python, Airflow and dbt Experience with Terraform and CI/CD Knowledge of Kubernetes Exposure to AWS alongside GCP Strong stakeholder management and the ability ...

migrate data sets into their AWS Snowflake platform. Designing and implementing dbt models that power clean, consistent metrics. Buidl out orchestartion pipelines using Airflow Creating clear documentation for dashboards, metrics, and models. Ensuring optimal performance and cost efficiency across Snowflake Supporting a stakeholder-first approach to data usability … access. KEY SKILLS AND REQUIREMENTS Extensive experience operating within Snowflake environments Airflow for data orchestration. Proficiency in SQL and dbt, with a strong understanding of data modeling best practices. Strong communication skills with the ability to translate complex needs into elegant data solutions. Focus on usability, documentation, and data ...

within the new architecture. You'll work across: Migrating legacy pipelines and workloads into Snowflake Building and refactoring pipelines using Python Developing orchestration through Airflow/MWAA Infrastructure and deployment using Terraform and Docker Supporting transformation and modelling using dbt Working across both legacy and modern AWS data environments … looking for a broad, hands-on Data Engineer who is comfortable working across data engineering, software engineering and infrastructure. Core experience: Python Airflow/MWAA Snowflake Terraform AWS Desirable experience: dbt Docker Redshift R Data platform migrations or modernisation programmes The ideal candidate will have worked within complex ...

Company DescriptionWise is a global technology company, building the best way to move and manage the world's money. Min fees. Max ease. Full speed. Whether people and businesses are sending money to another country ...

growth payments platform. You will modernize the stack and mentor engineers on data quality and reliability. Responsibilities Design and maintain ELT with DBT and Airflow Scale batch and streaming with Spark Define data contracts with product and engineering Improve lineage, testing and observability Requirements 5+ years in data engineering … with Python DBT and Airflow in production Experience with Spark or similar frameworks Great communication and documentation habits Remote within UK or EU time zones Learning budget and conference allowance #J-18808-Ljbffr ...

Inside IR35 Location: London - Hybrid, 3 days per week onsite Duration: 6 months The role will focus on GCP, BigQuery, Python, Airflow/Cloud Composer, dbt, Terraform and CI/CD, working closely with architects, engineers, product and governance teams to deliver secure, scalable data solutions. Key experience … BigQuery Python, Airflow & dbt Terraform & CI/CD Data platforms/lakehouse environments Kubernetes AWS alongside GCP Strong stakeholder management If this sounds relevant to you, please drop me a message or send your CV Thanks, Callum ...

ship confidently. Key Responsibilities Own and evolve the core AI platform infrastructure supporting production ML services. Build and maintain orchestration pipelines using Dagster , Airflow or similar tooling. Deploy and manage workloads using Kubernetes and cloud-native infrastructure. Develop platform tooling and services in Python . Implement and maintain Terraform … + 10% bonus Working Model: Hybrid, 1-2 days per week onsite in Central London Tech Stack: Python, Azure, Kubernetes, Terraform, Dagster, Airflow, ML Infrastructure, LLMs, Vector Databases Visa Sponsorship: Exceptional candidates only Interested? Please apply below. ...

FullRemoteTalent is seeking a Data Engineer to own pipelines and analytics for a high-growth payments platform. This role involves modernizing the stack, mentoring engineers on data quality and reliability, and scaling batch and streaming ...

United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. You will contribute to data ...

Foundry or AI Search, with Docker and Kubernetes Understanding of schema validation frameworks (e.g., JSON Schema, Pydantic) and MLOps tools such as MLflow or Airflow Strong communication ability to explain trade-offs around cost, latency and accuracy to technical and non-technical audiences Nice to have Experience using Azure … agent and tool interoperability Hands-on work with vector databases like Pinecone, Weaviate, Qdrant or pgvector Familiarity with distributed systems, workflow engines (Temporal, Airflow or Dagster) and event-driven architectures Experience with open-source LLMs or Small Language Models for custom deployments Knowledge of AI safety and governance: guardrails ...

seeking a Lead Data Engineer to own a business domain within a modern Snowflake, Airflow, and dbt data platform, combining hands-on technical leadership with team coordination in Greater London. You will shape how data products are designed, built, and improved, ensuring stable platform operations, high data quality ...

experienced Full Stack Data Engineer to join our Customer Data Science team. You will design, build and maintain robust data pipelines, orchestration with Airflow, and Databricks-based solutions to empower data scientists with reliable data foundations and scalable platforms. You will collaborate across product and delivery partners, contribute ...

backend systems and data-intensive applications. You are experienced with Scala or Java and at least one distributed data processing framework such as Scio, Apache Beam, Spark, or Flink. You know how to design and build reliable backend services and scalable data pipelines that support business-critical workflows. … have experience designing data models that enable accurate, high-volume financial processes. You have worked with workflow orchestration platforms such as Flyte, Airflow, or similar technologies. You care deeply about engineering fundamentals, including continuous delivery, automated testing, monitoring, and writing production-quality code. You enjoy collaborating with cross-functional ...

apart) Experience using Google Cloud Platform and/or Google Cloud Engineer qualifications (could set you apart) Experience of PySpark, BigQuery, Git, DataFlow, DataProc, Airflow, or equivalents (could set you apart) Core Competencies Demonstrates strong expertise in data analytics, with proficiency in SQL and Python, while effectively managing stakeholder … Experience Stakeholder Relationship Management Google Cloud Platform Experience ATS Optimization Keywords Hard Skills Data Analytics SQL Python Analytics Techniques PySpark BigQuery Git DataFlow DataProc Airflow Soft Skills Excellent Communication Skills User Enablement Problem-Solving Certifications & Qualifications Google Cloud Engineer Qualifications Industry Keywords Credit Bureau Data D&A Governance Compliance ...

design, API development, and data modelling for production machine learning systems. Build and operate production solutions using technologies such as Python, TensorFlow, PyTorch, Redis, Airflow, Docker, Kubernetes, and cloud-native infrastructure. Design and deploy scalable batch and real-time ML pipelines supporting customer-facing applications. Develop and operationalise … designing, building, or scaling RAG systems. Experience in search, ranking, recommendations, personalisation, content intelligence, moderation, NLP, computer vision, or related AI domains. Experience with Airflow or comparable workflow orchestration platforms. Experience with cloud-native environments and container technologies, including Docker and Kubernetes. Experience developing model evaluation frameworks ...

design, API development, and data modelling for production machine learning systems. Build and operate production solutions using technologies such as Python, TensorFlow, PyTorch, Redis, Airflow, Docker, Kubernetes, and cloud-native infrastructure. Design and deploy scalable batch and real-time ML pipelines supporting customer-facing applications. Develop and operationalise … designing, building, or scaling RAG systems. Experience in search, ranking, recommendations, personalisation, content intelligence, moderation, NLP, computer vision, or related AI domains. Experience with Airflow or comparable workflow orchestration platforms. Experience with cloud-native environments and container technologies, including Docker and Kubernetes. Experience developing model evaluation frameworks ...