426 to 450 of 480 Apache Airflow Jobs in the UK

help strengthen pre and post-change validation, enhance observability, and support the continuous improvement of automation capabilities. Technology Stack Linux/UNIX Python Ansible Apache Airflow Prometheus Grafana Loki VMware F5 What We're Looking For We're looking for engineers with: Strong Linux/UNIX administration fundamentals ...

risk through data, building robust pipelines and driving insights to support Wise's secure and compliant framework. You will work with Python, DBT, Airflow and REST APIs to create scalable data solutions, collaborate across security domains, and define KPIs for continuous monitoring and improved efficiency. #J-18808-Ljbffr ...

building internal developer tools, DSLs, or compilers, multi-language proficiency such as Go, C, or Rust, cybersecurity, workflow orchestration with Prefect or Airflow, high-growth SaaS Условия: Competitive salary Meaningful equity One of the first engineering hires Direct access to the founders Huge ownership from day one Build products ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and "infrastructure as a code" tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment #J-18808-Ljbffr ...

global market. About the role We are looking for a Lead Data Engineer to take ownership of a business domain within our modern Snowflake, Airflow, and dbt data platform. This role will combine hands-on technical leadership with team coordination, helping ensure stable platform operations, strong data quality … leading small data engineering teams, ideally 2–3 engineers. Experience designing, building, and maintaining data processing systems at scale. Experience with Azure cloud services, Airflow, dbt, Kubernetes, GitHub, and Terraform. Experience working in fast-moving, high-growth, or startup-like environments is preferred. Strong communication skills with the ability ...

deployments Troubleshoot integration failures, data inconsistencies, and production issues Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

Develop batch and ELT/ETL data workflows for ingesting and processing large datasets* Work with technologies such as Python, Java, Spark, SQL, and Airflow* Apply good software engineering practices, including testing, documentation, and CI/CD* Contribute to cloud‐based systems (AWS), focusing on reliability, monitoring, and automation … data engineering concepts such as data lakes, batch processing, and distributed systems* Experience building or operating data pipelines using tools like Spark and Airflow* Familiarity with cloud platforms (AWS preferred) and production monitoring* Clear communication skills and a collaborative approach to problem‐solving.**Why join us?**Join our team ...

Lead Data Engineer Location: West Midlands | Hybrid 3 days per week onsite The Role Were looking for an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a ...

The Role We’re looking for an experienced Lead Data Engineer to provide hands‐on technical leadership within a data engineering team for a large retail brand. You’ll lead the design and delivery of ...

order to build scalable and robust solutions Utilize Docker to create containerized microservices Automate infrastructure configuration using Terraform and Puppet Use tools such as Airflow and Snowflake to build data pipelines Integrate various JavaScript libraries such as React to deliver seamless user experiences Design data models and persist data … Server Use T‐SQL to manipulate and transform data Utilize Kafka and SQS for messaging Improve search using Apache Solr Build caching layers using Redis Use TeamCity and Jenkins to automate deployments Write automated unit, integration and deployment tests Build tooling to automate away repetitive tasks Qualifications Blackstone seeks ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

Docker An interest or experience in data/network security would be a nice-to-have Datadog/Grafana/Prometheus Data related products (Airflow, Jupyter, Spark, dbt etc) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems … Stack... Python as our main programming language Terraform for our infrastructure definition Kubernetes for data services and task orchestration Argo CD for application deployments Airflow purely for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora ...

days on site A London Trading firm is building a trading data platform from scratch, in house, on open source. Spark, Kafka, Polaris, Ranger, Airflow, MinIO. This is the person who builds it. Why now The analytics function is running on legacy warehouses that were never built … ingestion pipelines for both streaming and batch sources Write the Spark jobs that clean, transform and land data in the lakehouse Build and own Airflow orchestration Work on cataloguing and access control with Polaris and Ranger Help migrate data off the legacy warehouses and retire them What we need ...

long-term fixes TypeScript and Node.js sit at the heart of the wider application stack, including the services and APIs surrounding the data platform. Airflow orchestrates pipelines and workflows, with Python also used across data processing and automation. Everything runs within a modern Azure environment. You're are encouraged … APIs around the resulting data Are confident with TypeScript and/or Python Understand relational, NoSQL and modern cloud-based storage Have worked with Airflow or similar orchestration tooling Care about data quality, clean engineering and genuine ownership Have experience building SaaS products or other production platforms Experience with ...

engineering & the platform Build and maintain ingestion and transformation pipelines across our stack. We use a modern data stack, such as AWS, Redshift, and Airflow Write production-quality Python for extraction, transformation, and automation tasks that fall outside dbt’s remit Help us modernise: we’re actively evolving this … plain language Ownership: you’re comfortable being the first and only person in this role, and building things properly from scratch Nice-to-haves Airflow or similar orchestration tools, and familiarity with data ingestion tools such as dlt or Airbyte Warehouse cost and performance tuning, particularly on Redshift Fintech ...

We tackle the most complex problems in quantitative finance, by bringing scientific clarity to financial complexity. From our London HQ, we unite world-class researchers and engineers in an environment that values deep exploration and ...

data scientists. How to make an Impact Architect and Implement : Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development and solve any complex performance bottlenecks, concurrency issues … deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

What will you be working on? Data Engineering Build and Optimise Pipelines: Develop, optimise, and maintain data pipelines using Python, REST APIs, DBT, and Airflow to ensure seamless data flow for compliance and risk management. Understanding Control Requirements & Continuous Monitoring: Partner closely with control owners across security domains (such … organisation. QualificationsTechnical Expectations: Proficient in Python, with experience in automation and building data pipelines. Strong expertise in working with tools like DBT and Airflow for ETL processes. Skilled in using REST APIs to develop and manage data integrations. Proficient in SQL and familiar with data visualization tools such ...

Data Engineering team, which builds and governs the data foundation that powers the business. You'll work within our stack — Python ingestion pipelines, Airflow orchestration, and Snowflake/Databricks — helping move data reliably and securely from source to decision-ready output.You'll implement features and fixes against a given … with solid experience building and operating production data pipelines Comfort investigating and root-causing pipeline issues independently before escalating Experience with workflow orchestration tools (Airflow or similar) and a cloud data warehouse/lakehouse (Snowflake, Databricks, or similar) Solid understanding of data pipeline concepts: idempotency, schema evolution, backfills ...