1 to 25 of 27 Contract Apache Airflow Jobs

Platform DevOps team to build and run highly reliable CI/CD pipelines and infrastructure for the bank’s modern data estate (AWS + Airflow + Databricks). Key Responsibilities Design, build and maintain Infrastructure as Code using CloudFormation and AWS services Manage and scale Apache Airflow … call rotation and incident resolution Essential Skills & Experience 5+ years as a DevOps/Platform Engineer Strong AWS experience + CloudFormation Production experience with Apache Airflow (MWAA) Lambda , Linux + Bash scripting CI/CD expertise (Git, Jenkins, GitHub Actions) IaC and GitOps practices Nice to Have Terraform ...

pipelines Proficiency with at least one cloud platform: Amazon Web Services (AWS) Microsoft Azure (Azure) Google Cloud Platform (GCP) Experience with orchestration tools (e.g., Apache Airflow) Experience with version control (e.g., Git) Solid understanding of data warehousing concepts and dimensional modeling To apply for this role please submit ...

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with: Apache Airflow Data modelling and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. To apply for this position please submit ...

Skills & Experience Proven experience as a Data Engineer , BI Engineer , or similar role. Strong proficiency in SQL , Python , and modern data engineering frameworks (e.g., Airflow, dbt, Spark, etc.). Experience with cloud platforms such as AWS, Azure, or GCP. Solid understanding of data warehousing and ETL/ELT architecture. ...

/CD for data pipelines. Good to have skills Although not essential, the following skills are desired by the client: Kafka/Kinesis, Airflow, CodePipeline. BI tools (Power BI/Tableau). Docker/OpenShift; metadata driven pipelines. 3-8+ years Data Engineering experience. Cloud data engineering ...

Owning features end-to-end - from stakeholder requirements through to algorithm hardening, edge-case handling, and value measurement Working with orchestration frameworks (Dagster/Airflow), experiment tracking (MLflow), and containerised infrastructure (Docker/ECS) Collaborating closely with business stakeholders and contributing to roadmap and feature prioritisation What ...

Required Data Integration & Ingestion Data Manipulation & Processing GitHub, GitHub Actions, Azure DevOps Azure Data Factory Databricks SQL Database Azure Synapse Stream Analytics AWS Glue Apache Airflow AWS Kinesis Amazon Redshift SonarQube PyTest Optional/Nice to Have Experience managing projects or running Scrum teams Experience with BPC/ ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this click apply for full job details ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this role Any experience within post-trade ...

understanding ML pipelines and models in a production setting. Direct experience with Google Cloud Platform, BigQuery, and associated tooling. Experience with workflow tools like Airflow or Kubeflow. Familiarity with dbt (Data Build Tool). Please send your CV for more information on these roles. Reasonable Adjustments: Respect and equality ...

Overview Lead a team of Data Analysts and own the delivery of high-impact analytical products using modern data engineering tooling (AWS, Databricks, dbt, Airflow). You will bridge the gap between business stakeholders and the data platform. Key Responsibilities Lead and mentor a team of Data Analysts Build … maintain self-serve analytical assets using dbt models and Databricks Design and orchestrate analytical pipelines in Airflow Translate complex business requirements into actionable insights and dashboards Perform deep-dive analysis across Retail, Commercial, Risk and Marketing domains Ensure all deliverables meet strict data governance and regulatory standards Work closely ...

with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI, CI/ ...

MLflow) • Experience with cloud-based ML tools (e.g. SageMaker), data and model versioning (e.g. DVC), CI/CD (e.g. GitHub Actions), workflow orchestration (e.g. Airflow/Dagster) and containerised solutions (e.g. Docker, ECS) nice to have • Experience in code testing (unit, integration, end-to-end tests) • Strong data engineering ...

data platforms and backend systems. Familiarity with data governance frameworks, privacy compliance, and automated data quality checks. Hands-on experience with big data tools (Airflow, Databricks) and data observability platforms. Collaborative mindset and experience working with cross-functional teams including ML and analytics specialists. Curiosity and enthusiasm for continuous ...

MLflow) Experience with cloud-based ML tools (e.g. SageMaker), data and model versioning (e.g. DVC), CI/CD (e.g. GitHub Actions), workflow orchestration (e.g. Airflow/Dagster) and containerised solutions (e.g. Docker, ECS) nice to have Experience in code testing (unit, integration, end-to-end tests) Strong data engineering ...

team of 6–10 Data Engineers (onshore + offshore) Own the technical roadmap for the data platform (AWS + Databricks + dbt + Airflow) Architect scalable, cost-efficient and governed data solutions Drive best practices in engineering, testing, observability and documentation Act as senior technical point of contact … Experience 8+ years total experience with at least 3 years in a Lead/Principal Data Engineer role Expert-level AWS , Databricks , dbt and Apache Airflow Proven experience leading teams in a regulated financial services environment Strong stakeholder management and ability to translate business needs into technical solutions ...

Familiarity with cloud platforms such as AWS, Azure or Google Cloud and associated MLOps practices. Experience with automation and orchestration tooling such as UiPath, Airflow or Kubeflow. Experience and Capability Strong experience delivering AI or automation solutions within enterprise environments. Ability to work with cross functional teams and translate ...

background as a Machine Learning Engineer. Technical Skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib). Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services). Expertise: Solid understanding of computer science fundamentals and time-series ...

Experience designing both real-time and batch systems at enterprise scale Excellent documentation and stakeholder communication skills Nice to Have Databricks, dbt or Airflow experience Previous UK banking or financial services architecture role ...

Data services (indexing, slots, pruning, partitioning, clustering) Expert-level Kubernetes & GKE resource tuning Hands-on experience with Dataflow job pipelines and worker optimisation Strong Airflow/Composer knowledge (DAG design, scheduling, PodOperator) Strong Dataflow processing pipelines development & schedulers knowledge Deep understanding of Cloud Logging routing, sinks, exclusion filters Experience ...

services, including indexing, slots, pruning, partitioning, and clustering. Expert-level Kubernetes & GKE resource tuning. Hands-on experience with Dataflow pipelines and worker optimisation. Strong Airflow/Composer knowledge (DAG design, scheduling, PodOperator). Deep understanding of Cloud Logging, routing, sinks, and exclusion filters. Experience with Cloud Spanner autoscaling, indexing ...

Familiarity with cloud platforms (AWS, Azure or Google Cloud) and MLOps practices. Knowledge of automation technologies including RPA platforms (e.g. UiPath) and orchestration tools (Airflow, Kubeflow). Soft Skills: Excellent problem-solving and analytical capabilities. Strong written and verbal communication skills. Effective stakeholder management, with the ability to liaise ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

Gateway, and Kinesis. Integrating third-party APIs into the data platform and transforming data for CRM delivery. Migrating R-based data streams into modern Airflow-managed Python/DBT pipelines. Ensuring observability and reliability using CloudWatch and automated monitoring. Supporting both BAU and new feature development within the data … services including Lambda, API Gateway, S3, Kinesis, and CloudWatch. Strong programming ability in Python and data transformation skills using SQL and DBT. Experience with Airflow for orchestration and scheduling. Familiarity with third-party API integration and scalable data delivery methods. Excellent communication and the ability to work ...