19 of 19 Apache Airflow Jobs in the North West

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … highly scalable data engineering solutions that power our client next-generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with our client technical ...

organisation. Responsibilities: Build scalable ETL and ELT pipelines within AWS Develop and optimise data models in Amazon Redshift Create and manage workflows using Apache Airflow Integrate NetSuite and other Finance data sources Establish trusted data-quality, monitoring and validation processes Own solutions from initial requirements through to production … Experience Required: Strong commercial Python and SQL experience Hands‐on experience with AWS data services Amazon Redshift or comparable cloud data‐warehouse experience Apache Airflow or similar orchestration experience Strong ETL/ELT, data‐modelling and production pipeline knowledge The confidence to work directly with Finance and technical ...

Amazon Redshift AWS Lambda Amazon Athena Amazon RDS AWS IAM Amazon CloudWatch Experience with services such as Kinesis, EMR, Step Functions or MWAA/Airflow would be advantageous. AWS data engineering roles commonly use services such as S3, Glue, Redshift, Lambda, Kinesis and IAM for data platforms and pipelines. … Automated deployments across environments. Monitoring and logging. Release and configuration management. Production support and incident troubleshooting. Python or another scripting language. AWS certification. Terraform. Apache Airflow/MWAA. Databricks or Spark. Kafka/Kinesis. dbt. Experience working in Agile/Scrum environments. Experience 4+ years of experience ...

required, we are seeking Engineers who have experience with one or more of the following technological areas: Analytics & Platform Frameworks Orchestration like Airflow or AWS Glue Database Technologies Data processing libraries & Schema Management tools like Pandas and Pydantic Storage Formats: Familiarity with data types such as JSON and Parquet ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working ...

Python and SQL programming skills, with experience developing and maintaining data pipelines • Familiarity with ETL/ELT technologies and data streaming tools such as Airflow, Azure, SSIS, Spark or Kafka • Experience working with databases and data platforms such as PostgreSQL, SQL Server, Oracle, Snowflake or similar technologies • Knowledge ...

formats like Parquet. MLOps Exposure: Experience with tools and frameworks for training, tracking, deploying, and monitoring models in production (e.g., MLflow, Optuna, Prefect, or Airflow). Background in supply chainforecasting, enterprise AI, or retail analytics M.S. or Ph.D. in Computer Science, Artificial Intelligence, Statistics, Data Science, or a related ...

client communication, presentations), Expertise in at least one of the following domains: Experience with TensorFlow, PyTorch, XGBoost, LightGBM, Proficiency with Git, Linux, Docker, MLflow, Airflow, AWS/GCP/Azure, Solid SQL and data visualization skills. Additional Information Nice to have: Proven experience leading projects and managing teams, Experience ...

practical experience. Preferred Skills Experience with gradient-boosted tree models, neural networks, and optimization solvers in production. Familiarity with data orchestration tools (e.g., Prefect, Airflow, dbt) and modern data lake architectures. Experience working closely with data scientists to operationalize research code. Multi-cloud experience across AWS, GCP, and Azure. ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

Fluent English – for international communication and collaboration. Additional Information Nice to have: Experience with LLMs, or Time Series, Familiarity with tools like Docker, MLflow, Airflow, AWS/GCP/Azure, Background in research or R&D, Experience working in cross-functional or international teams. Soft Skills: Curiosity and problem ...

including building and publishing images to internal registries Implement Infrastructure as Code using Terraform (including network, security and compute resources) Support workflow orchestration, including Airflow Contribute to platform security, auditing and cost management best practices Participate as a core contributor within the team and support architecture discussions/working ...

with product squads and business stakeholders Develop robust, production-quality software in Python and build large-scale data workflows using orchestration platforms such as Airflow, while driving technical execution across complex initiatives. Use expertise in cloud computing (AWS and Azure) to build and optimize AI infrastructure Champion ML governance ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

analytics and AI/ML at enterprise scale.* Develop and optimise batch and streaming ETL/ELT pipelines using Azure Data Factory, PySpark, SQL, Apache Spark and Airflow, integrated with modern tooling like dbt and Git-based CI/CD.* Implement data cataloguing, lineage tracking and governance frameworks ...

will line manage, guide agile sprint delivery, and ensure robust, scalable data solutions across the organisation. You will work with BigQuery, GCP, Looker and Airflow on a legacy Microsoft SQL Server estate, collaborating with stakeholders to translate business requirements into high‐quality outputs and maintain governance standards while #J ...

organisation. You’ll manage sprint deliveries and guide an engineering team modernising our data landscape. Working across BigQuery, Google Cloud Platform (GCP), Looker, Airflow and a legacy Microsoft SQL Server estate, you’ll collaborate with stakeholders to understand business requirements and ensure the team deliver high quality work. This … hybrid work from home policy. Preferred Skills and Experience Recent commercial experience in any of the follow technologies would be advantageous: BigQuery, GCP, Looker, Airflow and Microsoft SQL Server Experience leading Agile teams, managing sprint deliverables and ceremonies Strong understanding of database design concepts and techniques with the ability ...