1 to 25 of 34 Apache Airflow Jobs in the North West

processing. In this role, you will architect and maintain production-grade data products and APIs, leveraging technologies such as Java/Scala, SQL, Spark, Airflow, and Kubernetes to deliver reliable, high-performance solutions. You will collaborate with cross-functional partners across Finance, Accounting, and Analytics to translate complex business … efficient code, ensuring adherence to best practices through code reviews Big Data Engineering Design, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache Airflow Optimise data storage, retrieval, and processing systems to ensure reliability, scalability, and performance Develop and fine-tune complex queries ...

software development team Hands‐on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands‐on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical ...

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical ...

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … support highly scalable data engineering solutions that power Moodys next-generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moodys technical teams ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

test, and deploy ETL pipelines to source, cleanse, and optimise data at scale. Automate and orchestrate data workflows: Develop and maintain orchestration solutions (e.g. Apache Airflow), implementing testing, alerting, monitoring, and data quality checks as first‐class concerns. Ensure data quality and observability: Embed data testing and validation … event streaming platforms including Kafka and its core concepts on topics, messages, and consumers. Experience building and operating automated data workflows using tools like Apache Airflow. Experience implementing data quality and validation checks throughout pipelines to ensure accuracy, completeness, consistency, and reliability. Practical experience implementing monitoring, observability and alerting ...

working knowledge Nice-to-have Databricks → AWS (or cross-platform) migration experience — the single strongest signal Parity/statistical testing methodology Data orchestration (Airflow, dbt, Step Functions) Data governance & compliance (PII/GDPR); regulated industry background (iGaming strongly preferred, but FS, banking considered) Soft skills Comfortable working ...

processing. In this role, you will architect and maintain production-grade data products and APIs, leveraging technologies such as Java/Scala, SQL, Spark, Airflow, and Kubernetes to deliver reliable, high-performance solutions. You will collaborate with cross-functional partners across Finance, Accounting, and Analytics to translate complex business … clean, maintainable, and efficient code, ensuring adherence to best practices through code reviewsBig Data EngineeringDesign, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache AirflowOptimise data storage, retrieval, and processing systems to ensure reliability, scalability, and performanceDevelop and fine-tune complex queries using Apache ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

engines and relational or analytical storage (for example Kafka, Spark, PostgreSQL). Orchestration. Defining and running data workflows with modern orchestration frameworks (for example Airflow, Dagster or Prefect). Data quality and lineage. Tools and techniques for data testing, documentation and lineage (for example Great Expectations or dbt). … tracks (we'd love depth in one of these; you don't need all of them) Azure/Databricks Azure Data Lake Storage, Databricks, Apache Spark, Delta Lake, Azure Data Factory, MLflow; and, for the AI edge, Databricks Mosaic AI, Vector Search and Unity Catalog, or Azure OpenAI, Azure ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor ...

Solid understanding of data modelling and modern architectures Strong knowledge of data governance, security and compliance Experience with orchestration tools such as Airflow Familiarity with CI/CD pipelines, Git workflows and automated testing Exposure to real-time, IoT or environmental telemetry data Experience & Qualifications Bachelor’s degree ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

degree in Computer Science or equivalent practical experienceExperience developing finance applications is a plusWorking experience with Terraform is a plusExperience creating workflows for Apache Airflow is a plus#LI-JB6What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work ...

data quality, validation, reconciliation, and audit controls across betting, trading, and customer data domains. Develop streaming and micro-batch data solutions using AWS Kinesis, Apache Kafka, or Amazon MSK to support real-time sportsbook and in-play data feeds. Build and maintain data solutions using AWS services including … architectures AWS certifications are a bonus You know your tools Solid experience with Python, SQL, and Spark — and pipeline tools such as dbt or Airflow Comfortable with data quality frameworks and testing — Great Expectations, dbt tests, or custom validation layers Experience with infrastructure-as-code (Terraform ...

degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience. Preferred Skills Experience with data orchestration tools (e.g., Prefect, Airflow, dbt). Familiarity with MLOps tooling such as MLflow, or experience supporting ML workloads in production. Exposure to LLM or agentic application patterns ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

following domains: Computer Vision, Large Language Models (LLMs), Time Series Analysis, Experience with TensorFlow, PyTorch, XGBoost, LightGBM, Proficiency with Git, Linux, Docker, MLflow, Airflow, AWS/GCP/Azure, Solid SQL and data visualization skills. Nice to have Proven experience leading projects and managing teams, Experience in presales, client ...

practical experience. Preferred Skills Experience with gradient-boosted tree models, neural networks, and optimization solvers in production. Familiarity with data orchestration tools (e.g., Prefect, Airflow, dbt) and modern data lake architectures. Experience working closely with data scientists to operationalize research code. Multi-cloud experience across AWS, GCP, and Azure. ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows.Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...