1 to 25 of 51 Apache Airflow Jobs in Manchester

Databrew. They are experienced in developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka. They are also proficient in using open source technologies like Apache Airflow and dbt, Spark/Python or Spark/Scala More ❯

object-oriented design principles, and data structures Extensive experience in developing microservices using Java, Python Experience in distributed computing frameworks like - Hive/Hadoop, Apache Spark. Good experience in Test driven development and automating test cases using Java/Python Experience in SQL/NoSQL (Oracle, Cassandra) database design … following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, Lambda Working experience with Terraform Experience in creating workflows for Apache Airflow About Roku Roku pioneered streaming to the TV. We connect users to the streaming content they love, enable content publishers to build More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

Gateway, and Python to maintain and enhance integrations. Analyze and understand existing R-based data pipelines created by data scientists. Migrate these pipelines into Airflow, dbt, and Terraform workflows. Modernize and scale legacy infrastructure running on AWS. Collaborate with engineering teams to ensure a smooth transition and system stability. … Phase 2: Legacy Pipeline Migration (Months 2-3) Analyze and understand existing R-based data pipelines created by data scientists. Migrate these pipelines into Airflow, dbt, and Terraform workflows. Modernize and scale legacy infrastructure running on AWS. Collaborate with engineering teams to ensure a smooth transition and system stability. … beneficial for interpreting existing scripts) Cloud & Infrastructure: AWS services including Lambda, API Gateway, S3, CloudWatch, Kinesis Firehose Terraform for infrastructure as code Orchestration & Transformation: Apache Airflow dbt CRM & Marketing Tools: Braze (preferred) Familiarity with other CRM/marketing automation tools such as Iterable or Salesforce Marketing Cloud is More ❯

Gateway, and Python to maintain and enhance integrations. Analyze and understand existing R-based data pipelines created by data scientists. Migrate these pipelines into Airflow, dbt, and Terraform workflows. Modernize and scale legacy infrastructure running on AWS. Collaborate with engineering teams to ensure a smooth transition and system stability. … Phase 2: Legacy Pipeline Migration (Months 2-3) Analyze and understand existing R-based data pipelines created by data scientists. Migrate these pipelines into Airflow, dbt, and Terraform workflows. Modernize and scale legacy infrastructure running on AWS. Collaborate with engineering teams to ensure a smooth transition and system stability. … beneficial for interpreting existing scripts) Cloud & Infrastructure: AWS services including Lambda, API Gateway, S3, CloudWatch, Kinesis Firehose Terraform for infrastructure as code Orchestration & Transformation: Apache Airflow dbt CRM & Marketing Tools: Braze (preferred) Familiarity with other CRM/marketing automation tools such as Iterable or Salesforce Marketing Cloud is More ❯

data modeling Proficient in data warehouse technologies (such as Amazon Redshift, Google BigQuery, or Snowflake) Hands-on experience with ETL tools and frameworks, including Apache Airflow, Talend, or dbt Strong programming ability in Python or another data-focused language Knowledgeable about data management best practices, including governance, security More ❯

documentation base, as well as contribute to it Strong software engineering, systems architecture, and unit testing capabilities and experience using pipeline tools such as Apache Airflow, Amazon SageMaker, or similar Familiarity with SQL and experience with AWS or other cloud providers Experience with GitLab CI/CD pipelines More ❯

and support automated monitoring systems to detect data anomalies, system failures, and performance issues and leverage advanced scripting and orchestration tools (e.g., Python, Bash, Apache Airflow) to automate workflows and reduce operational overhead. Root Cause Analysis & Incident Management: Lead post-incident reviews, perform root cause analysis for data More ❯

with big data technologies such as Hadoop, Spark, or Hive Familiarity with data warehousing and ETL tools such as Amazon Redshift, Google BigQuery, or Apache Airflow Proficiency in Python and at least one other programming language such as Java, or Scala Willingness to mentor more junior members of More ❯

improve engagement and performance Python (for scripting and data wrangling) SQL (strong experience in querying and transforming data) Experience with building analytics data layers Airflow, dbt (for orchestration and transformation) Cloud data warehouses like BigQuery or Redshift Terraform, Kubernetes (for infrastructure as code and deployment) Seniority level Seniority level More ❯

with big data technologies such as Hadoop, Spark, or Hive. Familiarity with data warehousing and ETL tools such as Amazon Redshift, Google BigQuery, or Apache Airflow. Proficiency in Python and at least one other programming language such as Java, or Scala. Willingness to mentor more junior members of the More ❯

operational targets crossing multiple departments Nice-to-have experience: Infrastructure as code (Terraform, cloud formation etc...) Cloud platforms (Aws, Azure, GCP etc..) Data orchestration (Airflow, Dagster etc...) The Team The Data & Analytics department serves as the central hub for data engineering and insights across our organization, comprising four essential More ❯

processes Hands-on experience with cloud platforms (GCP, AWS, Azure) and their data-specific services Proficiency in Python, SQL, and data orchestration tools (e.g., Airflow, DBT) Experience with modern data warehouse technologies (BigQuery, Snowflake, Redshift, etc.) Strong understanding of data modeling, data governance, and data quality principles Excellent communication More ❯

following: A track record of consolidating data sources, creating pipelines, and ideally a software engineering mindset. Excellent experience with Containers. Strong knowledge of Python, Airflow, SQL, and Linux. Proficient in AWS Lambda, ECS, Athena, S3, Kinesis, Glue, CloudWatch/Terraform/CDK, Tableau Online. They are particularly keen on More ❯

series forecasting, and model evaluation Familiarity with cloud platforms (AWS and/or GCP) Experience with productionising models using tools like Kubernetes, Docker, and Airflow Good knowledge of NLP techniques for text analysis Ability to design and run effective experiments (A/B, multivariate) Excellent communicator – able to explain More ❯

communication and stakeholder management skills. Desirable: Experience working with large-scale retail datasets (e.g., POS, CRM, supply chain). Familiarity with tools like dbt, Airflow, or MLflow. Master’s or PhD in Data Science, Statistics, Computer Science, or related fields. Benefits: Competitive salary and performance bonuses Flexible working options More ❯

coding practices. Results-driven, with the capacity to meet strict deadlines and service level agreements (SLAs). Ideally, familiarity with workflow management tools like Airflow, SQL database management, Linux command-line interface, Microsoft Dynamics 365 Finance and Operations, and Power Apps. The Package: Basic annual salary up to More ❯

Dataplex, Dataflow, Cloud Functions, Cloud Run, Pub/Sub, GCS, Vertex AI, GKE) or similar cloud platforms Familiarity with open-source data-stack tools (Airflow, DBT, Kafka, Great Expectation, etc.) Appreciation of modern cloud data stack, headless BI, analytics engineering, Data Mesh, and Lake House Although not essential, it More ❯

to provide the necessary infrastructure, resources, and interfaces for data loading and LLM workflows. Use Python and large-scale data workflow orchestration platforms (e.g. Airflow) to build software artifacts for ETL, integrating diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems Design More ❯

Data Scientists use (we don't expect applicants to have experience with all of these): Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. We are looking to grow our More ❯

platform supports industry-leading Data Science tools. While experience with all these technologies is not expected, familiarity is beneficial: Python and Databricks Spark, MLFlow, Airflow Google Cloud Platform dbt and BigQuery Examples of our data science work can be found on our engineering blog . What We’re Looking More ❯

Technologies That Our Data Scientists Use (we Don’t Expect You To Have Experience With All Of These) Python and Databricks Spark, MLFlow, and Airflow for ML Workflows Google Cloud Platform for our analytics infrastructure dbt and BigQuery for data modelling and warehousing Some examples of our data science More ❯

Technologies That Our Data Scientists Use (we Don't Expect You To Have Experience With All Of These) Python and Databricks Spark, MLFlow, and Airflow for ML Workflows Google Cloud Platform for our analytics infrastructure dbt and BigQuery for data modelling and warehousing Some examples of our data science More ❯