20 of 20 Apache Airflow Jobs in the East of England

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

production-quality ETL/ELT pipelines and automated data workflows. Experience integrating heterogeneous data sources, databases and APIs. Experience with workflow orchestration tooling (e.g. Airflow, Dagster or dbt) for scheduling and operating production data pipelines. Good understanding of data quality, metadata, lineage, testing and reproducibility. Ability to evaluate technical ...

frameworks for large-scale batch and near real-time processing, with hands-on experience using orchestration and transformation technologies such as Airflow and dbt. Strong knowledge of software engineering best practices, including coding standards, code reviews, testing strategies, version control systems such as Git, and CI/CD processes. ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

analytics Model and manage data across data warehouses and lakes and operational databases Build batch and streaming workflows, orchestrated with tools such as Airflow, Prefect, Dagster, or dbt Own data quality, validation, lineage, and observability ensuring the data feeding our models is accurate, timely, and trustworthy Engineer features … similar) Technical Skills Python (essential) – pandas, NumPy, and the modern data/ML ecosystem SQL – advanced querying, modelling, and optimisation Data pipelines & orchestration – Airflow, Prefect, Dagster, or dbt Data warehouses/lakes – BigQuery, Snowflake, Redshift, or Databricks ML frameworks – PyTorch, TensorFlow, or scikit‐learn LLM & AI tooling – OpenAI, Anthropic ...

Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions. Applied ML in production: embeddings and semantic ...

advanced analytics — ideally in a hybrid PLG/sales-led SaaS environmentStrong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster)Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables)Experience working directly with product ...

advanced analytics - ideally in a hybrid PLG/sales-led SaaS environment Strong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster) Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables) Experience working directly with ...

Batch, ECS, EKS, S3, RDS or Postgres, Athena, SageMaker, Bedrock, or equivalent services. Scientific workflow and data orchestration frameworks such as Nextflow, Dagster, Airflow or similar. Platform security, privacy or governance controls for sensitive biomedical or clinical data. Identity platforms and federation technologies such as Entra ...

multi-modal representation learning is a plus We’d love to see that you’ve worked with big data systems (Spark, S3, and Airflow) and can program (Java, Scala, or Python) Good understanding of system architecture. Have experience in big data technologies and streaming architecture, data pipelines, etc. Demonstrated ...

Batch, ECS, EKS, S3, RDS or Postgres, Athena, SageMaker, Bedrock, or equivalent services. Scientific workflow and data orchestration frameworks such as Nextflow, Dagster, Airflow or similar. Platform security, privacy or governance controls for sensitive biomedical or clinical data. Identity platforms and federation technologies such as Entra ...

Skill Requirements:*** Strong experience in **Solution Architecture** and **Cloud-Native Architecture Design*** Hands-on expertise in **Snowflake Architecture*** Experience with **Data Lakehouse architectures**, including **Apache Iceberg*** Proficiency in designing and managing **S3-based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge … large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across:* Snowflake* Airflow* Iceberg* Focus on **platform reliability, scalability, and observability*** Experience designing and operating **data platforms** in production environments #J-18808-Ljbffr ...

devices and platforms to surface key metrics. The role emphasizes coding in Python, strong SQL, and experience with large data ecosystems (HDFS, Hive, Spark, Airflow). You will influence architecture and own initiatives from concept to delivery. Hybrid work from Cambridge Office. #J-18808-Ljbffr ...

frameworks for large-scale batch and near real-time processing, with hands-on experience using orchestration and transformation technologies such as Airflow and dbt. Strong knowledge of software engineering best practices, including coding standards, code reviews, testing strategies, version control systems such as Git, and CI/CD processes. … frameworks for large-scale batch and near real-time processing, with hands-on experience using orchestration and transformation technologies such as Airflow and dbt. Strong knowledge of software engineering best practices, including coding standards, code reviews, testing strategies, version control systems such as Git, and CI/CD processes. ...

data analysis packages such as Numpy, Pandas, etc. • Experience with SQL and/or other database technologies • Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis • Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

Senior Data Engineer to design, build and optimise scalable data solutions across our AWS data platform. You will work with Redshift, Python, Airflow and AWS services to develop reliable data pipelines, support real‐time data processing, and enable high‐quality analytics and business insights. Responsibilities Build and maintain scalable … data pipelines and ETL processes using Python, SQL, Airflow, AWS Glue and Redshift. Develop batch and near real‐time data solutions to support operational and analytical needs. Optimise data performance, reliability and cost efficiency across the platform. Implement monitoring, alerting and operational support processes. Champion engineering best practice, code ...

technologies like Spark, Flink etc.,Solid grasp of Kubernetes, CI/CD and TerraformWorking knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbtAbility to provide clear input, guide, empower junior members of the team to achieve desired outcomesFamiliarity with machine learning and AI tools … technologies like Spark, Flink etc.,Solid grasp of Kubernetes, CI/CD and TerraformWorking knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbtAbility to provide clear input, guide, empower junior members of the team to achieve desired outcomesFamiliarity with machine learning and AI tools ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...