101 to 125 of 849 Apache Airflow Jobs in England

But if not? We’ll help you get there: Understanding of cloud computing security concepts Proficiency with IaC tools like Terraform or CloudFormation. Experience with workflow orchestration tools (e.g., Airflow, Dagster). Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in Agile environments with tools like Jira and Git. More ❯

incl. query efficiency and optimization) Expert experience of Cloud Data Platforms (AWS, Snowflake and/or Databricks) Qualification preferred, not mandatory Significant experience of Automation and Integrations tools (FiveTran, Airflow, Astronomer or similar) Significant experience with IoC tools (Terraform, Docker, Kubernetes or similar) Significant experience with CI/CD tools (Jenkins, GitHub Actions, CircleCI or similar) Preferred Experience: Experience More ❯

Proficiency in AWS, Google Cloud Platform, or Azure. What Would Make You Stand Out Experience with PySpark and structured streaming. Experience with orchestrating complex workflows using tools such as Airflow, Dagster or Prefect. Familiarity with infrastructure as code and with CI/CD tools such as Jenkins, Github Actions, etc to automate testing, deployment, and integration processes. Hands-on More ❯

But if not? We’ll help you get there: Understanding of cloud computing security concepts Proficiency with IaC tools like Terraform or CloudFormation. Experience with workflow orchestration tools (e.g., Airflow, Dagster). Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in Agile environments with tools like Jira and Git. More ❯

incl. query efficiency and optimization) Expert experience of Cloud Data Platforms (AWS, Snowflake and/or Databricks) Qualification preferred, not mandatory Significant experience of Automation and Integrations tools (FiveTran, Airflow, Astronomer or similar) Significant experience with IoC tools (Terraform, Docker, Kubernetes or similar) Significant experience with CI/CD tools (Jenkins, GitHub Actions, CircleCI or similar) Preferred Experience: Experience More ❯

libraries (e.g., GDAL, GeoPandas, Rasterio). •Familiarity with data cataloguing tools and practices. •Prior experience in a startup or high-growth tech company. •Familiarity with containerisation (Docker), orchestration tools (Airflow, Prefect), and CI/CD workflows. •Understanding of foundational MLOps and data-centric AI practices. •Experience of working in an Agile environment. What We Offer: •The opportunity to shape More ❯

similar transformation frameworks; Hands-on experience with AWS data services (Redshift, S3, Glue, Lambda); Proficiency with Python for data processing and pipeline development; Experience with workflow orchestration tools like Airflow or similar; Knowledge of streaming data technologies such as Kafka; Familiarity with infrastructure-as-code tools (Terraform, CloudFormation); Experience with version control systems (Git) and CI/CD practices More ❯

Expertise in Python and SQL for data processing, analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and unstructured data More ❯

plenty of incredible developers at Octopus Energy who are willing to teach if you're willing to learn! Required experience: Python Git Nice to have: SQL dbt Github CircleCI Airflow Kubernetes Terraform A Cloud warehouse provider e.g. Databricks, GCP, Snowflake AWS We aren't necessarily looking for someone who is "10-out-of-10" in all these areas; but More ❯

focused data solutions We'd love to talk to you if: You've got solid experience working with Python, SQL, Spar and data pipeline tools such as dbt or Airflow You're comfortable working across cloud platforms - especially AWS (Glue, Lambda, Athena), Azure (Data Factory, Synapse), or GCP (Dataflow, BigQuery, Cloud Composer) You have a good understanding of data More ❯

such as Machine Learning, Artificial Intelligence, Statistical & Data Analysis, Computational Algorithms, Data Engineering, etc. Experience working with a variety of complex, large datasets. Experience building automated pipelines (e.g., Jenkins, Airflow, etc.). Experience building or understanding end-to-end, distributed, and high-performance software infrastructures. Proven ability to work collaboratively as part of a team. Proven ability to take More ❯

Experience as Senior Engineer on a data warehouse, ideally Snowflake Experience building and/or maintaining a CI/CD pipeline Experience using modern orchestration tooling e.g. Prefect, Luigi, Airflow Experience developing infrastructure in Terraform or a similar IAC tool Experience using Docker A positive and proactive attitude to problem solving A team player who is willing to help More ❯

Experience required Extensive background in delivering production-grade, data-centric applications for quantitative trading and analytics, with demonstrated expertise in: The python data engineering stack (Polars, Parquet, FastAPI, Jupyter, Airflow, Streamlit, Ray) High-performance data stores and query engines (Starburst, Snowflake) Real-time streaming analytics technologies (Kafka, Flink) Cloud container technologies (AWS, Azure, GCP, Docker, Kubernetes) Proven success in More ❯

learn to name a few of the open-source libraries we use extensively. We implement the systems that require the highest data throughput in Java and C++. We use Airflow for workflow management, Kafka for data pipelines, Bitbucket for source control, Jenkins for continuous integration, Grafana + Prometheus for metrics collection, ELK for log shipping and monitoring, Docker and More ❯

Experience with Python or other scripting languages Good working knowledge of the AWS data management stack (RDS, S3, Redshift, Athena, Glue, QuickSight) or Google data management stack (Cloud Storage, Airflow, Big Query, DataPlex, Looker) About Our Process We can be flexible with the structure of our interview process if someone's circumstances or timescales require it but our general More ❯

Cloud Data Warehouse - Snowflake AWS Data Solutions - Kinesis, SNS, SQS, S3, ECS, Lambda Data Governance & Quality - Collate & Monte Carlo Infrastructure as Code - Terraform Data Integration & Transformation - Python, DBT, Fivetran, Airflow CI/CD - Github Actions/Jenkins Nice to Have Experience Understanding of various data architecture paradigms (e.g., Data Lakehouse, Data Warehouse, Data Mesh) and their applicability to different More ❯

succeed in this role you will already have some of the following skills: Hands-on experience of working with open source data orchestration systems such as Dagster, Prefect, or Airflow Solid understanding of distributed compute engines such as Spark/Databricks Confidence using Docker, Kubernetes, and Helm in cloud environments Experience building software for public cloud environments (Azure preferred More ❯

scikit-learn, etc. We implement the systems that require the highest data throughput in Java. We implement most of our long running services and analytics in C#. We use Airflow for workflow management, Kafka for data pipelines, Bitbucket for source control, Jenkins for continuous integration, ELK for logs, Grafana, Prometheus & InfluxDb for metrics, Docker and Kubernetes for containerisation, OpenStack More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

We are seeking a skilled, motivated Data Engineer to join our dynamic and innovative team. As a Data Engineer in the Data & Insight Team you will work as part of a team of data engineers to design, develop, test and More ❯

in the office in average. In this role you will be working within a new, greenfield division of the business, using a brand-new technology stack including Snowflake, dbt, Airflow and AWS. This function provide data for Machine Learning and Artificial Intelligence capabilities, helping them to provide the best possible service offering to their customers. You'll work on … a strong financial backing, and the chance to make a real impact! We're looking for the following experience: Extensive hands-on experience with Snowflake Extensive experience with dbt, Airflow, AWS and Terraform Exellent scripting skills in SQL Experience developing solutions entirely from scratch Great communication skills, with the ability to understand and translate complex requirements into technical solutions More ❯

based work as well as migration tasks Due to your seniority you will also be tasked with mentoring junior engineers Work extensively and proficiently with Snowflake, AWS, DBT, Terraform, Airflow, SQL, and Python. Requirements: 3+ Years data engineering experience Snowflake experience Proficiency across an AWS tech stack DBT expertise Terraform experience Expert SQL and Python Data Modelling Data Vault … Apache Airflow My client have very limited interview slots and they are looking to fill this vacancy ASAP. I have limited slots for 1st stage interviews next week so if you’re interest, get in touch ASAP with a copy of your most up to date CV and email me at m.fox@tenthrevolution.com or call me on More ❯