126 to 150 of 458 Apache Airflow Jobs in England

maintaining data pipelines at scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms ...

every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship modelling and Dimensional modellingExperience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery) and Cloud ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

agile environments at scale Domain expertise in retail, hospitality or consumer technology is valuable Familiarity with modern data stack tooling (e.g. Snowflake, dbt, Looker, Airflow, or #J-18808-Ljbffr ...

using: Kafka as our message transport Flink for (near) real time processing Datahub as our catalog BigQuery as our warehouse Airflow (GCP Composer) for scheduling DBT for data transformation Montecarlo for monitoring The platform here encompasses the end-to-end, first for real time/streaming use cases ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks* Ability to think strategically and relate architectural decisions ...

another language used for data processing Batch and event-driven processing. Experience with Kafka or an equivalent streaming system, a workflow orchestrator such as Airflow, Dagster or Prefect, and transformation tooling such as dbt or equivalent SQL-based workflows Cloud warehouse, lake or lakehouse experience , and integrating data from ...

Data Engineer within GCP environments Hands-on experience with BigQuery, Dataflow and Spanner Strong programming capability in Python, Java or Scala Experience with Apache Beam/Dataflow and distributed processing frameworks Experience designing and managing ETL/ELT pipelines Solid understanding of data modelling and database design Experience with … workflow orchestration tools such as Apache Airflow Strong understanding of data governance, security and scalability Desirable Skills for the GCP Data Engineer: Experience with Pub/Sub, Cloud Composer or Cloud Data Fusion Exposure to CI/CD, DevOps and Infrastructure as Code Experience with real-time streaming ...

operating ETL/ELT pipelines and large-scale data architectures. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with similar technologies is essential). Ability ...

operating ETL/ELT pipelines and large‐scale data architectures. Hands‐on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with similar technologies is essential). Ability ...

deeply hands-on with Python, PySpark and Spark Modernising and migrating large-scale data pipelines across AWS Designing and challenging approaches across AWS Glue, Airflow and associated technologies Shaping scalable data architectures, frameworks and engineering patterns Supporting the development of new data products and reusable capabilities Championing AI-assisted … quality and delivery speed Raising engineering standards and capability across multiple teams What we're looking for Deep hands-on Python/PySpark and Apache Spark expertise Strong AWS data engineering , particularly AWS Glue Strong Apache Airflow experience Experience delivering complex data transformation or modernisation programmes Strong ...

skills; experience with Oracle databases . Experience designing and supporting ETL/ELT pipelines and data quality frameworks . Working knowledge of Autosys& Apache Airflow Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift) Familiarity with AWS S3 for large-scale data storage. ...

production problems and taking responsibility for critical infrastructure. Startup Mindset: Proactive, resourceful and comfortable taking ownership in a fast-moving engineering environment. Bonus : with Apache Airflow, real-time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial-market systems. Benefits An Ambitious, Collaborative Culture: Join ...

self-motivated, inquisitive mindset and a methodical, agile approach to problem-solving Solid Linux administration and networking skills, plus Git-based workflows Familiarity with Apache Airflow, Dask and JupyterHub Experience with modern build systems (especially Bazel) and CI/CD pipelines. What we offer At Macquarie ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

systems, orchestration, governance, and scalable data product design Experience with technologies and ecosystems such as GCP, AWS, Azure, Databricks, Snowflake, Kafka, Spark, dbt, Kubernetes, Airflow, ML/AI enablement, or similar modern platform tooling Strong commercial acumen and ability to engage credibly with senior technical and business stakeholders Experience ...

triaging, escalating, and driving incidents to resolution under pressure. What you'll do: Design, develop, and maintain high-performance, product-centric data pipelines using Airflow, DBT, and Python. Architect and optimize the massive-scale data warehouse and lakehouse that serves as our single source of truth for all customer ...

data technology such as BigQuery, Snowflake, Redshift, Hive, Spark, Vertica, or StarRocks; Experience with ETL/ELT tools or orchestration frameworks such as Airflow, dbt, or internal frameworks; Proficiency in at least one programming language for data and services, such as Python, Java, Golang, Kotlin, or JavaScript/TypeScript ...

/ELT frameworks, and cloud data platforms such as Azure, AWS, or GCP. Data modelling and pipeline orchestration using tools such as Airflow, Data Factory, dbt, or similar. Integrate CDP data sources across digital, CRM, transactional, and behavioural data; support identity resolution, consent handling, audience activation, journey events, trigger ...

Snowflake and dbt experience: modelling, semantic layers, agents & skills, testing, performance and cost tuning. Production experience with a pipeline orchestrator, ideally Dagster (or similar Airflow, Prefect), including dependent workloads, partial failures, and observability. Modern cloud infrastructure (Azure/AWS/GCP) and Kubernetes experience deploying and operating containerised workloads ...

related technical operational roles, ideally within a sustainability context, but not required. Hands-on experience with modern data stack tools (e.g., Snowflake, DBT, Airflow)Strong SQL and Python fluencyAdvanced data visualization (e.g., Tableau, PowerBI)Experience writing technical specifications, reviewing code or data models, and supporting engineering workflowsDemonstrated ability ...

metrics layers. Experience applying AI to Analytics Engineering workflows or developing AI-ready data products. Knowledge of orchestration and ingestion tools such as Airflow, Dagster or Fivetran. Python experience for automation or data engineering tasks. Experience working in regulated financial services or another highly data-driven organisation. Snowflake experience ...

data platforms, or significantly evolving existing ones. Experience with data quality and observability, using dbt or complementary tooling. Familiarity with orchestration tools such as Airflow, Dagster, or Prefect. Experience with semantic layers, metrics frameworks, or analytics consumption patterns. Exposure to data governance, access control, or analytics enablement at scale. ...

various platforms Experience briefing, designing and implementing ETL solutions end-to-end, from requirements gathering through to deployment Experience with data orchestration tools (e.g. Airflow, Prefect) and building scalable data pipelines Strong Git and collaborative development skills (e.g. GitHub, GitLab) Strong understanding of data fundamentals: MDM, data integrity best ...

Spark and Databricks AWS services (e.g. IAM, S3, Redis, ECS) CI/CD tooling and best practices Streaming and batch data systems (e.g. Kafka, Airflow, RabbitMQ) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa Subsidised counselling and coaching with Self Space Cycle ...