1 to 25 of 28 Apache Airflow Jobs in the South East

Experience with cloud platforms such as AWS, Azure, or GCP. Knowledge of Data Warehouse and Data Lake architectures. Experience with orchestration tools such as Airflow, ADF, or equivalent. Understanding of data governance, data quality, and security practices. Familiarity with CI/CD pipelines and version control tools. Preferred Skills ...

stack evolves. Nice to HaveExperience with stream processing frameworks such as Flink or Spark Streaming. Experience with task orchestration tools such as Apache Airflow or similar systems. Familiarity with additional technologies: GraphQL, React, HTML5, JavaScript, CSS, Postgres, Gradle, BERT.Experience working with and designing infrastructures for large-scale data ...

stack evolves. Nice to Have Experience with stream processing frameworks such as Flink or Spark Streaming. Experience with task orchestration tools such as Apache Airflow or similar systems. Experience working with and designing infrastructures for large‐scale data processing (Hive, Snowflake, NoSQL databases). Experience with data governance ...

Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments CI/CD experience with a focus on production ownership and reliability ...

Requirements Passion for building scalable, resilient cloud data platforms. Strong experience with Databricks or Snowflake on AWS. Proven Python skills, including Spark and Airflow expertise. Advanced SQL skills and end-to-end data modelling experience. Experience building batch and real-time data integrations. Hands-on CICD skills with ...

Kanban. Strong analytical and problem‐solving skills, with a focus on delivering high-quality solutions. Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies. Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non‐technical audiences. What we offer ...

Scrum or Kanban.Strong analytical and problem-solving skills, with a focus on delivering high-quality solutions.Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies.Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non-technical audiences.Unfortunately we are unable ...

maintaining scalable data pipelines and integrations. Developing robust ETL and data transformation processes to support analytics and reporting requirements. Working with Snowflake, DBT, Apache Airflow, AWS, SQL and Python to deliver high-quality data solutions. Collaborating with Data Architecture, DevOps and business stakeholders to ensure trusted and accessible … ideally within AWS. Data modelling, transformation and orchestration methodologies. Delivering high-quality, well-documented and thoroughly tested data solutions. Experience with Snowflake, DBT and Apache Airflow. Interest in AI and emerging technologies within the data engineering landscape. THE BENEFITS: You will receive a salary ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Familiarity with Kafka concepts (producers ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

PostgreSQL migration projects. Knowledge of modern analytics platforms such as Databricks, Snowflake, Synapse, Fabric, or Redshift. Familiarity with ETL/ELT tooling including dbt, Airflow, Informatica, Talend, SSIS, or similar technologies. Contract Details Position: Data Architect Engagement: 3-6 Month Contract Day Rate: £400 per day IR35: Outside IR35 ...

cloud-native platforms in AWS and/or GCP.Experience designing and supporting data pipelines within complex technical environments.Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster.Experience implementing monitoring, logging, observability and platform governance practices.Strong understanding of software engineering principles, testing and deployment best practices.Ability to work collaboratively ...

native platforms in AWS and/or GCP.Experience designing and supporting data pipelines within complex technical environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience implementing monitoring, logging, observability and platform governance practices. Strong understanding of software engineering principles, testing and deployment best practices. Ability ...

data platform. You'll play a key role in building and optimising scalable data pipelines and analytics solutions, working across AWS, Snowflake, dbt and Airflow to deliver high-quality data products. The Company They are a well-established media organisation with a large and diverse audience, using data … models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline ...

enhancement programme. Investment Data Engineer, key skills: Strong commercial experience in Data Engineering, particularly using Snowflake Proven experience developing and maintaining workflows using Apache Airflow and DBT (Data Build Tool) Experience integrating and supporting Bloomberg data feeds, services or associated investment data solutions Strong SQL development and data … with minimal ramp-up time Strong analytical and problem-solving skills Excellent stakeholder communication and collaboration abilities Investment Data Engineer, Bloomberg, SQL, Python, DBT, Airflow Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built ...

Backend Engineering background Experience building APIs and system integrations Experience working within complex, data-intensive environments Strong understanding of software engineering best practices Desirable Airflow Snowflake AWS Exposure to data warehousing and modern analytics platforms Experience supporting reporting, commercial analytics, or performance data solutions IDEAL BACKGROUND We're especially ...

building and optimising Semantic Layers for BI, reporting and analytics consumption Expert Level Data Modelling Strong hands-on experience with: Snowflake AWS dbt Databricks Airflow Ability to work in a fast-paced delivery environment and remain close to the technical implementation Desirable -Customer Loyalty, Rewards or Membership Programme experience ...

We are recruiting for an Analytical Engineer on behalf of one of the UK's most recognised and celebrated personal lines insurers. The successful candidate will be working alongside a talented team of Data & AI ...

used in modern AI systems Strong Python programming experience A habit of making, and justifying, architectural calls, not just executing them Nice to Have: Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, MongoDB, Qdrant/Milvus/pgvector, Pandas/Polars, cloud infra (GCP/AWS/Azure ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

Azure , with GCP particularly desirable Knowledge of technologies such as BigQuery, Snowflake, Redshift, Synapse, Databricks or equivalent Experience with modern tooling such as dbt, Airflow and Terraform , or comparable technologies Strong knowledge of ETL/ELT and data integration patterns Understanding of CI/CD, automated testing and DataOps ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

include: Designing and building scalable data pipelines using Python and SQL Developing solutions with Snowflake, dbt and modern orchestration tools such as Dagster or Airflow Building and improving cloud-native data infrastructure on AWS (or transferable GCP experience) Developing scalable data models and supporting the wider platform architecture Improving … platforms in fast-moving technology businesses. You'll ideally have: Strong Python and SQL experience Experience with Snowflake Experience using dbt Experience with Dagster, Airflow or another modern orchestration framework AWS experience (GCP also considered) Strong understanding of data modelling principles Experience building end-to-end data platforms ...

infrastructure: AWS and/or GCP, Kubernetes APIs and services: REST, gRPC Distributed systems: event-driven architectures, including Kafka Orchestration Frameworks: LangGraph, LangChain, AirFlow, etc Agentic AI and ML Integration of commercial and open-source LLMs into agentic workflows Agent and orchestration frameworks such as LangChain, LlamaIndex, Semantic Kernel … workflow automation solutions. Development and integration of microservices-based architectures . Strong understanding of event-driven programming and distributed systems. Hands-on experience with Apache Kafka and messaging platforms. Experience working with Enterprise Data Hubs and data integration platforms. Strong PostgreSQL database design and development skills. Experience with ...