1 to 25 of 37 Apache Airflow Jobs in London

automate workflows, and deliver secure, high-performance data platforms using AWS-native technologies. Key Responsibilities Develop and maintain data pipelines using Python , SQL , and Apache Airflow . Build and manage containerised data applications using Docker . Design and optimise solutions using AWS services including Redshift, OpenSearch, Lambda, Glue … data governance, security, and best practice standards. Essential Skills Active Security Clearance Strong experience with Python and SQL . Hands-on experience with Apache Airflow and Docker . Proven experience with AWS data services, including Redshift, OpenSearch, Lambda, Glue, Step Functions, and Batch . Experience with Git , GitLab ...

producing a clear blueprint for future delivery. Key responsibilities include: Mapping and documenting the current data architecture and reporting ecosystem Analysing existing Python and Apache Airflow data pipelines Tracing data flows from multiple partner data sources through Google Cloud environments into reporting platforms Reviewing and documenting transformation logic … platforms Excellent documentation and communication skills Experience with Tableau and Power BI environments Knowledge of Google Cloud Platform (GCP) Familiarity with PostgreSQL, Python and Apache Airflow Experience within digital media, publishing, advertising technology or affiliate commerce environments Understanding of programmatic advertising and revenue reporting models Previous experience supporting ...

producing a clear blueprint for future delivery. Key responsibilities include: Mapping and documenting the current data architecture and reporting ecosystem Analysing existing Python and Apache Airflow data pipelines Tracing data flows from multiple partner data sources through Google Cloud environments into reporting platforms Reviewing and documenting transformation logic … platforms Excellent documentation and communication skills Experience with Tableau and Power BI environments Knowledge of Google Cloud Platform (GCP) Familiarity with PostgreSQL, Python and Apache Airflow Experience within digital media, publishing, advertising technology or affiliate commerce environments Understanding of programmatic advertising and revenue reporting models Previous experience supporting ...

delivery setting. Proven experience across data analysis, extraction, transformation, and loading, with hands-on knowledge of relevant technologies such as Spark, Python, Kafka, SQL, Airflow, and cloud-based ETL services. Experience designing schemas for data warehouses and data marts, assessing data quality issues, and integrating quality checks into pipelines. ...

cost optimization, Horizon Catalog, Cortex). dbt (modular analytics engineering, transformations, testing). AWS Python proficiency as a data engineer and Orchestration tools (e.g. Airflow) Data and Dimensional modelling (star schema, Kimball/Inmon methodologies, OBT). AI - copilots, agentic AI, MCP Servers Strong understanding of DataOps, CI/ ...

Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience … with modern data warehousing solutions (e.g., Snowflake, BigQuery, Redshift) Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows) Strong understanding of data modelling and lakehouse architecture Databricks certifications (e.g., Databricks Certified Data Engineer) Experience with streaming data pipelines (Kafka, Structured Streaming) Experience with CI/CD and DevOps practices Knowledge ...

environments Solid experience with SQL and data modelling concepts Strong analytical and problem-solving skills Nice to Have Skills Experience with data pipeline tools (Airflow, Kafka, Spark) Knowledge of cloud platforms (AWS, Azure or GCP) Familiarity with relational and NoSQL databases Additional programming experience (Python, Scala) Essential Requirements Active ...

Nice to Have: * Microsoft Azure Solutions Architect Expert* AWS Solutions Architect Professional* Databricks Certifications* CDMP Certification* Terraform, Bicep or Infrastructure-as-Code experience* dbt, Airflow or modern orchestration frameworks* Experience supporting AI, GenAI or RAG-based architectures* Experience within Financial Services, Healthcare, Life Sciences, Energy, Retail or Manufacturing Robert ...

Redux, Redux Hooks, and other modern state management libraries. Kotlin on Android and Swift on iOS. Data Analytics/Engineering: SQL, Python, Snowflake, dbt, Airflow & Looker. AI at Marshmallow: Sierra.ai for conversational AI, Python and LangChain for agentic AI solutions, SageMaker and MLflow for Machine Learning, Cursor and Claude ...

approaches, fit to the use case Build and maintain transformation layers in dbt (or equivalent), with appropriate testing, documentation and lineage Operate orchestration tooling (Airflow, Dagster, Prefect or equivalent) and the workflows around it Manage cloud data warehouses and lakehouses (Snowflake, BigQuery, Redshift, Synapse, Databricks) Implement data quality, testing … Python skills (or Scala/Java where relevant) for pipeline development and tooling Hands-on experience with dbt or equivalent transformation tooling Orchestration experience (Airflow, Dagster, Prefect or equivalent) Production experience with at least one major cloud warehouse/lakehouse (Snowflake, BigQuery, Redshift, Synapse, Databricks) Cloud platform experience ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Snowflake Toolkit: Hands-on with Snowpipe, Streams, Tasks, Dynamic Tables, and Stored Procedures. Nice to Have: Exposure to DBT, Snowpark, Snowflake AI/ML, Airflow, and Snowflake administration (RBAC, warehouse sizing). Mindset: Strong problem-solving abilities with an ownership mindset for end-to-end delivery. This ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

operating production data systems Strong SQL and Python skills with expertise in large-scale data processing Hands-on experience with modern data stack tools — Airflow, dbt, ClickHouse, BigQuery, Snowflake, or similar Experience with event-driven, real-time or near real-time architectures A first-class approach to data correctness ...

implementing batch and stream data processing systems. Experience with Datadog, SQL, Terraform, Docker, AWS (ECS, Kinesis, SQS, CloudWatch), GCP (BigQuery). Knowledge of Airflow, Kafka, Flink, dbt, Kubernetes is highly beneficial. Good understanding of AI platforms, experience building infrastructure for LLM orchestration (e.g., vector databases, RAG pipelines, GPU scaling ...

creating. You'll have experience with: AWS and cloud-native technologies Snowflake or similar cloud data platforms Data orchestration tools such as Airflow, Prefect or similar Amazon S3 and modern data architectures Postgres and semi-structured data models Designing and building event-driven architectures and scalable systems Building ...

debugging production systems, and making architectural decisions alongside your team. Deep experience with large-scale data processing and distributed systems (e.g., Spark, Flink, Databricks, Airflow, or similar). You can evaluate trade-offs, not just delegate them. A track record of leading multiple high-performing engineering teams – while staying ...

Strong hands-on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast-paced fintech or technology environment A collaborative leadership style with the ability ...

platforms and pipelines (batch and real-time ingestion). Hands-on familiarity with: Snowflake (or equivalent cloud data warehouse), Looker (or other BI tools), Airflow, Pub/Sub, or similar orchestration and ingestion frameworks. Experience with data infrastructure monitoring and optimization. Strong communication skills. Ability to translate business goals ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like private healthcare & dental insurance, a generous work from ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

data and performance insights What We're Looking For Strong background in data analytics Advanced SQL skills and experience building data pipelines (e.g. DBT, Airflow) Experience with marketing analytics tools (e.g. Google Analytics, GTM, Braze, Appsflyer, Firebase) Proven experience running A/B tests and defining KPIs and attribution ...

GenAI or advanced NLP projects such as NER, table extraction, OCR integrations, or GNN solutions. Familiarity with orchestration and deployment tools: Airflow, Redis, Flask/Django/FastAPI, SQL, R‐Shiny/Dash/Streamlit. Openness to evaluate and adopt emerging technologies and programming languages as needed. Public contributions ...

commercial experience in analytics within a marketing or digital environment Advanced SQL skills and experience building data pipelines with tools such as DBT or Airflow Experience with marketing analytics platforms such as Google Analytics, GTM, Braze, Appsflyer, or Firebase Strong understanding of experimentation, A B testing, and attribution modelling ...