51 to 75 of 255 Apache Airflow Jobs in London

tooling, or cloud cost management across at least one major cloud provider (Azure, AWS, or GCP). Technical Skills Data engineering: strong proficiency with Apache Spark, dbt, Airflow/Azure Data Factory or equivalent orchestration, Data Lake/Apache Iceberg, and advanced SQL for large‐scale transformations. ...

with using and integrating BI tools such as QuickSight, Looker, Power BI, Tableau • Business Intelligence (BI) knowledge/experience • Knowledge on tools like Terraform, Airflow, DBT and Harness • Modern data engineering practises The environment is that of Facebook or Google, relaxed open with time to think and make ...

marketing teams to align data solutions with business needs Requirements Strong SQL skills and experience with ETL/ELT and pipeline orchestration tools (e.g., Airflow, dbt, Data Factory) Experience with cloud data platforms (e.g., Azure, AWS, or GCP) Proficiency in data modelling (dimensional, entity-relationship, event-driven) Understanding ...

MarTech, and marketing teams to align data solutions with business needsRequirementsStrong SQL skills and experience with ETL/ELT and pipeline orchestration tools (e.g., Airflow, dbt, Data Factory)Experience with cloud data platforms (e.g., Azure, AWS, or GCP)Proficiency in data modelling (dimensional, entity-relationship, event-driven)Understanding ...

/or Genetics datasets Proficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks , etc.) Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational support Experience operationalising analytical workflows ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture design to coaching ...

Omics and/or Genetics datasetsProficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks, etc.)Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational supportExperience operationalising analytical workflows ...

consulting experience within BFSI, leading complex enterprise transformations. Platform Expertise: Deep technical proficiency in modern data platforms (Databricks, Snowflake) and toolsets (dbt, Airflow, Kafka). Advanced Architecture & AI: Mastery of Lakehouse, Data Mesh, Streaming, MLOps, and GenAI integration to drive continuous innovation. Governance & Compliance: Expertise in zero-trust security ...

wider stakeholders. Technical Environment Experience with some of the following technologies would be beneficial: Python SQL Cloud platforms – AWS, Azure or GCP Databricks Apache Spark/PySpark Kafka/event-driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed ...

high-growth environments.Technical Skills:Experience building and operating ETL/ELT pipelines.Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with some of these or similar ...

environments. Technical Skills: Experience building and operating ETL/ELT pipelines. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with some of these or similar ...

Google Cloud. Key skills: Strong Python & SQL experience Hands-on Azure Databricks, Delta Lake & Unity Catalog Strong GCP/BigQuery/GCS experience Apache Airflow/Cloud Composer for pipeline orchestration Experience building ELT/data pipelines and cross-cloud data transfers Understanding of IAM, RBAC, secrets management ...

datasets— hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations.• SQL— strong proficiency, including query optimization on large datasets.Nice-to-Haves• Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines.• AWS experience (Azure or GCP equally welcome).• Machine Learning experience, particularly applied ...

components such as S3, Kinesis, DMS, SNS, Glue, Athena, RDSExperience with implementing Rest APIs.Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

databases like Snowflake/Redshift/BigQueryExperience with Python/PySparkExperience with implementing Rest APIs.Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

databases like Snowflake/Redshift/BigQueryExperience with Python/PySparkExperience with implementing Rest APIs.Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practicesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

Optimise Python performance through effective use of loops , vectorization , threading , and understanding of the Global Interpreter Lock (GIL) . Desirable Experience Experience with Kubeflow , Airflow , or dbt . Hands-on experience with Google Cloud Platform (GCP) . Knowledge of containerisation (Docker, Kubernetes) and CI/CD practices. Familiarity with ...

relational/graph data models. Cloud & Infrastructure: Hands-on experience with AWS services, Infrastructure as Code (Terraform), Docker, Kubernetes, and orchestration tools (dbt, Airflow). Standards: Experience working within GDS (Government Digital Service) design standards and secure development frameworks. eg GOV.UK Frontend, GOV.UK Design System and the Service Manual ...

smart every day.Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship modelling and Dimensional modellingExperience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery) and Cloud ...

every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship modelling and Dimensional modellingExperience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery) and Cloud ...

business stakeholders and translate requirements into scalable data solutionsExperience mentoring, coaching or supporting the development of other analytics engineersKnowledge of orchestration tools such as Airflow or ComposerFamiliarity with cloud platforms, particularly Google Cloud Platform (GCP)And it would be great if you had any of the following...Experience with real ...

care). Knowledge of regulatory reporting and treasury operations in retail bankingExposure to Python, Go or similar languages.Experience working with orchestration frameworks such as Airflow/LuigiHave previously used dbt, dataform or similar tooling.Used to AGILE ways of working (Kanban, Scrum)The Interview Process:Our interview process involves ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks* Ability to think strategically and relate architectural decisions ...