201 to 225 of 225 Apache Airflow Jobs in London

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

future AI deploymentsTroubleshoot integration failures, data inconsistencies, and production issuesKey SkillsPython & SQL (production-grade pipeline development)REST APIs, webhooks, OAuth, event-driven architectureOrchestration tools: Airflow, Prefect, or DagsterCloud platforms: AWS, GCP, or AzureDocker & Kubernetes Microsoft 365 & Microsoft CopilotDesirable SkillsVector databases and embedding pipelinesReal-time streaming (Kafka, Flink)RPA tooling … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar).Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes).Experience connecting ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

ingestion, processing, and governance of high-quality consumer survey data at scale, and drive migration from legacy pipelines to a modern stack (BigQuery, dbt, Airflow/Astronomer) with minimal disruption to clients. You’ll collaborate across Data Eng, Software Eng, ML Eng, Research and Data Operations to translate needs ...

events, to building and shipping production ML systems that balance performance, cost, and user experience. This means working across BigQuery (trillion-row datasets), Airflow orchestration, real-time serving infrastructure (BigTable), APIs, and constant collaboration with Product, Design, Engineering, and Platform teams.Key Responsibilities: Develop, test, and productionize … pipelines, including data, features, training, and servingMake technical decisions considering cost, latency, complexity, and maintainability Navigate distributed systems (BigQuery, BigTable, Airflow, DynamoDB) to build reliable, scalable solutionsSet up monitoring, A/B testing, and metrics frameworks to measure real user impactDebug complex issues across data pipelines, ML models ...

include: Designing and building scalable data pipelines using Python and SQL Developing solutions with Snowflake, dbt and modern orchestration tools such as Dagster or Airflow Building and improving cloud-native data infrastructure on AWS (or transferable GCP experience) Developing scalable data models and supporting the wider platform architecture Improving … platforms in fast-moving technology businesses. You'll ideally have: Strong Python and SQL experience Experience with Snowflake Experience using dbt Experience with Dagster, Airflow or another modern orchestration framework AWS experience (GCP also considered) Strong understanding of data modelling principles Experience building end-to-end data platforms ...

We tackle the most complex problems in quantitative finance, by bringing scientific clarity to financial complexity.From our London HQ, we unite world-class researchers and engineers in an environment that values deep exploration and methodical ...

will combine strong technical craft with business understanding. They will work hands-on data integration, transformation, building data warehouse and data products using dbt, Airflow, BigQuery, and BI tools while delivering business value to Dojo and our Customers.About the team... Our team is responsible for building data solutions … scoped data products.Build performant, cost-conscious data pipelines in BigQuery using dbt, and design insightful data visualisationDevelop scalable, performant, and reliable pipeline orchestrations using Airflow, dbt, Airbyte, and Fivetran.Use Python to automate workflows, validate data quality, and streamline analytical operations.Contribute to shared standards, design reusable metrics, and actively ...

support.What will you be working on Data Engineering Build and Optimise Pipelines: Develop, optimise, and maintain data pipelines using Python, REST APIs, DBT, and Airflow to ensure seamless data flow for compliance and risk management.Understanding Control Requirements & Continuous Monitoring: Partner closely with control owners across security domains (such … efficiency throughout the organisation.QualificationsTechnical Expectations:Proficient in Python, with experience in automation and building data pipelines.Strong expertise in working with tools like DBT and Airflow for ETL processes.Skilled in using REST APIs to develop and manage data integrations.Proficient in SQL and familiar with data visualization tools such as Looker ...

Why Sony Interactive Entertainment? Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

Why Sony Interactive Entertainment? Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

Why Sony Interactive Entertainment Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

Why Sony Interactive Entertainment Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

requiring strong data engineering expertise, experience working with financial data, and the ability to operate within a fast-paced investment environment. Key Skills Snowflake Airflow DBT Bloomberg Integrations Responsibilities Design, build and maintain scalable data pipelines and workflows. Develop and enhance data solutions using Snowflake, Airflow and DBT. … Drive best practices across data engineering, automation and platform support. Required Experience Experience within Investment Management, Asset Management or Financial Services Expertise with Snowflake, Airflow and DBT. Experience integrating Bloomberg data and services into enterprise platforms. Strong SQL and data modelling skills. Experience working with large-scale financial ...

requiring strong data engineering expertise, experience working with financial data, and the ability to operate within a fast-paced investment environment. Key Skills Snowflake Airflow DBT Bloomberg Integrations Responsibilities Design, build and maintain scalable data pipelines and workflows. Develop and enhance data solutions using Snowflake, Airflow and DBT. … Drive best practices across data engineering, automation and platform support. Required Experience Experience within Investment Management, Asset Management or Financial Services Expertise with Snowflake, Airflow and DBT. Experience integrating Bloomberg data and services into enterprise platforms. Strong SQL and data modelling skills. Experience working with large-scale financial ...

migrate data sets into their AWS Snowflake platform. Designing and implementing dbt models that power clean, consistent metrics. Buidl out orchestartion pipelines using Airflow Creating clear documentation for dashboards, metrics, and models. Ensuring optimal performance and cost efficiency across Snowflake Supporting a stakeholder-first approach to data usability … access. KEY SKILLS AND REQUIREMENTS Extensive experience operating within Snowflake environments Airflow for data orchestration. Proficiency in SQL and dbt, with a strong understanding of data modeling best practices. Strong communication skills with the ability to translate complex needs into elegant data solutions. Focus on usability, documentation, and data ...

Company DescriptionWise is a global technology company, building the best way to move and manage the world’s money.Min fees. Max ease. Full speed.Whether people and businesses are sending money to another country, spending abroad ...

value, and/or FICC options trading Experience with Linux‐based, concurrent, high‐throughput, low‐latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced ...

store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree ...

deliver high-quality consumer survey data at scale. This includes accelerating our migration from legacy pipelines into our modern data stack (BigQuery, dbt, Airflow/Astronomer), while ensuring near-zero disruption to client delivery. This role is critical to GWI’s future. Our Data Platform underpins everything we deliver … looking for: Product management experience in data platforms, data infrastructure, or highly technical B2B environments. Strong technical fluency — comfortable discussing data modelling, orchestration (e.g. Airflow), transformation frameworks (e.g. dbt), APIs, and platform architecture trade-offs. Experience working with modern data technologies such as Bigquery, Airflow, Astronomer. Ability ...

Exploration of new data sources to test whether they can be of value to Zempler Bank Implement models into production using tools such as Airflow, PyFlink, BentoML, Git, Jenkins and Octopus Participate in business projects Manage the relationship with the model stakeholder Assist in requirements gathering and project scope … highly imbalanced datasets for binary classification problems Event streaming data engineering tools such as Flink or Spark Experience with MLOps and tools such as Airflow, BentoML and MLFlow Use of Docker, CI/CD and SDLC (e.g. JIRA, Git, Jenkins, Octopus) Why Zempler Bank What makes us Zempler ...

ways to use AI to speed up development, automate repetitive tasks, and solve complex data problems.Automate Everything: Manage and scale our ETL pipelines using Airflow and Jenkins, ensuring our codebase is version-controlled in GitHub.Foster a Data Community: You aren't just building in a vacuum. You’ll lead … managing tool roadmaps.Direct experience connecting, cleaning, and harmonizing data from many different sources and tools into a unified BI layer.Experience with ETL automation tools (Airflow, Jenkins) and managing production codebases using GitHub.Technical Toolkit: Advanced proficiency in SQL. Strong technical background in Python and Java, with the ability ...

tuned foundational Large Language Models (LLMs), along with sub-agents, tools, and MCP integrationsArchitect end-to-end ML pipelines, integrating data processing (e.g. Spark, Airflow) with model training, evaluation, and deployment workflows Drive experimentation frameworks, including A/B testing and offline evaluation, to continuously improve model performance … fine-tuning Large Language Models (LLMs) and building production AI AgentsProven experience designing scalable data and ML pipelines using tools such as Spark, Airflow, or similar distributed systems Demonstrated ability to operate as a senior individual contributor, influencing technical strategy and decision-making without direct authorityExperience partnering effectively across ...

product, how ready are they for Fin.Making the data usable. Landing everything in well-modelled, well-tested dbt models in Snowflake orchestrated in Airflow, and syncing it out to Salesforce, Marketo, Clay and the rest of the GTM stack.Owning quality and trust. Instrumenting your pipelines with tests, alerts … outcomes rather than only technical ones.Experience with the modern data stack: dbt, a cloud warehouse (we use Snowflake), and a DAG orchestrator (we use Airflow).Experience building with LLMs in production: prompt design and evaluation, tool/function calling, web search and crawling, cost and latency control.Experience with entity ...

product, how ready are they for Fin. Making the data usable. Landing everything in well‐modelled, well‐tested dbt models in Snowflake orchestrated in Airflow, and syncing it out to Salesforce, Marketo, Clay and the rest of the GTM stack. Owning quality and trust. Instrumenting your pipelines with tests … rather than only technical ones. Experience with the modern data stack: dbt, a cloud warehouse (we use Snowflake), and a DAG orchestrator (we use Airflow). Experience building with LLMs in production: prompt design and evaluation, tool/function calling, web search and crawling, cost and latency control. Experience ...