251 to 275 of 315 Apache Airflow Jobs in London

coursework in financial engineering, or industry internshipsExperience developing in a production-facing environment and familiarity with standard concepts and tooling, e.g., CI/CD, Airflow"Friends of Voleon" Candidate Referral ProgramIf you have a great candidate in mind for this role and would like to have the potential ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow If you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website. ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow If you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website. ...

Databricks REST APIs, Prometheus, or Azure Monitor. Maintain best practices for data engineering performance, testing, and documentation. Technical Skills Required Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake. Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies. Experience tuning job configurations (e.g. … memory management, parallelism, caching, adaptive query execution). Familiarity with orchestration tools: Databricks Workflows, Airflow, or Azure Data Factory. Proficiency in Python or Scala for data engineering and pipeline development. Hands-on experience with Azure, AWS, or multi-cloud Databricks deployments. Knowledge of data storage layers (Azure Data Lake ...

cost efficiency, scalability, and reliabilityContribute to the target architecture and long-term platform directionData Engineering & DeliveryDesign and build robust, scalable data pipelines (Snowflake, dbt, Airflow, ADF, Python)Apply strong practices in test-driven development, monitoring, logging, and observabilityOwn more complex technical solutions and problem-solvingSupport production issues and drive … Python, with hands-on experience building data pipelines and transformations at scaleHands-on experience with a modern cloud data stack, including Snowflake, dbt, Airflow, and Databricks, on a cloud platform (Azure preferred)Demonstrated experience delivering data lineage and traceability within a regulated environment (e.g. SOX, financial services, healthcare ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based pipelines … production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys & Apache Airflow. Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with AWS S3 for large-scale data storage. ...

including model deployment, monitoring, evaluation, and lifecycle management Strong hands‐on understanding of modern data and AI tooling, ideally including SQL, Python, Snowflake, dbt, Airflow, Metabase, Tableau, and AWS‐based data services Experience with data ingestion, streaming, and change data capture tooling, such as Fivetran, Estuary, or similar platforms … standards, and enablement for analytics engineers Ability to oversee API‐based and batch data extraction processes, particularly those built in Python and orchestrated using Airflow Understanding of machine learning tooling and lifecycle management, ideally including DVC, MLflow, Evidently AI, or equivalent platforms for experiment tracking, versioning, monitoring, and evaluation ...

product and the company. Your Contribution Will Be The stack is modern: StarRocks as our primary analytical layer, Flink for processing, DBT for transformation, Airflow for orchestration and various tooling for surfacing insights. The hard work of making it genuinely reliable is still in front of us — and that …/lakehouse architecture patterns. Verbal and written fluency in English is required — we're a distributed team and written communication is important. Preferred Qualifications Airflow for orchestration. Experience with Apache Hudi. Experience with financial, transactional, or payment data. What We Offer at Yuno Competitive Compensation. Remote Work ...

bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than … Salesforce) and sales enablement platforms. Bonus Points If You Have: Experience selling to data teams, developers, or technical buyers. Background in data orchestration or Airflow-related technologies. Prior success in a startup or high-growth environment. The estimated total compensation for this role is up to £225,000, along ...

data analysis packages such as Numpy, Pandas, etc. Experience with SQL and/or other database technologies Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

staking, or other on-chain systems. Full-stack development with Python (FastAPI). Strong relational data modeling fundamentals. GitOps and Kubernetes. Job orchestration systems (Airflow, Prefect, etc.). For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice ...

taken a model through its full lifecycle before, from training through to serving it in production. You work with Python, PyTorch, SageMaker, and Airflow day to day. You default to owning problems end to end rather than waiting to be told what's next, and you're comfortable when ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

simulation platformEffectively integrate machine-learned graphics subsystems into the simulation platformImplement production quality software in PythonAbout youEssential Experience with workflow orchestration systems (e.g. Airflow, Dagster, Flyte, etc.) and/or developing data intensive applicationsExcellent development skills in PythonDeep knowledge of Kubernetes at the user levelGood sense of systems ...

Spark for building and optimizing large-scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure ...

Talent Partners. Role We’re hiring a Senior Data Engineer to work on the analytics engineering platform at Depop. You’ll work on developing Airflow pipelines, the deployment of dbt models, managing data ingestion through Fivetran and developing internal tooling. You’ll also help enable data scientists … stakeholders on core datasets and key reporting. Responsibilities As a Senior Data Engineer, you can expect to: Build and maintain orchestration pipelines on Astronomer (Airflow), including automated model deployment for dbt users. Design and manage data ingestion and orchestration using Fivetran at scale Contribute to the team’s vision ...

FastAPI) a huge plus; Experience in Kubernetes, Docker, Spark and related monitoring tools (e.g. DataDog, Grafana, Prometheus) for DataOps a huge plus; Experience with Airflow a huge plus; Experience with dbt for pipeline modelling also beneficial; Skilled at shaping needs into a solid set of requirements and designing scalable … main programming language Databricks Spark for data processing Databricks/dbt/SQL for data modelling and analytics Streamlit for data applications and visualisations Airflow for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Terraform for our infrastructure ...

wherever your expertise can have the greatest impact. Our application data lives primarily in CockroachDB, with a modern data platform built around BigQuery, dbt, Airflow and pub/sub. Data flows both ways: into analytics and modelling, and back into the products our customers use. That culture translates into … comfortable working beyond a narrowly defined specialism when that is what the problem requires. Experience that will set you apart Experience with BigQuery, Airflow and dbt, or equivalent modern data technologies. Experience with CockroachDB, Postgres or other distributed/relational databases, including CDC or event-driven data pipelines. Knowledge ...

MongoDB. And work with PostgreSQL too. The Senior Data Engineer will be well versed in Kafka. The Senior Data Engineer will know all about Apache Spark, Airflow and Iceberg. The Senior Data Engineer will have solid cloud infrastructure knowledge e.g. Google Cloud (GCP). The Senior Data Engineer ...

CDO. What you’ll need for this role Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures Demonstrated experience leading or technically mentoring a team of data engineers ...

data from multiple sources, turning chaos into a reliable, secure foundation. You’ll work with founders to shape architecture, be hands-on with Python, Airflow, dbt and distributed systems, and build infrastructure enabling autonomous software agents. This is a visa-sponsored, early-stage opportunity with high ownership and impact ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg).Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg).dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing scalable ...

payments or fintech industry. Nice to Have Running multi‐tenant AI platforms or GPU/resource management at scale. Data pipelines and orchestration (Airflow, Prefect) and warehouses like Databricks, Snowflake, or BigQuery. Spanish proficiency. Tech Stack Languages — Python (AI/ML), Go and Kotlin (platform services). AI & LLMs … APIs and open models, agent frameworks, MCP, RAG. AI evaluation & observability — LangFuse, LangSmith, Braintrust, MLflow. Data & retrieval — vector databases, feature stores. Messaging — Apache Kafka, SQS. Databases — PostgreSQL, Redis. Infrastructure — AWS, Kubernetes, Docker, Terraform. Observability — Datadog, OpenTelemetry. CI/CD — GitHub Actions, ArgoCD. Version Control — Git/GitHub. What ...

About Us Perk (formerly TravelPerk) is the intelligent platform for travel and spend management. Built to tackle the time-consuming, manual work that gets in the way of real work, our tools automate everything from ...