151 to 175 of 225 Apache Airflow Jobs in London

focused on what works in production. Comfortable owning ambiguity and making trade-overs. Languages: English A plus, but not required: workflow orchestration tools (Temporal, Airflow, Step Functions); experience building internal tools or agents; familiarity with regulated or enterprise environments; exposure to MLOps or AI evaluation frameworks. Why Planet Planet ...

team is working on the most impactful opportunitiesNICE TO HAVE BUT NOT ESSENTIALExperience delivering high quality data and self-serve analytics using tools like Airflow, dbt and Looker.Practically used + applied AI/machine learning techniques and built predictive models to get deeper insightsKnowledge of the FinTech ecosystemAdditional InformationWhat ...

computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge-sharing sessions) and helping ...

understanding of functional paradigms and type theory. Confident JVM knowledge. Modern programming languages knowledge in our primary languages (Java, Python, Clojure). Experience with Airflow or other Python‐based workflow orchestration tools. Proficiency in domain‐driven design and domain modeling. Exposure to Kubernetes, Docker, Linux, Kafka, RabbitMQ, or git. ...

theorem.A good understanding of functional paradigms and type theory.Confident JVM knowledge.Modern programming languages knowledge in our primary languages (Java, Python, Clojure).Experience with Airflow or other Python-based workflow orchestration tools.Proficiency in domain-driven design and domain modeling.Exposure to Kubernetes, Docker, Linux, Kafka, RabbitMQ, or git.Working knowledge ...

training techniques such as DDP or FSDP.Experience with observability tools such as Prometheus, Grafana, Datadog, or OpenTelemetry.Experience with data pipeline orchestration tools such as Airflow, Flyte, Ray, Metaflow, or Argo Workflows.Experience with containerisation and infrastructure tooling such as Docker, Kubernetes, or Terraform.Experience profiling or optimising ML systems, for example ...

such as DDP or FSDP. Experience with observability tools such as Prometheus, Grafana, Datadog, or OpenTelemetry. Experience with data pipeline orchestration tools such as Airflow, Flyte, Ray, Metaflow, or Argo Workflows. Experience with containerisation and infrastructure tooling such as Docker, Kubernetes, or Terraform.Experience profiling or optimising ML systems ...

data quality and consistency across platforms while complying with governance policies.What You Bring:Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs.Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB, Cosmos … lakehouse architectures.- Familiarity with big data processing frameworks like Spark.Data Engineering Knowledge:- Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory).- Understanding of JSON, XML, APIs, and data ingestion technologies.Cloud Experience:- Familiarity with cloud platforms (Azure, AWS, GCP) is a plus.Preferred Qualifications ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB … lakehouse architectures. - Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge: - Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory). - Understanding of JSON, XML, APIs, and data ingestion technologies. Cloud Experience: - Familiarity with cloud platforms (Azure ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB … lakehouse architectures. - Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge: - Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory). - Understanding of JSON, XML, APIs, and data ingestion technologies. Cloud Experience: - Familiarity with cloud platforms (Azure ...

used in modern AI systems Strong Python programming experience A habit of making, and justifying, architectural calls, not just executing them Nice to Have: Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, MongoDB, Qdrant/Milvus/pgvector, Pandas/Polars, cloud infra (GCP/AWS/Azure ...

used in modern AI systems Strong Python programming experience A habit of making, and justifying, architectural calls, not just executing them Nice to Have: Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, MongoDB, Qdrant/Milvus/pgvector, Pandas/Polars, cloud infra (GCP/AWS/Azure ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

About this role: At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software. You can’t spell “Data Orchestration” without “Data”, and we’re looking … data platforms from cloud-native component parts - we have a healthy mix of build vs. buy; sometimes building is the right choice. Experience with Apache Airflow. #LI-Hybrid At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion ...

3+ years experience working in version control environments using platforms such as Gitlab or Github 2+ years experience working with job orchestration tools like Apache Airflow, both as a pipeline developer and reviewer 2+ years experience with data ingestion and transformation frameworks (e.g., dbt, Fivetran, or custom/ ...

influencing architecture direction across multiple engineering and data teams. Key experience: Enterprise-scale Data Architecture AWS cloud data platforms Spark/PySpark/Python Airflow & ELT/data processing Data Lakehouse & Data Products Streaming/Event-Driven Architecture Data Governance Architecture standards, patterns & technical design reviews Experience with delivering ...

automated manual operational processes (not tests). Built solutions that replaced or eliminated manual work. Worked with operational/scheduling tools (Control-M, Airflow, Step Functions, Kubernetes jobs, etc.). Process Understanding & Design Can understand operational workflows and identify which parts can be automated. Works with ops teams ...

lifecycle, exception management. · Experience troubleshooting market and static data related issues that impact trade flow. · Familiarity with enterprise scheduling tools such as Active Batch, Airflow, Control-M, AutoSys · Experience using/building monitoring on platforms like ITRS Geneos, Zabbix ...

coursework in financial engineering, or industry internshipsExperience developing in a production-facing environment and familiarity with standard concepts and tooling, e.g., CI/CD, Airflow“Friends of Voleon” Candidate Referral ProgramIf you have a great candidate in mind for this role and would like to have the potential ...

using Databricks REST APIs, Prometheus, or Azure Monitor.• Maintain best practices for data engineering performance, testing, and documentation.Technical Skills Required• Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake.• Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies.• Experience tuning job configurations (e.g. … memory management, parallelism, caching, adaptive query execution).• Familiarity with orchestration tools: Databricks Workflows, Airflow, or Azure Data Factory.• Proficiency in Python or Scala for data engineering and pipeline development.• Hands-on experience with Azure, AWS, or multi-cloud Databricks deployments.• Knowledge of data storage layers (Azure Data Lake ...

idiomatic Python used for reliable, well-tested systems (not exploratory notebooks).• Design and operation of large-scale ETL/ELT pipelines, including orchestration (Airflow, Prefect, or similar) and reliability patterns such as retries, backfills, and exactly-once semantics.• Deep experience with Azure, including AKS, Helm, EventHub, observability tools … Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark.• Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics).• Strong KDB experience ...

database platforms (e.g., PostgreSQL, ClickHouse, MSSQL, Redis, FoundationDB).Familiarity with specific middleware (e.g., Kafka, Consul), HPC schedulers (e.g., Slurm), Kubernetes, or workflow orchestrators (e.g., Airflow, Prefect).Experience with on-premise deployments of development tools like GitLab, Artifactory, CICD, or JupyterHub.Knowledge of other programming languages such as Golang, Rust ...

Experience setting technical direction and influencing engineering practices across teams. Hands‐on experience with cloud ML platforms (e.g., AWS SageMaker) and orchestration tools (e.g., Airflow, Step Functions) Strong communication skills and the ability to collaborate across Data Science, Engineering, and Data Platforms. How you work You are comfortable operating ...

compelling narrativesHustler-mentality. You can take work beyond the analysis and get things done.Good to have:Experience in Fintech domain Knowledge of utilities like Airflow/Github/DBTExperience with user-facing products and data*** Please note we are not able to offer visa sponsorship for this level ***Additional ...

platform engineer and presenting a business case to a C‐minus‐1. Preferred, not required Experience with the modern data stack: Snowflake, Databricks, dbt, Airflow, BigQuery, Spark, or similar. Data observability, data quality, or AI observability experience is a strong plus. Familiarity with MEDDPICC, Command of the Message ...