401 to 425 of 480 Apache Airflow Jobs in the UK

product and the company. Your Contribution Will Be The stack is modern: StarRocks as our primary analytical layer, Flink for processing, DBT for transformation, Airflow for orchestration and various tooling for surfacing insights. The hard work of making it genuinely reliable is still in front of us — and that …/lakehouse architecture patterns. Verbal and written fluency in English is required — we're a distributed team and written communication is important. Preferred Qualifications Airflow for orchestration. Experience with Apache Hudi. Experience with financial, transactional, or payment data. What We Offer at Yuno Competitive Compensation. Remote Work ...

bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than … Salesforce) and sales enablement platforms. Bonus Points If You Have: Experience selling to data teams, developers, or technical buyers. Background in data orchestration or Airflow-related technologies. Prior success in a startup or high-growth environment. The estimated total compensation for this role is up to £225,000, along ...

data platform, blending hands-on delivery with leadership and mentorship responsibilities. The role requires strong GCP experience (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), advanced Python and SQL, and a track record of building scalable batch and real-time pipelines. #J-18808-Ljbffr ...

data analysis packages such as Numpy, Pandas, etc. Experience with SQL and/or other database technologies Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

game production cycle or long-term project. A strong understanding of Photogrammetry workflows and software. Proficiency in Python. Experience with build systems like Jenkins, Airflow, or Celery. An understanding of a 3D asset pipeline and color management. Ability to think creatively and develop innovative solutions to complex problems. PLUSESPlease ...

staking, or other on-chain systems. Full-stack development with Python (FastAPI). Strong relational data modeling fundamentals. GitOps and Kubernetes. Job orchestration systems (Airflow, Prefect, etc.). For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice ...

taken a model through its full lifecycle before, from training through to serving it in production. You work with Python, PyTorch, SageMaker, and Airflow day to day. You default to owning problems end to end rather than waiting to be told what's next, and you're comfortable when ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

simulation platformEffectively integrate machine-learned graphics subsystems into the simulation platformImplement production quality software in PythonAbout youEssential Experience with workflow orchestration systems (e.g. Airflow, Dagster, Flyte, etc.) and/or developing data intensive applicationsExcellent development skills in PythonDeep knowledge of Kubernetes at the user levelGood sense of systems ...

Spark for building and optimizing large-scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure ...

Description Job Title Technical Lead Multi-tenant Data Integration Platform (GCP/Airflow) Role purpose Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables ...

/ML, or data engineeringExperience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with emerging technologies in data engineeringAbility ...

Talent Partners. Role We’re hiring a Senior Data Engineer to work on the analytics engineering platform at Depop. You’ll work on developing Airflow pipelines, the deployment of dbt models, managing data ingestion through Fivetran and developing internal tooling. You’ll also help enable data scientists … stakeholders on core datasets and key reporting. Responsibilities As a Senior Data Engineer, you can expect to: Build and maintain orchestration pipelines on Astronomer (Airflow), including automated model deployment for dbt users. Design and manage data ingestion and orchestration using Fivetran at scale Contribute to the team’s vision ...

FastAPI) a huge plus; Experience in Kubernetes, Docker, Spark and related monitoring tools (e.g. DataDog, Grafana, Prometheus) for DataOps a huge plus; Experience with Airflow a huge plus; Experience with dbt for pipeline modelling also beneficial; Skilled at shaping needs into a solid set of requirements and designing scalable … main programming language Databricks Spark for data processing Databricks/dbt/SQL for data modelling and analytics Streamlit for data applications and visualisations Airflow for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Terraform for our infrastructure ...

wherever your expertise can have the greatest impact. Our application data lives primarily in CockroachDB, with a modern data platform built around BigQuery, dbt, Airflow and pub/sub. Data flows both ways: into analytics and modelling, and back into the products our customers use. That culture translates into … comfortable working beyond a narrowly defined specialism when that is what the problem requires. Experience that will set you apart Experience with BigQuery, Airflow and dbt, or equivalent modern data technologies. Experience with CockroachDB, Postgres or other distributed/relational databases, including CDC or event-driven data pipelines. Knowledge ...

skills: - Strong commercial full stack engineering capabilities - node, Typescript, Python, Java etc - Solid front end understanding - React, Next, Vue.js etc - Experience building scalable pipelines (Apache Spark, Airflow, dbt etc) - Experience building data lakes, data warehouses and/or real-time streaming solutions would be a real plus - Strong … enterprise technology environment, then please apply now to be considered. £90,000 - £100,000 + remote working + excellent benefits React, Angular, TypeScript, Spark, Airflow, dbt, Kubernetes, OpenShift, AWS, Azure, PostgreSQL, GitHub, ArgoCD ...

MongoDB. And work with PostgreSQL too. The Senior Data Engineer will be well versed in Kafka. The Senior Data Engineer will know all about Apache Spark, Airflow and Iceberg. The Senior Data Engineer will have solid cloud infrastructure knowledge e.g. Google Cloud (GCP). The Senior Data Engineer ...

CDO. What you’ll need for this role Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures Demonstrated experience leading or technically mentoring a team of data engineers ...

data from multiple sources, turning chaos into a reliable, secure foundation. You’ll work with founders to shape architecture, be hands-on with Python, Airflow, dbt and distributed systems, and build infrastructure enabling autonomous software agents. This is a visa-sponsored, early-stage opportunity with high ownership and impact ...

Refactoring and upgrading existing ML models into modular, production-ready contracts Building resilient ML pipelines using DAG-based orchestration tools like Argo Workflows or Airflow, with proper retry logic and error handling Defining and automating evaluation pipelines using golden datasets so new models meet quality thresholds before release Designing ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg).Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg).dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing scalable ...

payments or fintech industry. Nice to Have Running multi‐tenant AI platforms or GPU/resource management at scale. Data pipelines and orchestration (Airflow, Prefect) and warehouses like Databricks, Snowflake, or BigQuery. Spanish proficiency. Tech Stack Languages — Python (AI/ML), Go and Kotlin (platform services). AI & LLMs … APIs and open models, agent frameworks, MCP, RAG. AI evaluation & observability — LangFuse, LangSmith, Braintrust, MLflow. Data & retrieval — vector databases, feature stores. Messaging — Apache Kafka, SQS. Databases — PostgreSQL, Redis. Infrastructure — AWS, Kubernetes, Docker, Terraform. Observability — Datadog, OpenTelemetry. CI/CD — GitHub Actions, ArgoCD. Version Control — Git/GitHub. What ...

/ML, data engineering)Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with AI evaluation and observability practices: evals ...

About Us Perk (formerly TravelPerk) is the intelligent platform for travel and spend management. Built to tackle the time-consuming, manual work that gets in the way of real work, our tools automate everything from ...