401 to 425 of 477 Apache Airflow Jobs in the UK

data platform, blending hands-on delivery with leadership and mentorship responsibilities. The role requires strong GCP experience (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), advanced Python and SQL, and a track record of building scalable batch and real-time pipelines. #J-18808-Ljbffr ...

data analysis packages such as Numpy, Pandas, etc. Experience with SQL and/or other database technologies Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

game production cycle or long-term project. A strong understanding of Photogrammetry workflows and software. Proficiency in Python. Experience with build systems like Jenkins, Airflow, or Celery. An understanding of a 3D asset pipeline and color management. Ability to think creatively and develop innovative solutions to complex problems. PLUSESPlease ...

staking, or other on-chain systems. Full-stack development with Python (FastAPI). Strong relational data modeling fundamentals. GitOps and Kubernetes. Job orchestration systems (Airflow, Prefect, etc.). For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice ...

taken a model through its full lifecycle before, from training through to serving it in production. You work with Python, PyTorch, SageMaker, and Airflow day to day. You default to owning problems end to end rather than waiting to be told what's next, and you're comfortable when ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

simulation platformEffectively integrate machine-learned graphics subsystems into the simulation platformImplement production quality software in PythonAbout youEssential Experience with workflow orchestration systems (e.g. Airflow, Dagster, Flyte, etc.) and/or developing data intensive applicationsExcellent development skills in PythonDeep knowledge of Kubernetes at the user levelGood sense of systems ...

Spark for building and optimizing large-scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure ...

Description Job Title Technical Lead Multi-tenant Data Integration Platform (GCP/Airflow) Role purpose Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables ...

/ML, or data engineeringExperience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with emerging technologies in data engineeringAbility ...

Talent Partners. Role We’re hiring a Senior Data Engineer to work on the analytics engineering platform at Depop. You’ll work on developing Airflow pipelines, the deployment of dbt models, managing data ingestion through Fivetran and developing internal tooling. You’ll also help enable data scientists … stakeholders on core datasets and key reporting. Responsibilities As a Senior Data Engineer, you can expect to: Build and maintain orchestration pipelines on Astronomer (Airflow), including automated model deployment for dbt users. Design and manage data ingestion and orchestration using Fivetran at scale Contribute to the team’s vision ...

FastAPI) a huge plus; Experience in Kubernetes, Docker, Spark and related monitoring tools (e.g. DataDog, Grafana, Prometheus) for DataOps a huge plus; Experience with Airflow a huge plus; Experience with dbt for pipeline modelling also beneficial; Skilled at shaping needs into a solid set of requirements and designing scalable … main programming language Databricks Spark for data processing Databricks/dbt/SQL for data modelling and analytics Streamlit for data applications and visualisations Airflow for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Terraform for our infrastructure ...

wherever your expertise can have the greatest impact. Our application data lives primarily in CockroachDB, with a modern data platform built around BigQuery, dbt, Airflow and pub/sub. Data flows both ways: into analytics and modelling, and back into the products our customers use. That culture translates into … comfortable working beyond a narrowly defined specialism when that is what the problem requires. Experience that will set you apart Experience with BigQuery, Airflow and dbt, or equivalent modern data technologies. Experience with CockroachDB, Postgres or other distributed/relational databases, including CDC or event-driven data pipelines. Knowledge ...

skills: - Strong commercial full stack engineering capabilities - node, Typescript, Python, Java etc - Solid front end understanding - React, Next, Vue.js etc - Experience building scalable pipelines (Apache Spark, Airflow, dbt etc) - Experience building data lakes, data warehouses and/or real-time streaming solutions would be a real plus - Strong … enterprise technology environment, then please apply now to be considered. £90,000 - £100,000 + remote working + excellent benefits React, Angular, TypeScript, Spark, Airflow, dbt, Kubernetes, OpenShift, AWS, Azure, PostgreSQL, GitHub, ArgoCD ...

MongoDB. And work with PostgreSQL too. The Senior Data Engineer will be well versed in Kafka. The Senior Data Engineer will know all about Apache Spark, Airflow and Iceberg. The Senior Data Engineer will have solid cloud infrastructure knowledge e.g. Google Cloud (GCP). The Senior Data Engineer ...

CDO. What you’ll need for this role Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures Demonstrated experience leading or technically mentoring a team of data engineers ...

data from multiple sources, turning chaos into a reliable, secure foundation. You’ll work with founders to shape architecture, be hands-on with Python, Airflow, dbt and distributed systems, and build infrastructure enabling autonomous software agents. This is a visa-sponsored, early-stage opportunity with high ownership and impact ...

Refactoring and upgrading existing ML models into modular, production-ready contracts Building resilient ML pipelines using DAG-based orchestration tools like Argo Workflows or Airflow, with proper retry logic and error handling Defining and automating evaluation pipelines using golden datasets so new models meet quality thresholds before release Designing ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg).Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg).dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing scalable ...

payments or fintech industry. Nice to Have Running multi‐tenant AI platforms or GPU/resource management at scale. Data pipelines and orchestration (Airflow, Prefect) and warehouses like Databricks, Snowflake, or BigQuery. Spanish proficiency. Tech Stack Languages — Python (AI/ML), Go and Kotlin (platform services). AI & LLMs … APIs and open models, agent frameworks, MCP, RAG. AI evaluation & observability — LangFuse, LangSmith, Braintrust, MLflow. Data & retrieval — vector databases, feature stores. Messaging — Apache Kafka, SQS. Databases — PostgreSQL, Redis. Infrastructure — AWS, Kubernetes, Docker, Terraform. Observability — Datadog, OpenTelemetry. CI/CD — GitHub Actions, ArgoCD. Version Control — Git/GitHub. What ...

/ML, data engineering)Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with AI evaluation and observability practices: evals ...

About Us Perk (formerly TravelPerk) is the intelligent platform for travel and spend management. Built to tackle the time-consuming, manual work that gets in the way of real work, our tools automate everything from ...

help strengthen pre and post-change validation, enhance observability, and support the continuous improvement of automation capabilities. Technology Stack Linux/UNIX Python Ansible Apache Airflow Prometheus Grafana Loki VMware F5 What We're Looking For We're looking for engineers with: Strong Linux/UNIX administration fundamentals ...

risk through data, building robust pipelines and driving insights to support Wise's secure and compliant framework. You will work with Python, DBT, Airflow and REST APIs to create scalable data solutions, collaborate across security domains, and define KPIs for continuous monitoring and improved efficiency. #J-18808-Ljbffr ...