201 to 225 of 237 Apache Airflow Jobs in the UK

If you need support in completing the application or if you require a different format of this document, please get in touch with at UKI.recruitment@tcs.com or call TCS London Office number 02031552100 with the subject line: “Application Support Request More ❯

Our Commodities and Global Markets data analytics team within technology is building a leading data and analytics platform. We partner and build batch and real-time analytics solutions for traders, quants, desk analysts, and central data teams across the business. More ❯

We're looking for a Senior Data Engineer to join Pleo and help us in our journey in our Business Analytics team. This team is responsible for delivering and enhancing high-quality, robust data solutions that drive commercial performance, revenue More ❯

Technical requirements: Highly proficient in Python. Experience working with data lakes; experience with Spark, Databricks. Understanding of common data transformation and storage formats, e.g. Apache Parquet. Good understanding of cloud environments (ideally Azure), and workflow management systems (e.g. Dagster, Airflow, Prefect). Follow best practices like code review More ❯

one or multiple systems. You know how to create repeatable and reusable products. Experience with workflow management tools such as Nextflow, WDL/Cromwell, Airflow, Prefect and Dagster Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows and pipelines Understanding of common data transformation and storage … formats, e.g. Apache Parquet Awareness of data standards such as GA4GH ( ) and FAIR ( ). Exposure of genotyping and imputation is highly advantageous Benefits: Competitive base salary Generous Pension Scheme - We invest in your future with employer contributions of up to 12%. 30 Days Holiday + Bank Holidays - Enjoy More ❯

including CI/CD, observability, version control, and testing Architect and evolve our data platform, including data warehousing (Redshift), lakehouse (Databricks), and orchestration tools (Airflow, Step Functions) Lead data governance, cataloging, compliance, and security efforts to ensure trustworthy and well-managed data Essential skills & experience: Proven leadership or management … and implementing scalable data platforms and ETL/ELT pipelines Knowledge of data warehousing, data lake architectures, and orchestration tools like Step Functions and Airflow Experience with infrastructure as code (e.g., Terraform) Understanding of data governance and quality practices Strong communication skills to explain technical concepts and influence stakeholders More ❯

tools and tech. The team are using Databricks and AWS and theyre keen for someone whos worked across data warehouse architecture, orchestration tools like Airflow, and configuration-driven development. Youll also work closely with analysts, scientists and other business teams, so youll need to be able to explain complex … modelling, and ETL/ELT pipelines Experience using tools like Databricks, Redshift, Snowflake, or similar Comfortable working with APIs, CLIs, and orchestration tools like Airflow Confident using Git and familiar with CI/CD processes (Azure DevOps or similar) Experience working in an Agile environment A proactive mindset you More ❯

tech. The team are using Databricks and AWS and they’re keen for someone who’s worked across data warehouse architecture, orchestration tools like Airflow, and configuration-driven development. You’ll also work closely with analysts, scientists and other business teams, so you’ll need to be able to … modelling, and ETL/ELT pipelines Experience using tools like Databricks, Redshift, Snowflake, or similar Comfortable working with APIs, CLIs, and orchestration tools like Airflow Confident using Git and familiar with CI/CD processes (Azure DevOps or similar) Experience working in an Agile environment A proactive mindset — you More ❯

Gateway, and EventBridge • Work with DynamoDB, Aurora Serverless, and S3 for cloud-native data solutions • Collaborate with frontend (Vue.js) and data platform engineers (Snowflake, Airflow) • Contribute to CI/CD pipelines, observability, and day-2 operational tooling • Partner with product and architecture teams to translate customer needs into working … able to interface confidently with both technical and non-technical audiences Bonus Experience • Familiarity with IaC frameworks (CloudFormation, Terraform, SAM) • Exposure to Snowflake, DBT, Airflow, or cost analytics/data pipeline tools • Knowledge of FinOps practices or cost intelligence platforms • Experience contributing to open-source platforms or cloud-native More ❯

Akkodis is a global leader in engineering, technology, and R&D, harnessing the power of connected data to drive digital transformation and innovation for a smarter, more sustainable future. As part of the Adecco Group, Akkodis combines the expertise of More ❯

Akkodis is a global leader in engineering, technology, and R&D, harnessing the power of connected data to drive digital transformation and innovation for a smarter, more sustainable future. As part of the Adecco Group, Akkodis combines the expertise of More ❯

World Wide Technology (WWT), a global technology integrator and IT solutions provider. World Wide Technology, established in 1990 in St. Louis, Missouri, collaborates with OEMs like Cisco and Dell EMC to offer infrastructure security and custom app development services to More ❯

Senior Data Engineer Hybrid working (3 days per week onsite in London) 6 Month contract initially, with good scope for extension Market rates (Umbrella-PAYE) One of our blue chip clients is looking for a Senior Data Engineer to join More ❯

On-site, Hybrid Manchester, Manchester, United Kingdom QA We are Xebia - a place where experts grow. For nearly two decades now, we've been developing digital solutions for clients from many industries and places across the globe. Among the brands More ❯

you'll define the path forward for data, integration, and governance , ensuring our technology aligns with business objectives. Work with cutting-edge tools like Apache Spark, Databricks, Kafka, Airflow, and Azure , while overseeing SQL Server, ETL, data pipelines, and streaming platforms . You'll also drive automation and … from you! You Will Data Strategy & Architecture Develop and maintain the data architecture roadmap, balancing legacy and modern data solutions. Evaluate emerging technologies (e.g., Apache Spark, Kafka) to future-proof our data landscape. Define and enforce data integration standards, ensuring consistency across systems. Solution Design & Implementation Oversee data flow … pipeline design, transformations, and warehouse architectures. Lead the technical implementation of SQL Server, Azure SQL, Databricks, and Airflow pipelines. Champion data mesh principles and federated analytics using Starburst (Trino). Enable real-time data streaming and analytics through Kafka. Integration & Automation Design and implement data flows across Dynamics More ❯

Lead Data Engineer - Snowflake, DBT, Airflow - London - Up to £100k I'm working with a key client of ours here at TRG who are looking to grow out their Data & Analytics function. My client are globally renowned for being a leader within their relative field. Whilst they are a … So they offer very flexible working arrangements through both WFH options and flexi working hours. Experience required... Expert in Snowflake Strong DBT experience Strong Airflow experience Expert knowledge and understanding of Data Warehousing Strong AWS experience This is a great opportunity to join outstanding organisation who pride themselves on … touch ASAP. Send across your CV to (url removed) or alternatively, give me a call on (phone number removed). Keywords: Snowflake, DBT, SQL, Airflow, AWS, Engineer, DWH, Data Warehouse, Data Warehousing, Architecture, London More ❯

flows through the pipeline. Collaborate with research to define data quality benchmarks . Optimize end-to-end performance across distributed data processing frameworks (e.g., Apache Spark, Ray, Airflow). Work with infrastructure teams to scale pipelines across thousands of GPUs . Work directly with the leadership on the … optimizing classifiers. Experience managing large-scale datasets and pipelines in production. Experience in managing and leading small teams of engineers. Expertise in Python , Spark , Airflow , or similar data frameworks. Understanding of modern infrastructure: Kubernetes , Terraform , object stores (e.g. S3, GCS) , and distributed computing environments. Strong communication and leadership skills More ❯

robust version control Setting up monitoring and alerting frameworks to track model drift, data quality, and inference health Leveraging orchestration tools such as Dagster, Airflow, or Prefect to manage and scale ML workflows Supporting ongoing infrastructure migration or optimisation initiatives (e.g. improving cost efficiency, latency, or reliability) Partnering with … experience deploying ML models into production environments, including both batch and real-time/streaming contexts Proficiency working with distributed computing frameworks such as Apache Spark , Dask, or similar Experience with cloud-native ML deployment , particularly on AWS , using services like ECS, EKS, Fargate, Lambda, S3, and more Familiarity … with orchestration and workflow scheduling tools such as Dagster , Airflow , or Prefect Knowledge of CI/CD best practices and tools (e.g. GitHub Actions, Jenkins, CodePipeline) Exposure to monitoring and observability tools for ML systems (e.g. Prometheus, Grafana, DataDog, WhyLabs, Evidently, etc.) Experience in building parallelised or distributed model More ❯

technology infrastructure driving the ecosystem that enables the business to make smarter decisions with data. Our primary technology is Google Cloud Platform supported by Airflow and dbt. As technical lead on the team you'll be working closely with architects, product managers and stakeholders, helping to provide technical knowledge … ecosystem, including maintaining a lakehouse on Google Cloud Platform with BigQuery at its core Deliver operations and orchestration for all things data, specifically on Airflow, Cloud Composer Support integrations with key platforms such as Fivertan, Tableau, and Alation Enhance our architecture, align it with our infosec, multi-cloud, and … BigQuery preferred. Design and deliver solutions on the technologies that drive data at the Guardian. This looks like: Google Cloud Platform: BigQuery, Kubernetes Python, Airflow, & DBT CI/CD & Terraform Nice-to-have : Spark & Scala on DataProc Demonstrated knowledge and experience of key operational concerns, including: Security & Networks Site More ❯

Title: Data Scientist Salary: £60,000-£67,500 + Benefits Location: West London - Hybrid (3 days p/w in-office) Tech: AWS, Snowflake, Airflow, DBT The Company: Immersum are supporting the growth leading PropTech company on a mission to revolutionise how the property sector understands people, places, and … ML models Strong analytical background with applied stats, EDA, and model validation techniques Confidence working with structured data pipelines and modern tooling (AWS, Snowflake, Airflow, DBT) Curiosity for emerging techniques and an eagerness to learn and innovate Excellent communication skills, especially when simplifying complex findings for non-technical teams More ❯

the setup; Sharing, enhancing and upskilling team members on available tools and best practices. What you'll need Strong aptitude with SQL, Python and Airflow; Experience in Kubernetes, Docker, Django, Spark and related monitoring tools for DevOps a big plus (e.g. Grafana, Prometheus); Experience with dbt for pipeline modeling … based pipelines built with dbt on Databricks Analysis via Python Jupyter notebooks Pyspark in Databricks workflows for heavy lifting Streamlit and Python for dashboarding Airflow DAGs with Python for ETL running on Kubernetes and Docker Django for custom app/database development Kubernetes for container management, with Grafana/ More ❯

/Tools we use: Python, Azure (Virtual Machines, Azure Web Apps, Cloud Storage, Azure ML), Anaconda packages, Git, GitHub, GitHub Actions, Terraform, SQL, Artifactory, Airflow, Docker, Kubernetes, Linux/Windows VMs. About You: Hands-on industry experience in some combination of Software Engineering, ML Engineering, Data Science, DevOps, and … necessary. Hands-on industry experience in some combination of the following technologies: Python ecosystem, Azure (VMs, Web Apps, Managed Databases), GitHub Actions, Terraform, Packer, Airflow, Docker, Kubernetes, Linux/Windows VM administration, Shell scripting (primary Bash but PowerShell as well). A solid understanding of modern security and networking More ❯

/Tools we use: Python, Azure (Virtual Machines, Azure Web Apps, Cloud Storage, Azure ML), Anaconda packages, Git, GitHub, GitHub Actions, Terraform, SQL, Artifactory, Airflow, Docker, Kubernetes, Linux/Windows VMs. About You Hands-on industry experience in some combination of Software Engineering, ML Engineering, Data Science, DevOps, and … necessary. Hands-on industry experience in some combination of the following technologies: Python ecosystem, Azure (VMs, Web Apps, Managed Databases), GitHub Actions, Terraform, Packer, Airflow, Docker, Kubernetes, Linux/Windows VM administration, Shell scripting (primary Bash but PowerShell as well). A solid understanding of modern security and networking More ❯

that support core trading, forecasting, risk, and PPA processes across all Octopus international regions. The role leverages a modern tech stack including SQL, Python, Airflow, Kubernetes, and various other cutting-edge technologies. You'll work with tools like dbt on Databricks, PySpark, Streamlit, and Django, ensuring robust data infrastructure … code standardization Take ownership of data platform improvements Share knowledge and upskill team members Requirements For Data Engineer Strong aptitude with SQL, Python and Airflow Experience in Kubernetes, Docker, Django, Spark and related monitoring tools Experience with dbt for pipeline modelling Ability to shape needs into requirements and design More ❯

trade-offs, collaborative design, and proactive roadmap planning, with an openness to exploring innovative solutions like generative AI where appropriate. We work with an Airflow/AWS/Fivetran/Snowflake/Looker stack and typically use Python and Docker in our pipelines. You’ll need to be highly … data lifecycle and ELT patterns Comfortable evaluating both business and technical requirements Skilled at working with large datasets and optimising data flows Experience with Airflow, AWS, Fivetran, Snowflake, Docker (or similar) Strong in Python, SQL, and cloud platforms (AWS or comparable) Experienced in handling real-time data pipelines Experienced More ❯