126 to 147 of 147 Apache Airflow Jobs in the UK excluding London

data platform, blending hands-on delivery with leadership and mentorship responsibilities. The role requires strong GCP experience (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), advanced Python and SQL, and a track record of building scalable batch and real-time pipelines. #J-18808-Ljbffr ...

game production cycle or long-term project. A strong understanding of Photogrammetry workflows and software. Proficiency in Python. Experience with build systems like Jenkins, Airflow, or Celery. An understanding of a 3D asset pipeline and color management. Ability to think creatively and develop innovative solutions to complex problems. PLUSESPlease ...

Description Job Title Technical Lead Multi-tenant Data Integration Platform (GCP/Airflow) Role purpose Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables ...

/ML, or data engineeringExperience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with emerging technologies in data engineeringAbility ...

/ML, data engineering)Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governancePractical cloud-native experience (AWS, Azure … SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with AI evaluation and observability practices: evals ...

help strengthen pre and post-change validation, enhance observability, and support the continuous improvement of automation capabilities. Technology Stack Linux/UNIX Python Ansible Apache Airflow Prometheus Grafana Loki VMware F5 What We're Looking For We're looking for engineers with: Strong Linux/UNIX administration fundamentals ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment ...

deployments Troubleshoot integration failures, data inconsistencies, and production issues Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

Lead Data Engineer Location: West Midlands | Hybrid 3 days per week onsite The Role Were looking for an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a ...

The Role We’re looking for an experienced Lead Data Engineer to provide hands‐on technical leadership within a data engineering team for a large retail brand. You’ll lead the design and delivery of ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment/config: Ansible (physical), Terraform/Argo CD/Helm (Kubernetes) Development environment: MacBook … Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code Degree in Computer Science or numerical degree Competitive salary ...

Designing and implementing data classification and governance-by-design Maintaining and improving a GCP/BigQuery lakehouse environment Building and supporting pipelines using Python, Airflow/Cloud Composer and dbt Identifying security vulnerabilities and supporting remediation Implementing robust CI/CD, testing and Terraform-based infrastructure Working across integrations … Deep knowledge of RBAC/IAM within cloud data environments Strong understanding of data security, governance and access controls Hands-on experience with Python, Airflow and dbt Experience with Terraform and CI/CD Knowledge of Kubernetes Exposure to AWS alongside GCP Strong stakeholder management and the ability ...

migrate data sets into their AWS Snowflake platform. Designing and implementing dbt models that power clean, consistent metrics. Buidl out orchestartion pipelines using Airflow Creating clear documentation for dashboards, metrics, and models. Ensuring optimal performance and cost efficiency across Snowflake Supporting a stakeholder-first approach to data usability … access. KEY SKILLS AND REQUIREMENTS Extensive experience operating within Snowflake environments Airflow for data orchestration. Proficiency in SQL and dbt, with a strong understanding of data modeling best practices. Strong communication skills with the ability to translate complex needs into elegant data solutions. Focus on usability, documentation, and data ...

within the new architecture. You'll work across: Migrating legacy pipelines and workloads into Snowflake Building and refactoring pipelines using Python Developing orchestration through Airflow/MWAA Infrastructure and deployment using Terraform and Docker Supporting transformation and modelling using dbt Working across both legacy and modern AWS data environments … looking for a broad, hands-on Data Engineer who is comfortable working across data engineering, software engineering and infrastructure. Core experience: Python Airflow/MWAA Snowflake Terraform AWS Desirable experience: dbt Docker Redshift R Data platform migrations or modernisation programmes The ideal candidate will have worked within complex ...

Inside IR35 Location: London - Hybrid, 3 days per week onsite Duration: 6 months The role will focus on GCP, BigQuery, Python, Airflow/Cloud Composer, dbt, Terraform and CI/CD, working closely with architects, engineers, product and governance teams to deliver secure, scalable data solutions. Key experience … BigQuery Python, Airflow & dbt Terraform & CI/CD Data platforms/lakehouse environments Kubernetes AWS alongside GCP Strong stakeholder management If this sounds relevant to you, please drop me a message or send your CV Thanks, Callum ...

United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. You will contribute to data ...

Consulting Global is seeking a Technical Lead to steer the development of a scalable, multi-tenant data integration platform on Google Cloud Platform using Apache Airflow. You will shape the technical direction, drive reliable ETLs, and raise the bar on automation, governance and performance across environments. You will mentor ...