76 to 93 of 93 Apache Airflow Jobs in England

Linux environment. - Experience in building automation tools and managing system configurations. - Knowledge of C++, KDB/q, and experience with technologies like Slurm, Airflow, Kafka, or AMPS. - Background in enhancing system stability, scalability, and performance while conducting root cause analyses to resolve incidents efficiently. - Observability skillset, monitoring and analysis ...

AI. Comfortable working in a fast-paced, agile environment. Excellent communication and collaboration skills. Nice to Have Experience with workflow orchestration tools (e.g., Temporal, Airflow). Familiarity with vector databases or semantic search tools. Experience building internal tooling or developer productivity enhancements using AI. Exposure to customer support ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 3 10 years experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

e.g. AWS, GCP, Azure), infrastructure-as-code, and scalable architecture patterns Hands-on experience with SQL, Python, and modern data integration frameworks such as Apache Kafka, airflow or dbt Proven ability to deliver enterprise-grade customer data solutions, including customer data platforms and or API based integration Excellent ...

scheduling using workflow tools Build and deploy containerised and serverless data solutions Apply software engineering fundamentals to data engineering challenges Technology Stack Snowflake dbt Airflow SQL (strong focus) Cloud platforms (AWS, Azure, or GCP) Containers and Function Apps SQLMesh (or similar data transformation frameworks) Requirements 2–3 years’ experience … containers and serverless/function-based architectures Good communication skills and a collaborative mindset Nice to Have Experience with Snowflake in production Exposure to Airflow orchestration Understanding of data versioning and testing frameworks Experience working in agile or product-led teams ...

scheduling using workflow tools Build and deploy containerised and serverless data solutions Apply software engineering fundamentals to data engineering challenges Technology Stack Snowflake dbt Airflow SQL (strong focus) Cloud platforms (AWS, Azure, or GCP) Containers and Function Apps SQLMesh (or similar data transformation frameworks) Requirements 2-3 years' experience … containers and serverless/function-based architectures Good communication skills and a collaborative mindset Nice to Have Experience with Snowflake in production Exposure to Airflow orchestration ...

Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations, and high-quality data validation Translate business needs into technical solutions with full ownership What You Need … engineering/analytics/software Strong Python + SQL 2+ years ETL, APIs, CI/CD, Docker, AWS Ideally but not essential experience with Airflow/Prefect/Dagster Background in Financial Services, FinTech, Insurance, PE/VC, or Banking Package & Benefits High performance bonus 25 days holiday Pension ...

models for ML, insights, and search pipelines Design production-ready data pipelines delivering high-quality datasets to partner teams Work with Databricks, AWS, dbt, Airflow, and Looker to implement robust solutions Lead technically complex initiatives and contribute to the team's roadmap Improve engineering efficiency through internal tooling … Requirements Proven track record delivering production-ready data engineering projects end-to-end Strong SQL and Python skills, with experience on platforms like Databricks, Airflow, dbt, and Looker Experience in data modelling, data warehousing, and performance/cost optimisation Skilled in stakeholder management, documentation, and project planning Experience developing ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue, Cloudwatch, etc. ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue, Cloudwatch, etc. ...

focused dashboards that power decision-making across the organisation Driving the build-out of automated workflows and data pipelines using cloud tooling (e.g. Databricks, Airflow) and the Microsoft Power Platform Shaping the direction of analytics architecture and ensuring data products align with broader Group standards Empowering store teams with … Confident SQL ability Experience working with cloud analytics platforms such as Databricks, Snowflake, or BigQuery Background delivering automated workflows or pipelines using tools like Airflow or Databricks Previous experience leading teams within agile frameworks, managing priorities, and working closely with stakeholders Strong understanding of data governance, version control ...

focused dashboards that power decision-making across the organisation Driving the build-out of automated workflows and data pipelines using cloud tooling (e.g. Databricks, Airflow) and the Microsoft Power Platform Shaping the direction of analytics architecture and ensuring data products align with broader Group standards Empowering store teams with … Confident SQL ability Experience working with cloud analytics platforms such as Databricks, Snowflake, or BigQuery Background delivering automated workflows or pipelines using tools like Airflow or Databricks Previous experience leading teams within agile frameworks, managing priorities, and working closely with stakeholders Strong understanding of data governance, version control ...

Title: Lead Data Engineer (Edinburgh)Location: Edinburgh, Midlothian, Scotland, UKCompany: Cathcart TechnologyI'm working with a world-class technology company in Edinburgh to help them find a Lead Data Engineer to join their team (hybrid ...

Akkodis is a global leader in engineering, technology, and R&D, harnessing the power of connected data to drive digital transformation and innovation for a smarter, more sustainable future. As part of the Adecco Group ...

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning ...

scale. In this role, you will: Build robust ML and MLOps pipelines for scalable, reproducible model development, deployment, and monitoring. Use tools such as Airflow for workflow orchestration and MLflow for experiment tracking, model registry, and lifecycle management. Work within an agile development environment and help prioritise high-value … development skills Hands-on experience developing ML and/or deep learning models for scientific or engineering problems Experience with MLOps tools such as Airflow, MLflow, and containerisation (e.g., Docker) Strong data-visualisation and storytelling skills Interest in materials discovery, computer vision, big data, or optimisation Collaborative communicator, organised ...

tools and frameworks for data governance, privacy, and quality monitoring, ensuring full compliance with data protection standards. Create resilient data workflows and automation within Airflow, Databricks, and other modern big data ecosystems. Implement and manage data observability and cataloguing tools (e.g., Monte Carlo, Atlan, DataHub) to enhance visibility … data platforms and backend systems. Familiarity with data governance frameworks, privacy compliance, and automated data quality checks. Hands-on experience with big data tools (Airflow, Databricks) and data observability platforms. Collaborative mindset and experience working with cross-functional teams including ML and analytics specialists. Curiosity and enthusiasm for continuous ...

seeking a skilled Informatica MDM/Customer 360 (C360) Engineer with experience in AWS , Unix/Linux , and modern job orchestration tools (Airflow preferred) . The ideal candidate will design, build, enhance, and support our enterprise master data solutions in a cloud-driven, distributed environment. Responsibilities Informatica MDM/… maintain Unix Shell Scripts for automation, job execution, and MDM environment maintenance. Troubleshoot production issues across Unix, network, and application layers. Job Orchestration (Airflow or equivalent) Design, manage, and monitor Airflow DAGs for MDM batch and workflow automation. Integrate MDM jobs (batch, cleanse, load, match/merge) into ...