26 to 50 of 51 Remote/Hybrid Apache Airflow Jobs

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

responsibilities Own, assess, improve, or rebuild existing ETL/ELT pipelines Build reliable data workflows using Python Work with orchestration tools such as Airflow or similar Improve data quality, validation, lineage, and troubleshooting Collaborate with consultants and non-technical stakeholders Help translate messy business/client data into structured ...

data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience with workflow orchestration tools such as Airflow or similar Ability to design scalable architectures, make pragmatic trade-offs, and communicate effectively with both technical and non-technical stakeholders Processing Your Data ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

skills ✔ Experience working with cloud platforms (AWS preferred) ✔ Knowledge of data modelling, data architecture and governance ✔ Experience with orchestration tools such as Airflow or Databricks Workflows ✔ Comfortable working closely with stakeholders, working independantly delivering on high profile projects and mentoring team members ✔ Experience delivering complex data solutions from scratch ...

skills ✔ Experience working with cloud platforms (AWS preferred) ✔ Knowledge of data modelling, data architecture and governance ✔ Experience with orchestration tools such as Airflow or Databricks Workflows ✔ Comfortable working closely with stakeholders, working independantly delivering on high profile projects and mentoring team members ✔ Experience delivering complex data solutions from scratch ...

Job Title : Analytical Engineer Target Start Date: ASAP Contract Type: Permanent Salary Range: £55,000 - £70,000 Location: Reigate - Hybrid Closing Date for applications: 8th June Analytical Engineer: We are looking for a talented Analytical ...

creating. You'll have experience with: AWS and cloud-native technologies Snowflake or similar cloud data platforms Data orchestration tools such as Airflow, Prefect or similar Amazon S3 and modern data architectures Postgres and semi-structured data models Event-driven architectures and scalable systems You'll understand how data ...

Robert Half Technology are assisting a rapidly growing sports fashion brand to recruit a Data Engineer on a contract basis. This is an exciting opportunity to join a fast-paced SME environment at a pivotal ...

language used for automation (e.g. Python, JavaScript, TypeScript). · Experience integrating automated tests into CI/CD pipelines, or using orchestration platforms such as Apache Airflow · Practical experience working with real and/or virtual devices (Android, iOS, emulators/simulators) and dealing with the challenges of scale ...

language used for automation (e.g. Python, JavaScript, TypeScript). · Experience integrating automated tests into CI/CD pipelines, or using orchestration platforms such as Apache Airflow · Practical experience working with real and/or virtual devices (Android, iOS, emulators/simulators) and dealing with the challenges of scale ...

Lead Data Engineer (Databricks & Cloud Migration) £650 per day (Inside IR35) Initial 6-Month Contract London – Hybrid Working (3 Days per Week Onsite) We are partnering with a leading international bank on a major Databricks ...

checks. Operational or production support mindset; experience in stabilizing live environments. Hands-on use of scheduling or workflow tools such as GitLab CI, Jenkins, Airflow, or Control M (or similar). Strong proficiency working with Linux platforms. Experience managing SSH servers. Ability to write and maintain Ansible scripts ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark – especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects … familiar with Airflow or other workflow orchestration tools, and worked with Kubernetes Understand data lake systems and file formats like Parquet, Orc, Athena Have some relevant AWS or Kafka certifications A vibrant, diverse company pushing ourselves and the technology to deliver beyond the cutting edge A team of motivated ...

Manchester, UK, to develop scalable APIs and manage big data solutions. Candidates should have 8+ years in software or data engineering, with skills in Apache Spark, Airflow, and a strong foundation in Java, Python, or Scala. The role requires collaboration with cross-functional teams to deliver high-quality ...

framework on top of our data platform. Implement MCP- and A2A-based architectures for agent collaboration and orchestration. Integrate AI services with data pipelines (Airflow, Spark, Trino, StarRocks, DBT). Build APIs, SDKs, and tooling to enable other teams to use the framework. Work with platform and application teams … Have) Strong programming skills in Python with 5+ years of experience Proven experience with data engineering or distributed systems (Spark, Hive, Trino, StarRocks, DBT, Airflow, Kubernetes). Deep understanding of LLM-based systems and agent frameworks (LangChain, LlamaIndex, DSPy, OpenAI MCP, A2A). Hands-on experience with API design ...

Senior Data Engineer | Python | Airbyte | Apache | BigQuery | EducationTech | Remote, London | £70k Looking to help shape Data Strategy for a scaling Tech-for-Good business I'm partnered with a AI-first data and analytics platform built for schools, multi-academy trusts and education groups. Their technology connects fragmented school … Engineer to build and maintain the data infrastructure that underpins the entire platform.You’ll own and extend the ELT pipeline - Airbyte (OSS) for extraction, Airflow on Astro for orchestration, dbt via Cosmos for transformation - and the BigQuery lakehouse beneath it, with a semantic layer on top.These are hands ...

distributed processing frameworks such as Spark or Databricks Build and optimise data workflows across cloud platforms (AWS, Azure, GCP) Implement orchestration frameworks such as Airflow Test, troubleshoot, and improve data products to ensure reliability and quality Collaborate with engineering teams to deliver robust, maintainable systems Key Requirements: Experience building … Experience working with at least some of: Distributed computing frameworks (e.g. Spark, Databricks) Cloud platforms (AWS, Azure, or GCP) Workflow orchestration tools (e.g. Airflow) Strong understanding of data system reliability, testing, and performance Experience troubleshooting and improving production data systems Strong problem-solving skills and attention to detail Ability ...

distributed processing frameworks such as Spark or Databricks Build and optimise data workflows across cloud platforms (AWS, Azure, GCP) Implement orchestration frameworks such as Airflow Test, troubleshoot, and improve data products to ensure reliability and quality Collaborate with engineering teams to deliver robust, maintainable systems Key Requirements: Experience building … Experience working with at least some of: Distributed computing frameworks (e.g. Spark, Databricks) Cloud platforms (AWS, Azure, or GCP) Workflow orchestration tools (e.g. Airflow) Strong understanding of data system reliability, testing, and performance Experience troubleshooting and improving production data systems Strong problem-solving skills and attention to detail Ability ...