51 to 75 of 130 Apache Airflow Jobs

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop ...

upholding data quality and governance standards What we're looking for Strong SQL and Python skills, with hands-on experience in dbt, Databricks, Airflow or Looker Solid analytics engineering fundamentals — data modelling, warehousing, performance optimisation, observability and governance Experience in a marketplace or e-commerce environment, with comfort around ...

upholding data quality and governance standards What we're looking for Strong SQL and Python skills, with hands-on experience in dbt, Databricks, Airflow or Looker Solid analytics engineering fundamentals — data modelling, warehousing, performance optimisation, observability and governance Experience in a marketplace or e-commerce environment, with comfort around ...

dimensional modelling Experience working with Snowflake or similar cloud data warehouses such as BigQuery Good Python capability, including orchestration tools such as Airflow Proven experience building and managing data ingestion pipelines Cloud experience within AWS or GCP environments Exposure to infrastructure as code tools such as Terraform Confidence leading ...

dimensional modelling Experience working with Snowflake or similar cloud data warehouses such as BigQuery Good Python capability, including orchestration tools such as Airflow Proven experience building and managing data ingestion pipelines Cloud experience within AWS or GCP environments Exposure to infrastructure as code tools such as Terraform Confidence leading ...

Deep experience with Spark and/or Databricks Strong cloud expertise across AWS, Azure and/or GCP Experience with orchestration technologies such as Airflow A track record of delivering scalable, resilient and high-quality data products The ability to influence customers and senior stakeholders Experience developing engineering capabilities ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...

infrastructure. Analyses and reports on test activities, results, issues and risks. Production data analysis to identify root cause of production environment issues. Using Apache Airflow for task orchestration and reviewing output logs. Basic understanding of EMR and reviewing output logs. Basic understanding of data processing using Apache ...

complete features, though it's not the primary focus of the role. Our Stack (100% Open Source) Backend: Golang, Python Infrastructure & Orchestration: Docker, Kubernetes, Airflow, Salt, Ansible, Chef Data: PostgreSQL, Redis, Kafka API: REST, GraphQL Frontend: React with TypeScript What you’ll do Own backend systems — design, build … networking, filesystem concepts Configuration management experience with Salt, Ansible, Chef, or similar tools Container and orchestration expertise — Docker, Kubernetes, or related technologies Workflow orchestration — Apache Airflow or similar pipeline frameworks Database design skills — schema design and query optimization in PostgreSQL or similar RDBMS Distributed systems knowledge — you think ...

technical trade‐offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with or strong advocacy for AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...

technical trade-offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with (or strong advocacy for) AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...

Experience working within cloud environments (AWS or Azure) Strong SQL skills and experience working with large-scale datasets Familiarity with orchestration tools such as Airflow or similar Good understanding of data modelling and distributed data processing concepts Comfortable working across multiple client environments and stakeholders 🎁 What’s on Offer ...

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

across the data engineering domain. Engage with data engineering technical detail when needed, understanding the data platform landscape including data ingestion, transformation (DBT), orchestration (Airflow), streaming (Kafka), and data warehousing (Snowflake), and able to articulate technical choices and trade‐offs to different stakeholders. Work at both conceptual and detailed … despite complexity in data requirements and priorities. What I need to know: Strong understanding of data engineering technologies including data ingestion, transformation (DBT), orchestration (Airflow/Astronomer). Experience with cloud data platforms (AWS data services, Azure data services) including cost management and optimisation Knowledge of data quality frameworks ...

prompt engineering skills and the ability to critically evaluate AI-generated code for correctness, security, and financial accuracy Desirable, but no essential: Experience with Apache Airflow or similar workflow schedulers REST or GraphQL API design experience Knowledge of fixed income, FX, or derivatives products and associated ...

responsibilities Own, assess, improve, or rebuild existing ETL/ELT pipelines Build reliable data workflows using Python Work with orchestration tools such as Airflow or similar Improve data quality, validation, lineage, and troubleshooting Collaborate with consultants and non-technical stakeholders Help translate messy business/client data into structured ...

data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience with workflow orchestration tools such as Airflow or similar Ability to design scalable architectures, make pragmatic trade-offs, and communicate effectively with both technical and non-technical stakeholders Processing Your Data ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

experience with Databricks, Spark and distributed data processing technologies Cloud expertise across AWS, Azure and/or GCP Experience using orchestration tools such as Airflow A strong understanding of data platform reliability, scalability and governance Proven experience influencing technical strategy and customer decision-making The ability to lead, mentor ...

engineering, model evaluation, and experiment design (A/B testing, holdout strategies, stratification). Experience developing production-quality data pipelines and automated workflows using Airflow or similar orchestration tools. Practical familiarity with graph databases and/or graph frameworks (Neo4j, AWS Neptune, GraphFrames, DGL, PyTorch Geometric) and graph algorithms ...