176 to 200 of 202 Apache Airflow Jobs

automation, reliability, and agility. Key Responsibilities Design, build, and optimise data pipelines across a modern data platform. Ingest, clean, and transform data using tools such as dbt, Snowflake, and Airflow . Collaborate with cross-functional teams to deliver data products aligned to business priorities. Develop scalable data models that support BI and analytics platforms including Tableau and Power BI. … and optimise complex queries. Hands-on experience with dbt (including testing and layered modelling). Practical knowledge of Snowflake for loading, transforming, and exporting datasets. Experience building and managing Airflow DAGs for pipeline orchestration. Understanding of BI tool requirements (e.g., Tableau, Power BI) and related performance considerations. Advanced Excel capability, including pivot tables and complex formulas. Familiarity with data More ❯

you'll be doing at BRITA You will play an active role in setting up and operating our new Data Analytics Platform, using modern technologies such as Databricks, dbt, Airflow, Terraform and Azure Data Services. At the same time, you will manage our existing BI landscape and help shape the transition to the new architecture. Your responsibilities will include … from us A hybrid data landscape with proven on-premise technologies (SQL Server, SSIS, SSAS, Power BI) and a modern Data Analytics Platform with technologies such as Databricks, dbt, Airflow, Azure Data Services, Terraform, GitHub, Python, and Fabric currently under development. A motivated team with principles such as trust, personal responsibility, transparency and continuous development. A structured operating process More ❯

automated pipelines, and shaping the foundational framework for how we leverage data to succeed. What You'll Do You'll develop and maintain data pipelines and automated processes in Airflow and Python You'll create SQL data models with dbt to power dashboards and applications You'll integrate third-party APIs and databases into our data flows You'll … notebook analytics and collaboration Circle CI for continuous deployment AWS cloud infrastructure Kubernetes for data services and task orchestration Google Analytics, Amplitude and Firebase for client applications event processing Airflow for job scheduling and tracking Parquet and Delta file formats on S3 for data lake storage Streamlit for data applications Why else you'll love it here Wondering what More ❯

term Interview Criteria: Telephonic + Zoom Direct Client Requirement Role: AWS Data Engineer We are seeking a skilled AWS Data Engineer who has experience working with Python, PySpark, lambda, Airflow, and Snowflake.Responsibilities: Design, build, and optimize ETLs using Python, PySpark, lambda, Airflow and other AWS services. Create SQL queries to segment, manipulate, and formatdata. Build automations to ingest … ingest data into Amazon Redshift for analytics and reporting. Requirements: Minimum 5 years of experience as Data Engineer. 3+ years of Python, PySpark, and Lambda. Must have experience with Airflow and Snowflake. Advanced SQL query development proficiency Understanding of data modelling principles and techniques. Knowledge of data security best practices and compliance requirements. Note: If you are interested, please More ❯

colleagues. Nice to Have Skills Experience working with GCP (BigQuery) or other modern cloud-native data warehouses (e.g. Snowflake, Redshift). Familiarity with data pipelining and orchestration systems (e.g. Airflow). Understanding of modern analytics architectures and data visualisation tools (we use Preset.io/Apache Superset). Exposure to CI/CD pipelines (GitLab CI preferred). Experience More ❯

colleagues. Nice to Have Skills Experience working with GCP (BigQuery) or other modern cloud-native data warehouses (e.g. Snowflake, Redshift). Familiarity with data pipelining and orchestration systems (e.g. Airflow). Understanding of modern analytics architectures and data visualisation tools (we use Preset.io/Apache Superset). Exposure to CI/CD pipelines (GitLab CI preferred). Experience More ❯

colleagues. Nice to Have Skills Experience working with GCP (BigQuery) or other modern cloud-native data warehouses (e.g. Snowflake, Redshift). Familiarity with data pipelining and orchestration systems (e.g. Airflow). Understanding of modern analytics architectures and data visualisation tools (we use Preset.io/Apache Superset). Exposure to CI/CD pipelines (GitLab CI preferred). Experience More ❯

colleagues. Nice to Have Skills Experience working with GCP (BigQuery) or other modern cloud-native data warehouses (e.g. Snowflake, Redshift). Familiarity with data pipelining and orchestration systems (e.g. Airflow). Understanding of modern analytics architectures and data visualisation tools (we use Preset.io/Apache Superset). Exposure to CI/CD pipelines (GitLab CI preferred). Experience More ❯

colleagues. Nice to Have Skills Experience working with GCP (BigQuery) or other modern cloud-native data warehouses (e.g. Snowflake, Redshift). Familiarity with data pipelining and orchestration systems (e.g. Airflow). Understanding of modern analytics architectures and data visualisation tools (we use Preset.io/Apache Superset). Exposure to CI/CD pipelines (GitLab CI preferred). Experience More ❯

you'll be doing at BRITA You will play an active role in setting up and operating our new Data Analytics Platform, using modern technologies such as Databricks, dbt, Airflow, Terraform and Azure Data Services. At the same time, you will manage our existing BI landscape and help shape the transition to the new architecture. Your responsibilities will include … from us A hybrid data landscape with proven on-premise technologies (SQL Server, SSIS, SSAS, Power BI) and a modern Data Analytics Platform with technologies such as Databricks, dbt, Airflow, Azure Data Services, Terraform, GitHub, Python, and Fabric currently under development. A motivated team with principles such as trust, personal responsibility, transparency and continuous development. A structured operating process More ❯

you'll be doing at BRITA You will play an active role in setting up and operating our new Data Analytics Platform, using modern technologies such as Databricks, dbt, Airflow, Terraform and Azure Data Services. At the same time, you will manage our existing BI landscape and help shape the transition to the new architecture. Your responsibilities will include … from us A hybrid data landscape with proven on-premise technologies (SQL Server, SSIS, SSAS, Power BI) and a modern Data Analytics Platform with technologies such as Databricks, dbt, Airflow, Azure Data Services, Terraform, GitHub, Python, and Fabric currently under development. A motivated team with principles such as trust, personal responsibility, transparency and continuous development. A structured operating process More ❯

you'll be doing at BRITA You will play an active role in setting up and operating our new Data Analytics Platform, using modern technologies such as Databricks, dbt, Airflow, Terraform and Azure Data Services. At the same time, you will manage our existing BI landscape and help shape the transition to the new architecture. Your responsibilities will include … from us A hybrid data landscape with proven on-premise technologies (SQL Server, SSIS, SSAS, Power BI) and a modern Data Analytics Platform with technologies such as Databricks, dbt, Airflow, Azure Data Services, Terraform, GitHub, Python, and Fabric currently under development. A motivated team with principles such as trust, personal responsibility, transparency and continuous development. A structured operating process More ❯

and governance through robust access controls, including RBAC, SSO, token policies, and pseudonymisation frameworks. Develop resilient data flows for both batch and streaming workloads using technologies such as Kafka, Airflow, DBT, and Terraform. Shape data strategy and standards by contributing to architectural decisions, authoring ADRs, and participating in reviews, data councils, and platform enablement initiatives. Qualifications What we’d … requirements Direct exposure to cloud-native data infrastructures (Databricks, Snowflake) especially in AWS environments is a plus Experience in building and maintaining batch and streaming data pipelines using Kafka, Airflow, or Spark Familiarity with governance frameworks, access controls (RBAC), and implementation of pseudonymisation and retention policies Exposure to enabling GenAI and ML workloads by preparing model-ready and vector More ❯

Join our rapidly expanding team as a hands-on Cloud Data Analytics Platform Engineer and play a pivotal role in shaping the future of data at Citi. We're building a cutting-edge, multi-cloud data analytics platform that empowers More ❯

Azure/Microsoft Fabric/Data Factory) and modern data warehouse technologies (Databricks, Snowflake) Experience with database technologies such as RDBMS (SQL Server, Oracle) or NoSQL (MongoDB) Knowledge in Apache technologies such as Spark, Kafka and Airflow to build scalable and efficient data pipelines Ability to design, build, and deploy data solutions that explore, capture, transform, and utilize More ❯

points and decision engines (e.g., XACML, OPA). • Familiarity with secure data transfer protocols, cross-domain solutions (CDS), and data protection. • Proficient with data management tools and languages (e.g., Apache NiFi, Kafka, Airflow, Python, Spark). • Excellent problem-solving skills and the ability to collaborate with cross-functional teams. More ❯

Job Title: Data Engineer Salary: £40k - £60k + Excellent Benefits Package Location: Huddersfield (Hybrid) Key Skills: SQL, PowerBI, Airflow Summary A new role has opened up for a Data Engineering with SQL, BI, Cloud and Airflow DAG to join a media-focused business. The role has opened up as the company are heavily investing, and have exciting plans … responsibilities will cover: Develop data models to support company Business Intelligence Write and optimize complex SQL queries Build, maintain and improve data pipelines Transform data using DBT, Snowflake and Airflow Ensure data is handled correctly and to relevant standards Collaborate with tech teams to collectively solve shared data challenges Key Skills SQL DBT Snowflake Airflow DAG Cloud Platform … successful candidate to Oscar. Email: to recommend someone for this role Job Title: Data Engineer Salary: £40k - £60k + Excellent Benefits Package Location: Huddersfield (Hybrid) Key Skills: SQL, PowerBI, Airflow Oscar Associates (UK) Limited is acting as an Employment Agency in relation to this vacancy. To understand more about what we do with your data please review our privacy More ❯

core of VAULT is big data at scale. Our systems handle massive ingestion pipelines, long-term storage, and high-performance querying. We leverage distributed technologies (Kafka, Spark, Flink, Cassandra, Airflow, etc.) to deliver resilient, low-latency access to trillions of records, while continuously optimizing for scalability, efficiency, and reliability. We'll trust you to: Build high-performance, distributed data … pipelines and services Lead technical direction and ensure alignment with business needs Collaborate across teams using modern open-source tech (Kafka, FastAPI, Airflow, Spark, etc.) Mentor junior engineers and contribute to an inclusive team culture You will need to have: Bachelor's in Computer Science, Engineering, or related field (or equivalent experience) 4+ years of professional experience in an … Deep background in distributed, high-volume, high-availability systems Fluency in AI development tools We would love to see: Experience with big data ecosystems (Kafka, Spark, Flink, Cassandra, Redis, Airflow) Familiarity with cloud platforms (AWS, Azure, GCP) and S3-compatible storage SaaS/PaaS development experience Container technologies (Docker, Kubernetes) Salary Range = 160000 - 240000 USD Annually + Benefits + More ❯

methodology and a Medallion architecture (bronze, silver, gold layers). Develop and maintain DBT projects and configure incremental loads with built-in unit testing. Support data pipeline orchestration with Airflow and work with AWS cloud tools. Help deliver a production-ready Data Mart with star schema design to power business reporting and dashboards (PowerBI experience a plus). Skills … Experience: Strong SQL expertise and hands-on experience with DBT. Familiarity with Kimball dimensional modelling concepts. Experience working with cloud data warehouses such as Redshift or Snowflake. Knowledge of Airflow for workflow management. Comfortable in AWS environments and data orchestration. Bonus: Python programming skills and familiarity with dashboarding tools. Contract Details: Duration: 3 months Rate: £450/day onside More ❯

Job Description: Responsibilities Architect, deploy, and maintain scalable data infrastructure in GCP with a focus on Cloud Composer (Airflow) and Google Kubernetes Engine (GKE Manage containerized workloads using Docker and Kubernetes, including lifecycle management, networking, monitoring, and troubleshooting. Collaborate with data engineering teams to migrate traditional workloads to containerized platforms. Implement and maintain CI/CD pipelines using tools … environments. Strong understanding of Kubernetes architecture, including control plane components and networking. Experience with Python (5+ years), especially in the context of automation and data workflows. Experience with Composer (Airflow) DAG development and orchestration. Proficiency in Linux-based infrastructure and scripting languages (Bash, Python, Java and GoLang are a plus Familiarity with container monitoring, logging, and troubleshooting tools. Preferred More ❯

Collaborate with ML researchers and biologists to translate raw data into actionable insights and high-quality training data Scale distributed systems using Kubernetes, Terraform, and orchestration tools such as Airflow, Flyte, or Temporal Write clean, extensible, and well-tested code to ensure long-term maintainability and collaboration across teams About You We are looking for data and platform engineers … Experience designing and implementing large-scale data storage systems (feature stores, timeseries databases, warehouses, or object stores) Strong distributed systems and infrastructure skills (Kubernetes, Terraform, orchestration frameworks such as Airflow/Flyte/Temporal) Hands-on cloud engineering experience (AWS, GCP, or Azure) Strong software engineering fundamentals, with a track record of writing maintainable, testable, and extensible code Familiarity More ❯

Collaborate with ML researchers and biologists to translate raw data into actionable insights and high-quality training data Scale distributed systems using Kubernetes, Terraform, and orchestration tools such as Airflow, Flyte, or Temporal Write clean, extensible, and well-tested code to ensure long-term maintainability and collaboration across teams About You We are looking for data and platform engineers … Experience designing and implementing large-scale data storage systems (feature stores, timeseries databases, warehouses, or object stores) Strong distributed systems and infrastructure skills (Kubernetes, Terraform, orchestration frameworks such as Airflow/Flyte/Temporal) Hands-on cloud engineering experience (AWS, GCP, or Azure) Strong software engineering fundamentals, with a track record of writing maintainable, testable, and extensible code Familiarity More ❯

the development of internal ML platforms, and help identify high-impact opportunities for rapid capability delivery. You'll also be responsible for building robust MLOps pipelines using tools like Airflow and MLflow, ensuring scalable and automated model deployment and governance. What you'll need to succeed Essential Skills: * Bachelor's degree (2:1 or above) in science, engineering, mathematics … or computer science * Strong Python development skills * Practical experience with machine learning or deep learning in scientific/engineering contexts * Familiarity with MLOps tools and practices (Airflow, MLflow) and containerisation (e.g., Docker) * Passion for working with real-world datasets and data visualisation * Interest in material discovery, computer vision, big data, and optimisation techniques * Excellent communication and collaboration skills * Problem More ❯

the development of internal ML platforms, and help identify high-impact opportunities for rapid capability delivery. You'll also be responsible for building robust MLOps pipelines using tools like Airflow and MLflow, ensuring scalable and automated model deployment and governance. What you'll need to succeed Essential Skills: * Bachelor's degree (2:1 or above) in science, engineering, mathematics … or computer science * Strong Python development skills * Practical experience with machine learning or deep learning in scientific/engineering contexts * Familiarity with MLOps tools and practices (Airflow, MLflow) and containerisation (e.g., Docker) * Passion for working with real-world datasets and data visualisation * Interest in material discovery, computer vision, big data, and optimisation techniques * Excellent communication and collaboration skills * Problem More ❯

concepts in simple, actionable terms. Collaborate with the central data engineering team managing the cloud infrastructure (BigQuery, GCP). Coordinate with third party partners responsible for pipeline orchestration and Airflow maintenance. Drive standardisation across naming conventions, definitions, and documentation to strengthen governance and data discoverability. Influence the organisation’s data roadmap, ensuring it aligns with evolving business priorities. Take … cloud data warehouses (ideally BigQuery). Proficiency in modular data modelling frameworks such as Dataform or dbt, with an emphasis on testing and documentation. Experience with orchestration tools (e.g. Airflow) for managing data pipelines. Familiarity with modern data stacks, version control (Git), and CI/CD best practices. Strong ability to validate and interrogate large datasets, with a high More ❯