10 of 10 Apache Airflow Jobs in West Yorkshire

outside of the city or country. We are seeking a highly skilled Data Engineer with expertise in managing, designing, and optimizing data pipelines utilizing Apache Airflow, Snowflake, and Apache Kafka. This individual will play a pivotal role in architecting robust, scalable, and efficient data solutions, ensuring … infrastructure Responsibilities: Develop and implement data models to support business requirements, optimizing for performance and scalability; Design, build, and maintain scalable data pipelines using Apache Airflow; Implement and maintain Kafka-based streaming data pipelines for real-time data processing and integration with various systems; Integration to third party ...

your specific data industry experience to keep our curriculum cutting-edge. This involves iterating on materials to reflect modern stacks (e.g. cloud data engineering, Apache Airflow, Kafka). You will collaborate with the Curriculum team to design highly effective educational experiences that give learners the AI and Data … data modelling, and managing data warehouses (experience with DBT is a plus). Pipelines & Orchestration: Familiarity with building data pipelines, covering batch processing (e.g., Apache Airflow) and an understanding of stream processing (e.g., Kafka/Spark). Data Statistics & Analytics: A strong grasp of statistical concepts (distributions, regression ...

modelling techniques. Experience designing data marts and semantic layers for BI tools (Power BI, Tableau, Looker). Familiarity with analytics engineering tools including dbt, Airflow, and Git for version control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ ...

Engineering, Data Quality, or Data Governance roles with production systems. Strong SQL and hands-on experience with one or more: Spark/Databricks, dbt, Airflow/Databricks Workflows, Kafka/Kinesis, Python. Experience implementing data quality frameworks and observability (e.g., Great Expectations, Soda, Monte Carlo, Databricks DQ, BigQuery ...

Engineering, Data Quality, or Data Governance roles with production systems. Strong SQL and hands-on experience with one or more: Spark/Databricks, dbt, Airflow/Databricks Workflows, Kafka/Kinesis, Python. Experience implementing data quality frameworks and observability (e.g., Great Expectations, Soda, Monte Carlo, Databricks DQ, BigQuery ...

Engineering, Data Quality, or Data Governance roles with production systems. Strong SQL and hands-on experience with one or more: Spark/Databricks, dbt, Airflow/Databricks Workflows, Kafka/Kinesis, Python. Experience implementing data quality frameworks and observability (e.g., Great Expectations, Soda, Monte Carlo, Databricks DQ, BigQuery ...

professional experience in Data Platform. Proficiency in SQL and programming languages like Python Working knowledge of ETL and ELT frameworks and orchestration. Tools like Airflow and dbt Experience working with cloud platforms e.g. Azure Knowledge of data lakes and data warehouses concepts and distributed systems like Kafka, Spark etc. ...

professional experience in Data Platform. Proficiency in SQL and programming languages like Python Working knowledge of ETL and ELT frameworks and orchestration. Tools like Airflow and dbt Experience working with cloud platforms e.g. Azure Knowledge of data lakes and data warehouses concepts and distributed systems like Kafka, Spark etc. ...

workflows Nice to Have Knowledge of FinOps principles and cloud cost optimisation Experience with orchestration tools such as Azure Data Factory, Databricks Workflows, or Airflow Exposure to containerisation tools (Kubernetes, Docker) Experience with data cataloguing tools Familiarity with unit testing in data pipelines Awareness of MLOps practices ...

MLflow to manage the end-to-end ML lifecycle, including experiment tracking, model registry, and deployment. Automate and orchestrate: Use orchestration tools like Airflow to manage complex ML workflows and ensure seamless execution at scale. Optimize infrastructure: Design efficient ML pipelines and leverage cloud services like AWS to ensure … Enterprise RAG Systems, including chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision ...