26 to 50 of 56 Apache Airflow Jobs in the North of England

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

degree in Computer Science or equivalent practical experienceExperience developing finance applications is a plusWorking experience with Terraform is a plusExperience creating workflows for Apache Airflow is a plus#LI-JB6What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work ...

data quality, validation, reconciliation, and audit controls across betting, trading, and customer data domains. Develop streaming and micro-batch data solutions using AWS Kinesis, Apache Kafka, or Amazon MSK to support real-time sportsbook and in-play data feeds. Build and maintain data solutions using AWS services including … architectures AWS certifications are a bonus You know your tools Solid experience with Python, SQL, and Spark — and pipeline tools such as dbt or Airflow Comfortable with data quality frameworks and testing — Great Expectations, dbt tests, or custom validation layers Experience with infrastructure-as-code (Terraform ...

degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience. Preferred Skills Experience with data orchestration tools (e.g., Prefect, Airflow, dbt). Familiarity with MLOps tooling such as MLflow, or experience supporting ML workloads in production. Exposure to LLM or agentic application patterns ...

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking … relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

following domains: Computer Vision, Large Language Models (LLMs), Time Series Analysis, Experience with TensorFlow, PyTorch, XGBoost, LightGBM, Proficiency with Git, Linux, Docker, MLflow, Airflow, AWS/GCP/Azure, Solid SQL and data visualization skills. Nice to have Proven experience leading projects and managing teams, Experience in presales, client ...

data engineering. You'll likely have experience with: Python SQL AWS, Azure or GCP Building production data pipelines Data modelling and data platforms Git Airflow or similar orchestration technologies Testing, debugging and improving data reliability We're particularly interested in engineers who have worked in consultancy, professional services ...

practical experience. Preferred Skills Experience with gradient-boosted tree models, neural networks, and optimization solvers in production. Familiarity with data orchestration tools (e.g., Prefect, Airflow, dbt) and modern data lake architectures. Experience working closely with data scientists to operationalize research code. Multi-cloud experience across AWS, GCP, and Azure. ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows.Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...

Bicep) Solid understanding of cloud networking (hub/spoke, private endpoints) Experience troubleshooting infrastructure and deployment issues Strong collaboration and communication skills Experience with Airflow or similar orchestration tools Knowledge of observability tooling (Prometheus, Grafana) Exposure to AI/ML platform environments Qualifications Relevant degree or equivalent industry experience ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing.Discover more about our Analytics Community ...

Cloud Run), AWS (Bedrock, SageMaker)* Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK* Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar.* Ability to design ...

Cloud Run), AWS (Bedrock, SageMaker) Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar. Ability to design ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

tools (Git), Fluent English – for international communication and collaboration. Nice to have: Experience with LLMs, or Time Series, Familiarity with tools like Docker, MLflow, Airflow, AWS/GCP/Azure, Background in research or R&D, Experience working in cross-functional or international teams. Soft Skills: Curiosity and problem ...

Hybrid : London or Manchester - 2 days per week. This role requires the ability to hold an active security clearance. Skills : Python and SQL . Apache Airflow for DAG orchestration and monitoring. AWS data services: Redshift , OpenSearch , Lambda , Glue , Step Functions , Batch . CI/CD pipelines and YAML ...

hybrid role bridging software and data engineering. You will architect and maintain production-grade data products and APIs using Java/Scala, SQL, Spark, Airflow, and Kubernetes to deliver reliable, high-performance solutions. You will collaborate with Finance, Accounting, and Analytics, drive data quality, and contribute to AI-powered ...

Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added advantage. ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled in PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added advantage About ...

stakeholder communication skills Desirable: Experience overseeing or supporting data ingestion pipelines, including APIs and event-driven data sources Familiarity with orchestration tools such as Airflow and modern ELT architectures Experience governing or implementing data CI/CD pipelines (e.g. dbt tests, deployment pipelines, automated checks). We currently … design input, reviews, and mentoring Influence how analytics data is built, trusted, and operated at scale Work with a modern stack: AWS, Snowflake, dbt, Airflow, SQL, and Python Join us as we redefine travel experiences and create memories for millions of passengers. At Jet2.com and Jet2holidays, your potential ...

design, build, and maintain data pipelines and infrastructure to support secure, reliable, and scalable public sector services. You will work with AWS, Terraform, Spark, Airflow, Docker, and GitLab CI/CD within a hybrid model, contributing to secure coding practices and collaborating across cross-functional teams. #J-18808-Ljbffr ...