76 to 100 of 216 Apache Airflow Jobs in England

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

TensorFlow, PyTorch). Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid ...

data warehousing and data modelling principles Experience with Snowflake and modern cloud data platforms Python development skills Experience with workflow orchestration tools such as Airflow Knowledge of streaming technologies and real-time data processing Experience implementing CI/CD and engineering best practices What They Offer Competitive salary ...

solution design responsibility. You'll ideally have: Strong commercial experience with Python & Snowflake Experience mentoring or technically leading other engineers Experience with dbt, Airflow or similar data engineering tooling A good understanding of data modelling and modern data platform architecture Experience working with AWS (ideally) or another major cloud ...

technical leadership and solution design responsibility. You'll ideally have: Strong commercial experience with Python & SnowflakeExperience mentoring or technically leading other engineersExperience with dbt, Airflow or similar data engineering toolingA good understanding of data modelling and modern data platform architectureExperience working with AWS (ideally) or another major cloud platformThe ...

that make vendor data consistent and useful to downstream consumers.- Improve the reliability and observability of existing services and pipelines.- Help migrate and strengthen Airflow-based workflows.- Participate in the shared on-call rota for systems used by the trading teams.- Hold engineering standards across the team: review, testing ...

asynchronous architectures and related messaging technologies (e.g. Kafka, MQ) Experience with cloud platforms (e.g. AWS, GCP, Azure) Experience with orchestration and container technologies (e.g. Airflow, Kubernetes, Docker) Experience with monitoring and alerting tools (e.g. CloudWatch, Prometheus, Grafana, Sentry/OTel ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing.Discover more about our Analytics Community ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

platforms)Experience applying DevSecOps practices within cloud or controlled environmentsExposure to large language models or generative AI systemsExperience with data engineering tools (e.g. Spark, Airflow, Databricks)Experience applying AI and automation to operational or productivity-focused use casesExperience working in secure, restricted, or air-gapped environmentsExperience supporting systems subject ...

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...

preferred knowledge of AI infrastructure. Proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

knowledge of AI infrastructure. Good proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

degree in Computer Science, Mathematics, Engineering, Physics or a related discipline from a Russell Group university. Desirable Experience Spark and/or Databricks. Airflow or similar orchestration tools. Distributed computing and large-scale data processing. Defence or National Security experience. Essential Requirements UK national and eligible to obtain ...

experience in data platform architecture and cloud-based data environments Excellent understanding of modern data stack technologies Expertise with Google Cloud Platform, BigQuery, dbt, Airflow, or similar technologies Experience designing data integration patterns and scalable data solutions Knowledge of data governance, security, privacy, and compliance principles Strong understanding ...

technical leadership and solution design responsibility. You'll ideally have: Strong commercial experience with Python & SnowflakeExperience mentoring or technically leading other engineersExperience with dbt, Airflow or similar data engineering toolingA good understanding of data modelling and modern data platform architectureExperience working with AWS (ideally) or another major cloud platformThe ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

will take ownership of the data platform, optimising it for scalability to ensure successful client onboarding. You'll use modern tools (such as Airflow, Prefect, Dagster or AWS Step Functions) for ETL design and orchestration, work on transformation logic to clean, validate and enrich data (including handling missing values ...

pipelines. Proficiency in SQL, Python, AWS, and other modern data engineering technologies. Experience with version control tools (e.g., Git) and workflow orchestration platforms (e.g., Airflow). Proven ability to test, troubleshoot, and improve data platforms, ensuring high levels of reliability and data quality. Strong interpersonal and communication skills with ...

Master• Data Integration & Ingestion – Skill• Data Manipulation and Processing – Skill• GITHUB, Action, Azure DevOps – Skill• Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest – SkillOptional skills:• Experience in project management, running a scrum team.• Experience working with BPC, Planning.• Exposure to working with external ...