76 to 100 of 281 Apache Airflow Jobs in England

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks* Ability to think strategically and relate architectural decisions ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing Vector Databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

Backstage experience - software catalog configuration, scaffolder templates, or plugin development* Familiarity with data platform concepts - medallion architecture, lakehouse patterns, or orchestration tools such as Airflow or dbt* Exposure managing Databricks* Practical experience with AI coding assistants (Claude Code, GitHub Copilot, Cursor, or similar) in a professional engineering context* Experience ...

Data Engineer within GCP environments Hands-on experience with BigQuery, Dataflow and Spanner Strong programming capability in Python, Java or Scala Experience with Apache Beam/Dataflow and distributed processing frameworks Experience designing and managing ETL/ELT pipelines Solid understanding of data modelling and database design Experience with … workflow orchestration tools such as Apache Airflow Strong understanding of data governance, security and scalability Desirable Skills for the GCP Data Engineer: Experience with Pub/Sub, Cloud Composer or Cloud Data Fusion Exposure to CI/CD, DevOps and Infrastructure as Code Experience with real-time streaming ...

find to improve model and system performance. Build reliable ML infrastructure Develop and maintain data and machine learning workflows using technologies such as Apache Airflow. Help improve CI/CD, automated testing, monitoring and reproducibility across our ML stack. Engineer on AWS Build and improve cloud infrastructure using services … expect you to know everything: PyTorch, TensorFlow or JAX Hugging Face LLM APIs and open-source models RAG and vector databases Apache Airflow AWS S3 ECS/EKS Redshift Docker Kubernetes GitHub Actions Pulumi or Terraform SQL Model monitoring and observability Distributed processing Financial markets, economics or commodities ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

PostgreSQL migration projects. Knowledge of modern analytics platforms such as Databricks, Snowflake, Synapse, Fabric, or Redshift. Familiarity with ETL/ELT tooling including dbt, Airflow, Informatica, Talend, SSIS, or similar technologies. Contract Details Position: Data Architect Engagement: 3-6 Month Contract Day Rate: £400 per day IR35: Outside IR35 ...

debugging, the advanced stuff that separates architects from diagram-drawers. Lakehouse fluency — Medallion architecture, Delta Lake, Unity Catalog, governance, orchestration (Workflows, DLT/Lakeflow, Airflow, ADF); Microsoft Fabric exposure a bonus. Battle-tested in migrations — you've led large transformation programs and have the scars and success stories ...

ontologies, and versioned/temporal data. Understanding of how data structures feed RAG and LLM retrieval quality. SQL proficiency; experience with orchestration tools (e.g., Airflow) and cloud data services. Familiarity with data governance, security, and privacy practices. Strong documentation and cross-team communication skills. WHO YOU ARE You think ...

taxonomies/ontologies, and versioned/temporal data.Understanding of how data structures feed RAG and LLM retrieval quality.SQL proficiency; experience with orchestration tools (e.g., Airflow) and cloud data services.Familiarity with data governance, security, and privacy practices.Strong documentation and cross-team communication skills.WHO YOU AREYou think about how to scale ...

technologies and practices such as Docker, Terraform, CI/CD pipelines and infrastructure‐as‐code. Familiarity with MLOps tools and practices such as MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management. Strong Python experience, with helpful knowledge of Spark or PySpark ...

analytics or data platforms, or significantly evolving existing ones.Experience with data quality and observability, using dbt or complementary tooling.Familiarity with orchestration tools such as Airflow, Dagster, or Prefect.Experience with semantic layers, metrics frameworks, or analytics consumption patterns.Exposure to data governance, access control, or analytics enablement at scale.Experience mentoring ...

metrics layers. Experience applying AI to Analytics Engineering workflows or developing AI-ready data products. Knowledge of orchestration and ingestion tools such as Airflow,DagsterorFivetran. Python experience for automation or data engineering tasks. Experience working in regulated financial services or another highly data-driven organisation. Snowflake experience, given ...

levels of the organization.Commitment to developing talent and building high-performing, diverse teams.Experience with platforms and tools such as AWS, Azure, Python, Spark, Airflow, Kubernetes, DBT, Databricks, and modern front-end frameworks.“I can apply in less than 4 minutes.” You’ve reviewed this job posting ...

sequence embeddings.You have experience with uplift modelling and conditional average treatment effects for personalisation. You are familiar with dbt, Snowflake, AWS (S3, SageMaker), Airflow, and Trino/data lake stacks.You have prior CRM/marketing/e-commerce ML experience — customer lifecycle segmentation, campaign attribution, contact strategy optimisation.Office: London ...

production problems and taking responsibility for critical infrastructure. Startup Mindset: Proactive, resourceful and comfortable taking ownership in a fast-moving engineering environment. Bonus : with Apache Airflow, real-time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial-market systems. Why Permutable Real Ownership: Take significant ...

testing frameworks for AI/ML systems Experience with LLM evaluation techniques, prompt engineering, or AI safety concepts Experience with data pipeline tools like Apache Airflow, Dagster, or similar Experience with analytics platforms, experimentation frameworks, or A/B testing infrastructure Background in data science, analytics engineering ...

evaluation or testing frameworks for AI/ML systemsExperience with LLM evaluation techniques, prompt engineering, or AI safety conceptsExperience with data pipeline tools like Apache Airflow, Dagster, or similarExperience with analytics platforms, experimentation frameworks, or A/B testing infrastructureBackground in data science, analytics engineering, or similar data ...

skillsExperience with relational and NoSQL databasesHands-on experience with vector databasesGood understanding of data modelling, performance and storage architectureExperience with technologies such as Spark, Airflow, Kafka or Elasticsearch/OpenSearchExperience with AI/ML platforms, cloud infrastructure or event-driven architectures would be useful, but you don’t need ...

.Databricks Mastery: Hands-on experience with production workloads on Databricks (Azure/AWS), PySpark, Spark SQL, Delta Lake, Unity Catalog, Python, SQL, orchestration tools (Airflow, Workflows, ADF), and streaming (Kafka/Event Hubs).Professional Skills: Strong consultative mindset, excellent communication, and ability to interface with traders and senior stakeholders. ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...