76 to 100 of 271 Apache Airflow Jobs in England

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks Ability to think strategically and relate architectural decisions ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing Vector Databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

scripting language — Python is required.Proficiency in at least one object-oriented language.Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto.Experience with AWS, GCP, or Looker (advantageous but not essential).Solid background in data modelling, including the design, implementation and optimisation of conceptual ...

Backstage experience - software catalog configuration, scaffolder templates, or plugin development* Familiarity with data platform concepts - medallion architecture, lakehouse patterns, or orchestration tools such as Airflow or dbt* Exposure managing Databricks* Practical experience with AI coding assistants (Claude Code, GitHub Copilot, Cursor, or similar) in a professional engineering context* Experience ...

find to improve model and system performance. Build reliable ML infrastructure Develop and maintain data and machine learning workflows using technologies such as Apache Airflow. Help improve CI/CD, automated testing, monitoring and reproducibility across our ML stack. Engineer on AWS Build and improve cloud infrastructure using services … expect you to know everything: PyTorch, TensorFlow or JAX Hugging Face LLM APIs and open-source models RAG and vector databases Apache Airflow AWS S3 ECS/EKS Redshift Docker Kubernetes GitHub Actions Pulumi or Terraform SQL Model monitoring and observability Distributed processing Financial markets, economics or commodities ...

infrastructure, and orchestration. What You’ll Be Doing Develop and maintain scalable data solutions using Snowflake Design and automate ETL/ELT pipelines with Airflow or Dagster Implement and manage infrastructure as code using Terraform Drive performance tuning, cost optimisation, and platform scalability Collaborate closely with data scientists, engineers ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

PostgreSQL migration projects. Knowledge of modern analytics platforms such as Databricks, Snowflake, Synapse, Fabric, or Redshift. Familiarity with ETL/ELT tooling including dbt, Airflow, Informatica, Talend, SSIS, or similar technologies. Contract Details Position: Data Architect Engagement: 3-6 Month Contract Day Rate: £400 per day IR35: Outside IR35 ...

debugging, the advanced stuff that separates architects from diagram-drawers. Lakehouse fluency — Medallion architecture, Delta Lake, Unity Catalog, governance, orchestration (Workflows, DLT/Lakeflow, Airflow, ADF); Microsoft Fabric exposure a bonus. Battle-tested in migrations — you've led large transformation programs and have the scars and success stories ...

taxonomies/ontologies, and versioned/temporal data.Understanding of how data structures feed RAG and LLM retrieval quality.SQL proficiency; experience with orchestration tools (e.g., Airflow) and cloud data services.Familiarity with data governance, security, and privacy practices.Strong documentation and cross-team communication skills.WHO YOU AREYou think about how to scale ...

building and maintaining modular, scalable data models that follow best practicesStrong understanding of dimensional modellingFamiliarity with AWS data services ( S3, Athena, Glue )Experience with Airflow for scheduling and orchestrating workflowsExperience working with data lakes or modern data warehouses (Snowflake, Redshift, BigQuery)A pragmatic problem solver who can balance technical ...

technologies and practices such as Docker, Terraform, CI/CD pipelines and infrastructure‐as‐code. Familiarity with MLOps tools and practices such as MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management. Strong Python experience, with helpful knowledge of Spark or PySpark ...

analytics or data platforms, or significantly evolving existing ones.Experience with data quality and observability, using dbt or complementary tooling.Familiarity with orchestration tools such as Airflow, Dagster, or Prefect.Experience with semantic layers, metrics frameworks, or analytics consumption patterns.Exposure to data governance, access control, or analytics enablement at scale.Experience mentoring ...

metrics layers. Experience applying AI to Analytics Engineering workflows or developing AI-ready data products. Knowledge of orchestration and ingestion tools such as Airflow,DagsterorFivetran. Python experience for automation or data engineering tasks. Experience working in regulated financial services or another highly data-driven organisation. Snowflake experience, given ...

levels of the organization.Commitment to developing talent and building high-performing, diverse teams.Experience with platforms and tools such as AWS, Azure, Python, Spark, Airflow, Kubernetes, DBT, Databricks, and modern front-end frameworks.“I can apply in less than 4 minutes.” You’ve reviewed this job posting ...

sequence embeddings.You have experience with uplift modelling and conditional average treatment effects for personalisation. You are familiar with dbt, Snowflake, AWS (S3, SageMaker), Airflow, and Trino/data lake stacks.You have prior CRM/marketing/e-commerce ML experience — customer lifecycle segmentation, campaign attribution, contact strategy optimisation.Office: London ...

production problems and taking responsibility for critical infrastructure. Startup Mindset: Proactive, resourceful and comfortable taking ownership in a fast-moving engineering environment. Bonus : with Apache Airflow, real-time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial-market systems. Why Permutable Real Ownership: Take significant ...

Environment Experience with some of the following is highly desirable: PostgreSQL | SQL Server | AWS Aurora | RDS | Azure | Snowflake | Databricks | Synapse | Fabric | dbt | Informatica | Talend | Airflow | SSIS Contract Details £510/day maximum - Outside IR35 London Hybrid - 2-3 days onsite per week Immediate start preferred Maximum 10-day notice ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

tools. Experience building and optimising scalable data pipelines and data warehouses. Knowledge of AWS technologies such as Redshift, S3 and Athena. Experience with Airflow, Fivetran or similar orchestration and integration tools. Strong Linux/Unix and Bash scripting skills. A solid understanding of data quality, governance, lineage and monitoring. ...

technical issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline.PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...