76 to 100 of 131 Apache Airflow Jobs in England

will shape what modern data infrastructure looks like in this space. You'll take full ownership of the backend, building custom data pipelines, orchestrating Airflow style workflows, and turning messy real world inputs into structured data ready for downstream AI models. You'll design APIs, optimise low latency queries ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

cloud‐native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

similar databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph-based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning ...

management. Data Center Optimization Collaborate with facilities and operations teams to optimize infrastructure deployments based on available rack space, power draw, thermal dynamics, and airflow efficiency. Contribute to the planning and provisioning of data center resources, ensuring support for both traditional workloads and AI‐intensive environments. Evaluate emerging technologies ...

visualisation languages and libraries (SQL, python, matplotlib, etc.) as well as some of the supporting tools and platforms (e.g. git, MLFlow, Kubernetes, Kubeflow, Trino, AirFlow). A strong understanding of statistical concepts (hypothesis testing, sampling, probability distributions). Excellent communication and collaboration skills to effectively bridge the gap between ...

forecasting, and statistical modeling in a product environment. Expertise in Python or R, fluency in SQL, and experience with large‐scale data tech (Databricks, Airflow, dBT a plus). Experience with FinTech, payments, crypto, or blockchain data is a strong plus. Advanced degree (MS, PhD) in a quantitative field ...

unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java Python or Scala* Experience in modern engineering practices and technologies* Good working knowledge of data ...

specialist consultancy renowned for delivering mission-critical solutions, working with modern technologies in a secure, cutting-edge environment Key Skills required: - PyTorch - TensorFlow - SciKit - Airflow or Prefect - ML flow (really nice to have) - Python - SQL - Data Modelling This role will require an active SC Clearance and British Citizenship ...

Python, Bash, Ansible, Puppet) Familiarity with containerisation (Docker, Kubernetes) Nice to have: Experience with CI/CD pipelines (GitLab, Jenkins) Exposure to data tooling (Airflow, NiFi, Elasticsearch) Experience with Cloudera/Hadoop ecosystems (Spark, Hive, HDFS) Why join? Work on modern data platforms at scale Collaborate with data scientists ...

Experience Data Modeling Data Integration & Ingestion Data Manipulation and Processing GitHub, GitHub Actions, Azure DevOps Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest Key Characteristics Technology expert who constantly pursues knowledge enhancement and has an inherent curiosity to understand work from multiple dimensions. ...

production Solid software engineering skills (Python, APIs, systems) Good understanding of MLOps stack (CI/CD, monitoring, pipelines) Familiarity with tools like MLflow, Airflow, Evidently, DVC, Docker, Kubernetes. Cloud experience (AWS/GCP/Azure) Some mentoring/leadership experience + desire to grow into management Nice to have ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

abstraction layers Model the cost, feasibility and risk of different LLMs and implementation approaches THE STACK AWS, Bedrock, LangGraph, LangChain, Python, FastAPI, PostgreSQL, Snowflake, Airflow, React, Next.js, Kubernetes. YOUR EXPERIENCE 15+ years in enterprise software engineering with a background in product engineering organisations or technology companies Deeply technical: recent ...

designing and developing scalable data pipelines Proficiency in SQL, Python, Databricks and/or Azure technologies Experience working with tools such as Git and Airflow Strong troubleshooting skills with the ability to test, maintain and improve data platforms A proven track record of delivering high-quality, dependable data solutions ...

We are recruiting for an Analytical Engineer on behalf of one of the UK's most recognised and celebrated personal lines insurers. The successful candidate will be working alongside a talented team of Data & AI ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

About this role: At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software. You can’t spell “Data Orchestration” without “Data”, and we’re looking … data platforms from cloud-native component parts - we have a healthy mix of build vs. buy; sometimes building is the right choice. Experience with Apache Airflow. #LI-Hybrid At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion ...

Ensure solutions meet security, compliance, and regulatory standards Key Skills Required: Experience in solution design/architecture Strong background in big data tools (Spark, Airflow, Trino, etc.) Knowledge of cloud platforms (AWS/Azure/GCP) Understanding of cybersecurity principles (IAM, encryption, secure design) Ability to work hands ...

integration patterns and data pipeline design Desirable Exposure to data governance frameworks and tooling (e.g. Collibra) Experience with orchestration platforms such as Astronomer/Airflow Knowledge of enterprise data architecture practices, including metadata management and lineage Education Bachelors or Masters in Information Technology, Computer Science or relevant field. Work ...

ideally within a large‐scale or regulated environment. AWS Experience: Strong working knowledge of AWS and cloud‐first data pipeline orchestration tools such as Airflow, Glue, or EMR. A Security Mindset: Understanding of compliance and security when managing sensitive data pipelines. Strong Automation Skills: Comfortable using Python and Shell ...

large‐scale or regulated environment. AWS Experience: You have a strong working knowledge of AWS and cloud‐first data pipeline orchestration tools (like Airflow, Glue, or EMR). A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines. Strong Automation Skills ...

love to see: Knowledge of Database Systems and trade offs in the distributed systems Familiarity with API Designs Familiarity with Orchestration Frameworks such as Apache Airflow, Argo Workflows, Dagster etc. Experience working with and designing systems utilizing AWS If indicated, please note that years of experience ...

infrastructure: AWS and/or GCP, Kubernetes APIs and services: REST, gRPC Distributed systems: event-driven architectures, including Kafka Orchestration Frameworks: LangGraph, LangChain, AirFlow, etc Activities Integration of commercial and open-source LLMs into agentic workflows Agent and orchestration frameworks such as LangChain, LlamaIndex, Semantic Kernel, or CrewAI Model ...