426 to 450 of 458 Apache Airflow Jobs in England

engineering & the platform Build and maintain ingestion and transformation pipelines across our stack. We use a modern data stack, such as AWS, Redshift, and Airflow Write production‐quality Python for extraction, transformation, and automation tasks that fall outside dbt’s remit Help us modernise: we’re actively evolving this … plain language Ownership: you’re comfortable being the first and only person in this role, and building things properly from scratch Nice‐to‐haves Airflow or similar orchestration tools, and familiarity with data ingestion tools such as dlt or Airbyte Warehouse cost and performance tuning, particularly on Redshift Fintech ...

We tackle the most complex problems in quantitative finance, by bringing scientific clarity to financial complexity. From our London HQ, we unite world-class researchers and engineers in an environment that values deep exploration and ...

**Senior Data Engineer (Data Modelling)****UK Remote-First** (occasional travel to parks, head office and team events)**Permanent | Full Time****Competitive salary + bonus and benefits****About the role**Do you think in facts and ...

data scientists. How to make an Impact Architect and Implement : Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development and solve any complex performance bottlenecks, concurrency issues … deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

What will you be working on? Data Engineering Build and Optimise Pipelines: Develop, optimise, and maintain data pipelines using Python, REST APIs, DBT, and Airflow to ensure seamless data flow for compliance and risk management. Understanding Control Requirements & Continuous Monitoring: Partner closely with control owners across security domains (such … organisation. QualificationsTechnical Expectations: Proficient in Python, with experience in automation and building data pipelines. Strong expertise in working with tools like DBT and Airflow for ETL processes. Skilled in using REST APIs to develop and manage data integrations. Proficient in SQL and familiar with data visualization tools such ...

Data Engineering team, which builds and governs the data foundation that powers the business. You'll work within our stack — Python ingestion pipelines, Airflow orchestration, and Snowflake/Databricks — helping move data reliably and securely from source to decision-ready output.You'll implement features and fixes against a given … with solid experience building and operating production data pipelines Comfort investigating and root-causing pipeline issues independently before escalating Experience with workflow orchestration tools (Airflow or similar) and a cloud data warehouse/lakehouse (Snowflake, Databricks, or similar) Solid understanding of data pipeline concepts: idempotency, schema evolution, backfills ...

Implement data-quality checks, testing and documentation Work with distributed data technologies such as Presto, Spark or Hive Use workflow orchestration tools such as Airflow Participate in code reviews and Git-based development workflows Collaborate with Data Engineers, Analysts and Engineering teams The core work is production data engineering … data pipelines Advanced SQL , including performance optimisation on large tables Strong Python for data engineering Experience with Presto, Spark, Hive or similar Experience with Airflow or another workflow scheduler Strong ETL/data pipeline development experience Git and code-review experience Strong understanding of data quality and accuracy Experience ...

that powers a suite of AI and machine learning services. Design, build, and maintain production-grade ML orchestration pipelines using tools such as Dagster, Airflow, Prefect, or similar technologies. Deploy, monitor, and scale machine learning models and LLM-powered solutions in production environments. Build and maintain cloud-native infrastructure … have: Strong commercial experience in Python software engineering. Experience deploying and managing machine learning infrastructure in production. Knowledge of orchestration tools such as Dagster, Airflow, Prefect, or similar. Hands-on experience with Kubernetes and containerised workloads. Expertise in Infrastructure as Code using Terraform, Bicep, Pulumi, or comparable technologies. Experience ...

Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment/config: Ansible (physical), Terraform/Argo CD/Helm (Kubernetes) Development environment: MacBook … Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code Degree in Computer Science or numerical degree Competitive salary ...

Robert Half Technology are assisting a market-leading threat intelligence organisation to recruit a Privacy & Data Security Engineer on a 2-year temporary contract basis. Hybrid working - London, 2-3 days per week. Role The ...

Data Scientist - Measurement, Experimentation & Causal Inference United Kingdom, London Why Sony Interactive Entertainment? Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment ...

driven user experience. Hybrid role with three on-site days in City of London. Responsibilities include building scalable data pipelines (BigQuery, Dataflow/Apache Beam, Airflow), ensuring data quality and observability (Looker, Monte Carlo), and collaborating with product engineering and data science teams to plan data tracking ...

ensure data quality end‐to‐end. The ideal candidate will have 8+ years in software/data engineering, strong SQL, and experience with Spark, Airflow, and cloud technologies. #J-18808-Ljbffr ...

value, and/or FICC options trading Experience with Linux-based, concurrent, high-throughput, low-latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced ...

store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree ...

Designing and implementing data classification and governance-by-design Maintaining and improving a GCP/BigQuery lakehouse environment Building and supporting pipelines using Python, Airflow/Cloud Composer and dbt Identifying security vulnerabilities and supporting remediation Implementing robust CI/CD, testing and Terraform-based infrastructure Working across integrations … Deep knowledge of RBAC/IAM within cloud data environments Strong understanding of data security, governance and access controls Hands-on experience with Python, Airflow and dbt Experience with Terraform and CI/CD Knowledge of Kubernetes Exposure to AWS alongside GCP Strong stakeholder management and the ability ...

Inside IR35 Location: London - Hybrid, 3 days per week onsite Duration: 6 months The role will focus on GCP, BigQuery, Python, Airflow/Cloud Composer, dbt, Terraform and CI/CD, working closely with architects, engineers, product and governance teams to deliver secure, scalable data solutions. Key experience … BigQuery Python, Airflow & dbt Terraform & CI/CD Data platforms/lakehouse environments Kubernetes AWS alongside GCP Strong stakeholder management If this sounds relevant to you, please drop me a message or send your CV Thanks, Callum ...

United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. You will contribute to data ...

product and commercial teams. You Need Expert Python and SQL, with experience building production ML systems and data pipelines (PyTorch, XGBoost, Spark/PySpark, Apache Iceberg, Ray, MLflow, SageMaker, Airflow, or equivalent). Deep experimentation and statistical inference skills, and a strong grasp of optimisation under uncertainty (bandits ...

Foundry or AI Search, with Docker and Kubernetes Understanding of schema validation frameworks (e.g., JSON Schema, Pydantic) and MLOps tools such as MLflow or Airflow Strong communication ability to explain trade-offs around cost, latency and accuracy to technical and non-technical audiences Nice to have Experience using Azure … agent and tool interoperability Hands-on work with vector databases like Pinecone, Weaviate, Qdrant or pgvector Familiarity with distributed systems, workflow engines (Temporal, Airflow or Dagster) and event-driven architectures Experience with open-source LLMs or Small Language Models for custom deployments Knowledge of AI safety and governance: guardrails ...

working on the most impactful opportunities NICE TO HAVE BUT NOT ESSENTIAL Experience delivering high quality data and self-serve analytics using tools like Airflow, dbt and Looker. Practically used + applied AI/machine learning techniques and built predictive models to get deeper insights Knowledge of the FinTech … working on the most impactful opportunities NICE TO HAVE BUT NOT ESSENTIAL Experience delivering high quality data and self-serve analytics using tools like Airflow, dbt and Looker. Practically used + applied AI/machine learning techniques and built predictive models to get deeper insights Knowledge of the FinTech ...

Exploration of new data sources to test whether they can be of value to Zempler Bank Implement models into production using tools such as Airflow, PyFlink, BentoML, Git, Jenkins and Octopus Participate in business projects Manage the relationship with the model stakeholder Assist in requirements gathering and project scope … highly imbalanced datasets for binary classification problems Event streaming data engineering tools such as Flink or Spark Experience with MLOps and tools such as Airflow, BentoML and MLFlow Use of Docker CI/CD and SDLC (e.g. JIRA, Git, Jenkins, Octopus) Job type Permanent Skills DataScientist Why Zempler Bank ...

Zohorecruit is seeking a Data Engineer in London for a 4-month hybrid engagement, with 3 days onsite. You will design, test and maintain scalable data architectures and ETL pipelines, collaborating with data teams to ...

seeking a Lead Data Engineer to own a business domain within a modern Snowflake, Airflow, and dbt data platform, combining hands-on technical leadership with team coordination in Greater London. You will shape how data products are designed, built, and improved, ensuring stable platform operations, high data quality ...

ideally within FinTech or Financial Services Exposure to Derivatives or other Financial instruments Strong SQL experience Strong Python experience (PySpark, Pandas, Jupyter Notebooks etc.) Airflow/Algo for workflow management Git Any exposure to compressed file formats, such as Parquet or HDF5, is highly advantageous Linux/Bash skills … highly desirable Job title: Data Analyst Client: Elite FinTech Salary: £65,000-£100,000 + Bonus Location: London Skills: SQL, Python, PySpark, Airflow, Linux #J-18808-Ljbffr ...