176 to 200 of 315 Apache Airflow Jobs in London

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...

architectures, or agentic systems. Experience designing external-facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications ...

libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like private healthcare & dental insurance, a generous work from ...

/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.)PySparkAWS cloud infrastructure: EMR, ECS, Athena, etcMLOps/DevOps: Terraform, Docker, Airflow, MLFlowMonitoring and evaluation platforms including New Relic and BraintrustMore information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy ...

organisation Deep understanding & hands‐on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

skills and fluency in Python or R, with experience applying analytical and statistical methods to business problems. Experience with orchestration tools (e.g., DBT, Airflow) for deploying reliable data workflows. Strong communication and empathy - ability to translate complex data concepts for non-technical stakeholders. Collaborativeproduct mindset- comfort working closely with ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...

models, schemas to support business requirementsDevelop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL)You have framework experience within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data qualityPreparing ...

models, schemas to support business requirementsDevelop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

enhancement programme. Investment Data Engineer, key skills: Strong commercial experience in Data Engineering, particularly using Snowflake Proven experience developing and maintaining workflows using Apache Airflow and DBT (Data Build Tool) Experience integrating and supporting Bloomberg data feeds, services or associated investment data solutions Strong SQL development and data … with minimal ramp-up time Strong analytical and problem-solving skills Excellent stakeholder communication and collaboration abilities Investment Data Engineer, Bloomberg, SQL, Python, DBT, Airflow Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built ...

within financial markets/securities . Experience working with GCP BigQuery . Experience with GCP Cloud SQL . Experience with Terraform . Experience with Apache Airflow . Understanding of security reference data, pricing data or corporate actions. The Project The wider project is focused on creating a multi ...

business intelligence in a security-conscious environment. The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth of the company. #J-18808-Ljbffr ...

development experience, with strong skills in Python. Hands‐on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow #J-18808-Ljbffr ...

tools such as make and CMake, and automated build systems such as spack or easybuild Experience with workflow orchestration tools such as Argo Workflow, Airflow, Nextflow, Snakemake, VisTrails, or Cromwell Experience with application performance tuning and optimization in parallel and distributed computing, including MPI, OpenMP, or Gloo Deep understanding ...

degree, preferably in Computer Science, Engineering, Mathematics, or similar technical discipline. Nice to Have Snowflake SQL expertise. AWS Cloud experience – including EC2 & S3. Airflow/MWAA or equivalent job scheduler experience. Kubernetes, EKS, Lambda, or serverless experience in AWS. Personal Attributes Team player with strong analytical and problem‐solving ...

similar databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph–based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

track record of delivering impact Desirable: Experience with GenAI/LLM evaluation, AI safety, prompt engineering, experimentation frameworks or data pipeline technologies such as Airflow or Dagster. What we’ll provide you This is a unique role that will put you in the position to be part ...

understanding of functional paradigms and type theory. Confident JVM knowledge. Modern programming languages knowledge in our primary languages (Java, Python, Clojure). Experience with Airflow or other Python‐based workflow orchestration tools. Proficiency in domain‐driven design and domain modeling. Exposure to Kubernetes, Docker, Linux, Kafka, RabbitMQ, or git. ...