76 to 100 of 240 Permanent Apache Airflow Jobs

scripts and CI/CD automationWorking knowledge of modern MLOps stacks including experiment tracking and artifact managementHands-on experience with orchestration tools (e.g., Kubeflow, Apache Airflow, Metaflow or Prefect)Demonstrated skills with Docker, Kubernetes and distributed deploymentsPractical knowledge of Infrastructure-as-Code (Terraform) and a major cloud provider ...

technical issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline.PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

messy data environments. Naturally curious who will dig a level deeper into the data and question assumptions. Desirable Skills (Nice to Have)Snowflake, DBT, Airflow: Practical experience working with Snowflake, DBT and AirflowTrayport is committed to creating and sustaining a collegial work environment in which all individuals are treated ...

lifecycle management and MLOpsExperience with market data, trading, sales, risk, or client analytics platformsHands-on experience with kdb+/q, Spark, Kafka, dbt, Airflow, or similar technologiesExperience leading platform transformation or modernization programmesTechnical SkillsPython, Rust, SQL, API design and distributed computing. Long-horizon agentic platforms, agent memory systems, agentic ...

Bedrock AgentCore or similarExperience with managed or self-hosted ML infrastructure, such as Amazon SageMaker, feature stores (e.g. Feast) and DAG tooling (e.g. Airflow, Metaflow, or similar)Experience with model experiment tracking, ideally MLflow or Weights & Biases, and with monitoring tools such as EvidentlyAI, CometML, Grafana or New RelicExperience ...

lake, lakehouse, warehouse or large-scale analytical platform delivery. • Python, Bash or other scripting and automation approaches. • Workflow orchestration and scheduling tools, such as Airflow, TWS, JS7 or similar. • CI/CD, version control and modern engineering delivery practices. • Data quality, testing, observability and operational resilience. • Working with enterprise ...

assistants as part of the software development lifecycle. Experience designing, building, and operating data pipelines and ETL/ELT workflows using Databricks, dbt, and Apache Airflow. Preferred Skills & ExperienceExperience with frameworks like ReactJS, Angular, or Vue.js to build interactive and responsive user interface. Experience building or designing scalable software ...

Python, Node.js, Java), frontend (React, Angular), APIs (REST/GraphQL) Machine Learning: Foundations of MLflow, Weights & Biases, LoRA, PEFT, SageMaker, Vertex AI, AzureML, Kubeflow, Airflow, Prefect, TorchServe, Triton, vLLM, SHAP, LIME, pandas, Spark, SQLLLM tooling: Familiarity with LangChain, Semantic Kernel, prompt chaining, vector search, and context management Performance & observability ...

models, schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools andtechnologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such asrelational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

TensorFlow, PyTorch). Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid ...

TensorFlow, PyTorch). Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid ...

data warehousing and data modelling principles Experience with Snowflake and modern cloud data platforms Python development skills Experience with workflow orchestration tools such as Airflow Knowledge of streaming technologies and real-time data processing Experience implementing CI/CD and engineering best practices What They Offer Competitive salary ...

solution design responsibility. You'll ideally have: Strong commercial experience with Python & Snowflake Experience mentoring or technically leading other engineers Experience with dbt, Airflow or similar data engineering tooling A good understanding of data modelling and modern data platform architecture Experience working with AWS (ideally) or another major cloud ...

technical leadership and solution design responsibility. You'll ideally have: Strong commercial experience with Python & SnowflakeExperience mentoring or technically leading other engineersExperience with dbt, Airflow or similar data engineering toolingA good understanding of data modelling and modern data platform architectureExperience working with AWS (ideally) or another major cloud platformThe ...

that make vendor data consistent and useful to downstream consumers.- Improve the reliability and observability of existing services and pipelines.- Help migrate and strengthen Airflow-based workflows.- Participate in the shared on-call rota for systems used by the trading teams.- Hold engineering standards across the team: review, testing ...

asynchronous architectures and related messaging technologies (e.g. Kafka, MQ) Experience with cloud platforms (e.g. AWS, GCP, Azure) Experience with orchestration and container technologies (e.g. Airflow, Kubernetes, Docker) Experience with monitoring and alerting tools (e.g. CloudWatch, Prometheus, Grafana, Sentry/OTel ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing.Discover more about our Analytics Community ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

platforms)Experience applying DevSecOps practices within cloud or controlled environmentsExposure to large language models or generative AI systemsExperience with data engineering tools (e.g. Spark, Airflow, Databricks)Experience applying AI and automation to operational or productivity-focused use casesExperience working in secure, restricted, or air-gapped environmentsExperience supporting systems subject ...

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...