76 to 100 of 251 Apache Airflow Jobs

Data Vault, dimensional modelling or complex entity resolution. Building data foundations for machine learning, anomaly detection or GenAI applications. Experience with Azure Data Factory, Airflow or similar orchestration technologies. Why consider it? This is an opportunity to take genuine technical ownership of an important data engineering capability, rather than ...

every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship modelling and Dimensional modellingExperience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery) and Cloud ...

hands-on experience in at least one data area and one platform area below. Data experience Orchestration: operating pipelines with tools such as Airflow, Dagster or Prefect, including dependencies, retries, backfills, reruns or schema changes. Storage and modelling: designing schemas and storage for relational databases, warehouses or lakehouses ...

with Pandas or NumPy, scikit-learn, and model evaluation or tuningExperience building ETL pipelines and querying APIs using tools such as Airflow or dbt, and Git-based version controlExperiencedeveloping predictive models and quantitative analyses that support intelligence assessments and operational decision-makingExperience applying NLP or text mining to reporting ...

with text, audio, or video in AI applications.Experience evolving shared AI infrastructure or platforms used by multiple product teams.Experience with orchestration tools such as Airflow or Argo Workflows.Familiarity with SQL, Spark, Snowflake, or other data processing technologies.Experience with AI security, privacy, responsible AI, prompt injection protection, or data leakage ...

audio, or video in AI applications. Experience evolving shared AI infrastructure or platforms used by multiple product teams. Experience with orchestration tools such as Airflow or Argo Workflows. Familiarity with SQL, Spark, Snowflake, or other data processing technologies. Experience with AI security, privacy, responsible AI, prompt injection protection ...

scripts and CI/CD automationWorking knowledge of modern MLOps stacks including experiment tracking and artifact managementHands-on experience with orchestration tools (e.g., Kubeflow, Apache Airflow, Metaflow or Prefect)Demonstrated skills with Docker, Kubernetes and distributed deploymentsPractical knowledge of Infrastructure-as-Code (Terraform) and a major cloud provider ...

technical issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline.PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

messy data environments. Naturally curious who will dig a level deeper into the data and question assumptions. Desirable Skills (Nice to Have)Snowflake, DBT, Airflow: Practical experience working with Snowflake, DBT and AirflowTrayport is committed to creating and sustaining a collegial work environment in which all individuals are treated ...

lifecycle management and MLOpsExperience with market data, trading, sales, risk, or client analytics platformsHands-on experience with kdb+/q, Spark, Kafka, dbt, Airflow, or similar technologiesExperience leading platform transformation or modernization programmesTechnical SkillsPython, Rust, SQL, API design and distributed computing. Long-horizon agentic platforms, agent memory systems, agentic ...

Bedrock AgentCore or similarExperience with managed or self-hosted ML infrastructure, such as Amazon SageMaker, feature stores (e.g. Feast) and DAG tooling (e.g. Airflow, Metaflow, or similar)Experience with model experiment tracking, ideally MLflow or Weights & Biases, and with monitoring tools such as EvidentlyAI, CometML, Grafana or New RelicExperience ...

lake, lakehouse, warehouse or large-scale analytical platform delivery. • Python, Bash or other scripting and automation approaches. • Workflow orchestration and scheduling tools, such as Airflow, TWS, JS7 or similar. • CI/CD, version control and modern engineering delivery practices. • Data quality, testing, observability and operational resilience. • Working with enterprise ...

assistants as part of the software development lifecycle. Experience designing, building, and operating data pipelines and ETL/ELT workflows using Databricks, dbt, and Apache Airflow. Preferred Skills & ExperienceExperience with frameworks like ReactJS, Angular, or Vue.js to build interactive and responsive user interface. Experience building or designing scalable software ...

Python, Node.js, Java), frontend (React, Angular), APIs (REST/GraphQL) Machine Learning: Foundations of MLflow, Weights & Biases, LoRA, PEFT, SageMaker, Vertex AI, AzureML, Kubeflow, Airflow, Prefect, TorchServe, Triton, vLLM, SHAP, LIME, pandas, Spark, SQLLLM tooling: Familiarity with LangChain, Semantic Kernel, prompt chaining, vector search, and context management Performance & observability ...

models, schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools andtechnologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such asrelational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

TensorFlow, PyTorch). Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid ...

TensorFlow, PyTorch). Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid ...

data warehousing and data modelling principles Experience with Snowflake and modern cloud data platforms Python development skills Experience with workflow orchestration tools such as Airflow Knowledge of streaming technologies and real-time data processing Experience implementing CI/CD and engineering best practices What They Offer Competitive salary ...

solution design responsibility. You'll ideally have: Strong commercial experience with Python & Snowflake Experience mentoring or technically leading other engineers Experience with dbt, Airflow or similar data engineering tooling A good understanding of data modelling and modern data platform architecture Experience working with AWS (ideally) or another major cloud ...

technical leadership and solution design responsibility. You'll ideally have: Strong commercial experience with Python & SnowflakeExperience mentoring or technically leading other engineersExperience with dbt, Airflow or similar data engineering toolingA good understanding of data modelling and modern data platform architectureExperience working with AWS (ideally) or another major cloud platformThe ...

that make vendor data consistent and useful to downstream consumers.- Improve the reliability and observability of existing services and pipelines.- Help migrate and strengthen Airflow-based workflows.- Participate in the shared on-call rota for systems used by the trading teams.- Hold engineering standards across the team: review, testing ...

asynchronous architectures and related messaging technologies (e.g. Kafka, MQ) Experience with cloud platforms (e.g. AWS, GCP, Azure) Experience with orchestration and container technologies (e.g. Airflow, Kubernetes, Docker) Experience with monitoring and alerting tools (e.g. CloudWatch, Prometheus, Grafana, Sentry/OTel ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...