26 to 50 of 126 Permanent Apache Airflow Jobs

across ML Ops, data architecture and data engineering best practices and scalable data solutions. Proficiency in Python and SQL, ideally with frameworks such as Airflow, PyTorch or Spark. Good understanding of supervised and unsupervised learning and how to choose/apply the right model to a business problem. Familiarity ...

cloud data platforms (AWS preferred) Deep experience with AWS, Databricks, Power Platform, and Redshift (Snowflake a plus) Proficiency in AWS Glue, Qlik Talend, DBT, Airflow, and modern data integration tools. Excellent knowledge of Python, SQL, PowerQuery (M), and preferably Scala or PySpark Working knowledge with enterprise architecture frameworks (e.g. ...

performance marketing data Experience working on large-scale data transformation or platform modernisation projects Tech Stack Warehouse: Snowflake Transformation: dbt BI: Looker Orchestration: Airflow Version Control: GitHub Other Tools: Adverity, OpenMetadata, Jira, Confluence Benefits Hybrid working (minimum of 3 days per week in office) 25 days annual leave plus ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

data/AI platforms Deep knowledge of GCP (BigQuery, Dataflow, Pub/Sub, Vertex AI) Experience with LLMs (e.g. Gemini or similar) Familiarity with Airflow/Composer and dbt Understanding of data governance and financial services regulation Strong stakeholder management skills What’s on Offer Competitive salary + bonus ...

/lending/risk/portfolio datasets Time series familiarity (trend/seasonality, cohorts over time, monitoring) Experience with data tools such as dbt, Airflow, Snowflake/BigQuery/Databricks (or similar) Experience creating automated data quality frameworks or reconciliation checks Benefits: £45,000 – £55,000 salary Hybrid working ...

Deep understanding of data platforms, orchestration , metadata management , CI/CD and architectural components such as DevOps, Azure Data Factory, BigQuery, Databricks, MS Fabric, Airflow, Snowflake etc. Demonstrated experience architecting solutions using medallion architectures , including curated gold-layer models designed for analytics consumption. Strong knowledge of unity catalog ...

associated services. Strong programming skills in Python, Java, or Scala for data engineering. Hands-on experience with data orchestration and workflow management tools (e.g., Airflow, Dagster, Prefect). Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion …/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge of data transformation and ETL pipelines and APIs. Security cleared or ability to obtain (SC or above). Desirable Skills ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

discipline Solid Python programming skills Good working knowledge of SQL Comfortable using Git for version control Desirables: Exposure to workflow orchestration tools (e.g. Prefect, Airflow, Dagster) Experience with cloud data warehouses (Azure SQL, Snowflake) or dbt Basic familiarity with Docker and BI tools (Power BI, Tableau) Interest in shipping ...

pipeline design Mastery of data modeling techniques Skilled in data ingestion, integration, and transformation using modern tools (Data Factory, Databricks, SQL DB, Synapse, Glue, Airflow, Kinesis, Redshift) Hands-on experience with DevOps tooling: GitHub Actions, Azure DevOps, SonarQube, PyTest Proven ability to coordinate incident, problem, and change management processes ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8–10 years ’ experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

What they're looking for Strong Python + data engineering fundamentals Experience building scalable data pipelines/platforms Exposure to distributed systems (Kafka, Spark, Airflow etc.) Interest in AI/ML systems is a plus ...

custom ETL pipelines in a cloud-native environment. • Automating internal processes and integrations across tools and APIs. • Supporting workflow orchestration using technologies such as Airflow or AWS Step Functions. • Containerising and deploying services using Docker. • Collaborating with data scientists to streamline their data workflows. • Contributing to improvements across orchestration ...

testable systems Solid background in microservices architecture Proficiency with Postgres & MongoDB (relational + non-relational) Experience with event-driven architectures and asynchronous workflows (Kafka, Airflow, etc.) Solid coding practices (clean, testable, automated) The mindset of a builder: thrives in fast-paced startup environments, takes ownership, solves complex challenges Bonus ...

based ID resolution, fuzzy match frameworks). Proven experience leading a development team on client lifecycle management products. Familiarity with BPMN tools (e.g., Camunda, Airflow, Drools) related to customization of modeler and engine. Nice to have Experience with enterprise architecture frameworks (TOGAF, Zachman) and threat modeling (STRIDE). Knowledge ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

peer to peer feedback and evolving healthy, curious and humble teams. Tech Stack: Python, Javascript/Typescript, React/React Native, AWS, GraphQL, Snowflake, Airflow, DDD. This is an incredible opportunity for a Senior Software Engineer to join a unique company as they embark on a period of significant ...

healthcare, fintech, or other regulated environments Exposure to LLM infrastructure, model serving, or agentic AI frameworks in production Familiarity with data platforms (BigQuery, Airflow, data pipelines) Experience with CI/CD pipelines (GitHub Actions, Cloud Build) and policy-as-code tools Exposure to observability stacks (OpenTelemetry, Prometheus, Cloud Monitoring ...

build the queries and processes required to migrate it into a new environment. As migration work concludes, the role will transition into leveraging Airflow and other tooling to automate operational and financial processes. This position requires a highly collaborative individual, as you will work closely with engineers and stakeholders … Work with solution architects to design migration-day data processes for partner onboarding. Build and maintain ETL pipelines and workflow automation using Snowflake and Apache Airflow. Document processes, learnings, and development work using version control best practices. Implement robust data quality checks and reconciliation processes throughout pipelines. Collaborate with ...

build the queries and processes required to migrate it into a new environment. As migration work concludes, the role will transition into leveraging Airflow and other tooling to automate operational and financial processes. This position requires a highly collaborative individual, as you will work closely with engineers and stakeholders … Work with solution architects to design migration-day data processes for partner onboarding. Build and maintain ETL pipelines and workflow automation using Snowflake and Apache Airflow. Document processes, learnings, and development work using version control best practices. Implement robust data quality checks and reconciliation processes throughout pipelines. Collaborate with ...