1 to 25 of 48 Remote Apache Airflow Jobs

data warehousing (Snowflake, Redshift, BigQuery or Synapse) Understanding of data modelling and CI/CD Desirable Spark, Databricks or Kafka Orchestration tools such as Airflow or dbt Infrastructure as Code (Terraform) Experience supporting ML workloads What's On Offer Competitive salary and bonus Hybrid working Pension, private healthcare, holiday ...

pipelines and warehouses. Strong SQL skills and proficiency in Python or another relevant programming language. Experience with modern data engineering tools, such as dbt, Airflow or Spark. Experience with cloud data warehouses, such as Snowflake, BigQuery or Amazon Redshift. A solid understanding of data modelling, ETL/ ...

Python and SQL programming skills, with experience developing and maintaining data pipelines Familiarity with ETL/ELT technologies and data streaming tools such as Airflow, Azure, SSIS, Spark or Kafka Experience working with databases and data platforms such as PostgreSQL, SQL Server, Oracle, Snowflake or similar technologies Knowledge ...

Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills with ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working ...

from various structured and unstructured sources (databases, APIs, streaming platforms) into BigQuery and Cloud Storage Implementing data ingestion and real-time processing using Dataflow (Apache Beam) and Pub/Sub for batch and streaming workflows Developing SQL transformation workflows with Dataform , including version control, testing, and automated scheduling with … built-in quality assertions Creating efficient, cost-optimized BigQuery queries with proper partitioning, clustering, and denormalization strategies Orchestrating complex workflows using Cloud Composer (Apache Airflow) and Cloud Functions for event-driven data processing Implementing centralized data governance and metadata management using Dataplex with automated cataloging and lineage tracking ...

from various structured and unstructured sources (databases, APIs, streaming platforms) into BigQuery and Cloud Storage Implementing data ingestion and real-time processing using Dataflow (Apache Beam) and Pub/Sub for batch and streaming workflows Developing SQL transformation workflows with Dataform , including version control, testing, and automated scheduling with … built-in quality assertions Creating efficient, cost-optimized BigQuery queries with proper partitioning, clustering, and denormalization strategies Orchestrating complex workflows using Cloud Composer (Apache Airflow) and Cloud Functions for event-driven data processing Implementing centralized data governance and metadata management using Dataplex with automated cataloging and lineage tracking ...

from various structured and unstructured sources (databases, APIs, streaming platforms) into BigQuery and Cloud Storage Implementing data ingestion and real-time processing using Dataflow (Apache Beam) and Pub/Sub for batch and streaming workflows Developing SQL transformation workflows with Dataform , including version control, testing, and automated scheduling with … built-in quality assertions Creating efficient, cost-optimized BigQuery queries with proper partitioning, clustering, and denormalization strategies Orchestrating complex workflows using Cloud Composer (Apache Airflow) and Cloud Functions for event-driven data processing Implementing centralized data governance and metadata management using Dataplex with automated cataloging and lineage tracking ...

/on-prem environments. The ideal candidate will have: * Experience in data engineering, ideally aerospace/defence or industrial sectors. * ETL/ELT tools (Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). * Strong Python and SQL; C++, C#, or MATLAB a plus. * Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg ...

Kanban. Strong analytical and problem-solving skills, with a focus on delivering high-quality solutions. Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies. Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non-technical audiences. Unfortunately ...

such as SageMaker, Vertex AI or Azure MLData engineering and MLOpsExperience with data warehouses such as Snowflake, BigQuery or RedshiftWorkflow orchestration using tools like Airflow or DagsterExperience with MLOps tools such as MLflow, Weights and Biases or similarAwareness of data and model drift, and how to monitor and respond ...

skills Experience building data pipelines and integrating data from multiple sources Exposure to technologies such as Power BI, Snowflake, SQL Server, PostgreSQL, Spark, Kafka, Airflow, Docker, and Kubernetes Experience within engineering, manufacturing, industrial, or other complex regulated environments would be advantageous Excellent stakeholder management and problem-solving abilities What ...

with reverse ETL and moving data between warehouses and operational systems Strong understanding of APIs Familiarity with modern data engineering tools such as dbt, Airflow, Fivetran or Census Excellent stakeholder management skills If you're an experienced Senior Data Engineer who enjoys solving complex integration challenges, working with modern ...

ADLS, and Azure Functions. Proven track record of designing and implementing metadata-driven data pipelines. Deep expertise in orchestration and data workflow automation e.g. Airflow, DBT. Strong understanding of CI/CD practices for data engineering. Experience with infrastructure as code (Terraform, Bicep, or ARM templates) (preferred). Solid ...

Snowpark Iceberg Tables Secure Data Sharing Snowflake Cortex Data Integration Fivetran SnapLogic Enterprise SaaS integration patterns Data Transformation & Orchestration dbt Cloud/dbt Core Apache Airflow (MWAA) CI/CD pipelines Event-driven orchestration Data Governance & Observability Collibra Data Catalogue Data Lineage Data Governance Frameworks Acceldata Data Observability ...

need both skillsets below - strength in one is enough: Hands-on: building AI/ML pipelines or prototypes; vector databases; prompt engineering; orchestration tools (Airflow/Prefect); Git Advisory: designing AI/data architectures; Agentic AI principles (LangGraph, CrewAI, AutoGen); mentoring specialists without formal authority; AI governance (security, privacy ...

pipelines Managing AWS cloud data services (Glue, DMS, Athena preferred) Setting up and managing a cloud data warehouse (Redshift preferred) Hands-on working with Airflow or an equivalent orchestrator in production (Dagster, Prefect) Solid understanding of data modelling: dimensional modelling, slowly changing dimensions, and event-sourced schemas Other Desired ...

production (not just training models in notebooks), including familiarity with serving patterns (e.g., REST APIs, batch inference, event-driven) Experience with orchestration tools (e.g., Airflow, Dagster) and cloud-native ML platforms (e.g., AWS SageMaker, GCP Vertex AI) Ability to influence decision-making regarding infrastructure and tooling, balancing 'build vs. ...

hands-on experience in at least one data area and one platform area below. Data experience Orchestration: operating pipelines with tools such as Airflow, Dagster or Prefect, including dependencies, retries, backfills, reruns or schema changes. Storage and modelling: designing schemas and storage for relational databases, warehouses or lakehouses ...

scripts and CI/CD automationWorking knowledge of modern MLOps stacks including experiment tracking and artifact managementHands-on experience with orchestration tools (e.g., Kubeflow, Apache Airflow, Metaflow or Prefect)Demonstrated skills with Docker, Kubernetes and distributed deploymentsPractical knowledge of Infrastructure-as-Code (Terraform) and a major cloud provider ...

issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline. PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

lake, lakehouse, warehouse or large-scale analytical platform delivery. • Python, Bash or other scripting and automation approaches. • Workflow orchestration and scheduling tools, such as Airflow, TWS, JS7 or similar. • CI/CD, version control and modern engineering delivery practices. • Data quality, testing, observability and operational resilience. • Working with enterprise ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...

times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £85k + bonus X3 weeks per year work anywhere ...

advanced analytics — ideally in a hybrid PLG/sales-led SaaS environmentStrong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster)Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables)Experience working directly with product ...