1 to 25 of 32 Remote/Hybrid Apache Airflow Jobs

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

professionals with strong Python development expertise, cloud engineering experience, and exposure to modern DevOps and data engineering practices. Must Have: AWS services; Terraform; Apache Spark; Python; Apache Airflow; Docker; GitLab; Security scanning agent; Jupyter; Notebooks; Key Responsibilities Design, develop, and maintain scalable Python applications and services Build … optimize cloud-native data processing pipelines Develop and manage workflows using orchestration tools such as Airflow Work with large-scale data processing frameworks including Apache Spark Deploy and maintain containerized applications using Docker Implement infrastructure automation using Terraform Integrate solutions with enterprise data and secure file transfer systems ...

data validation and application testing processes across distributed systems Investigate production issues using data analysis to identify root causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats … additional programming language (e.g., Java) Experience with scripting languages such as Bash Strong understanding of automated testing frameworks and methodologies Hands-on experience with Apache Airflow and log analysis Familiarity with data processing tools such as Apache Spark Experience working with cloud platforms and services, including: Monitoring ...

producing a clear blueprint for future delivery. Key responsibilities include: * Mapping and documenting the current data architecture and reporting ecosystem * Analysing existing Python and Apache Airflow data pipelines * Tracing data flows from multiple partner data sources through Google Cloud environments into reporting platforms * Reviewing and documenting transformation logic … platforms * Excellent documentation and communication skills * Experience with Tableau and Power BI environments * Knowledge of Google Cloud Platform (GCP) * Familiarity with PostgreSQL, Python and Apache Airflow * Experience within digital media, publishing, advertising technology or affiliate commerce environments * Understanding of programmatic advertising and revenue reporting models * Previous experience supporting ...

producing a clear blueprint for future delivery. Key responsibilities include: * Mapping and documenting the current data architecture and reporting ecosystem * Analysing existing Python and Apache Airflow data pipelines * Tracing data flows from multiple partner data sources through Google Cloud environments into reporting platforms * Reviewing and documenting transformation logic … platforms * Excellent documentation and communication skills * Experience with Tableau and Power BI environments * Knowledge of Google Cloud Platform (GCP) * Familiarity with PostgreSQL, Python and Apache Airflow * Experience within digital media, publishing, advertising technology or affiliate commerce environments * Understanding of programmatic advertising and revenue reporting models * Previous experience supporting ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills with ...

technical stakeholders Strong communication, problem-solving and analytical skills Desirable Experience Experience with Azure, AWS or GCP Exposure to orchestration tools such as Airflow, Dagster or Fabric Pipelines Experience with Git, CI/CD pipelines and DataOps practices Knowledge of Microsoft Purview, Unity Catalog or similar governance tools Experience ...

model on hybrid/on-prem environments. Hard Skills Experience in data engineering, ideally aerospace/defence or industrial sectors. ETL/ELT tools (Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg ...

including optimisation of complex queries (E). Experience building data pipelines and ETL/ELT workflows using tools such as: Azure Data Factory, Databricks, Airflow, Luigi, or similar (E) Strong understanding of data modelling (E) Programming skills in Python and/or Scala for data processing (D). Experience ...

data validation frameworks Experience of testing data platforms such as Snowflake, BigQuery, Redshift, Databricks, Azure Synapse Experience of testing pipelines in things like Airflow, Azure Data Factory, AWS Glue, Kafka Data Concepts ETL/ELT, data modelling, data warehousing, data governance The role is 100% remote, however you must ...

Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience … with modern data warehousing solutions (e.g., Snowflake, BigQuery, Redshift) Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows) Strong understanding of data modelling and lakehouse architecture Databricks certifications (e.g., Databricks Certified Data Engineer) Experience with streaming data pipelines (Kafka, Structured Streaming) Experience with CI/CD and DevOps practices Knowledge ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

warehousing concepts Exposure to data transformation tools such as DBT is beneficial Basic knowledge of Python, ideally in the context of orchestration tools like Airflow Confident communication skills and the ability to work effectively in a remote team A proactive approach with the ability to take ownership and solve ...

production-grade data pipelines Solid understanding of integration patterns including REST APIs, webhooks, OAuth and event-driven architectures Experience with orchestration tools such as Airflow, Prefect or Dagster Familiarity with cloud environments (AWS, GCP or Azure) Experience with containerisation tools such as Docker and Kubernetes Proven experience integrating multiple ...

combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud TLNT1 ...

combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud JBLE1 ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

demonstrable performance metrics. Preferred Bachelor's degree in Computer Science or Software Engineering. Experience deploying AI within AWS and MS Azure. Experience using Docker, Airflow and Openshift. Cloud Certification. Data science model review, code refactoring, model optimization, containerisation, deployment, versioning, monitoring of model quality and non-functional requirements. ...

environments and know how to bring out the best in people. Technically strong: You're comfortable with modern data stacks like Databricks, Spark, Airflow, SQL and Python - and you can go deep when needed. Cloud-savvy: You understand distributed systems and cloud platforms, and how to make them work ...