20 of 20 Contract Apache Airflow Jobs

pipelines Proficiency with at least one cloud platform: Amazon Web Services (AWS) Microsoft Azure (Azure) Google Cloud Platform (GCP) Experience with orchestration tools (e.g., Apache Airflow) Experience with version control (e.g., Git) Solid understanding of data warehousing concepts and dimensional modeling To apply for this role please submit ...

Data Engineer: Experience with Databricks , Kafka , or Kinesis for real-time data streaming Knowledge of containerisation (Docker, ECS) and modern orchestration tools such as Airflow Familiarity with machine learning model deployment pipelines or data lakehouse architectures Data Engineer, AWS Data Engineer ...

Skills & Experience Proven experience as a Data Engineer , BI Engineer , or similar role. Strong proficiency in SQL , Python , and modern data engineering frameworks (e.g., Airflow, dbt, Spark, etc.). Experience with cloud platforms such as AWS, Azure, or GCP. Solid understanding of data warehousing and ETL/ELT architecture. ...

PostgreSQL, MySQL) Strong knowledge of data warehouse, data lake, and lakehouse architectures Experience with ETL/ELT frameworks and orchestration tools (e.g. Informatica, Talend, Apache Airflow) Proven experience establishing and maintaining data governance frameworks Strong understanding of data security, access controls, and compliance requirements Excellent problem-solving, communication ...

Oracle, SQL Server, PostgreSQL, MySQL). Knowledge of data warehouse and lakehouse architectures. Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow). Experience establishing and maintaining data governance frameworks. Experience complying with data security requirements. Experience designing data models. Excellent problem-solving, communication ...

data structures, performance, and optimisation strategies Familiarity with GCP and ecosystem knowledge: BigQuery, Composer, Dataproc, Cloud Run, Dataplex Hands-on experience with orchestration (like Airflow, Dagster, Databricks Workflows) and distributed processing in a cloud environment Experience with analytical data modelling (star and snowflake schemas), DWH, ETL/ELT patterns ...

tasks, working with the Snowflake Data Marketplace, Performance Tuning and Knowledge of ETL/ELT pipelines and data integration tools (e.g., Informatica, Matillion, DBT, Airflow). Experience with APIs, stored procedures, and automation scripts is a plus. Strong analytical and problem-solving mindset. Clear communicator, able to work across ...

DataOps practices, including CI/CD, containerisation, and workflow orchestration Hands-on experience with ETL/ELT frameworks and big data tools (e.g. Spark, Airflow, Hive) Proficiency in programming languages such as Python, Java, and SQL Experience with SQL and NoSQL database design and optimisation Strong understanding of batch ...

with SQL (including Snowflake and/or Oracle) Solid Python skills for data engineering and automation Experience with modern data tooling such as dbt, Airflow, ADF, Matillion or similar Hands-on Power BI experience including semantic models, DAX and deployment pipelines Proven experience running production data platforms, including monitoring ...

cloud data platforms (AWS, Azure or GCP) Exposure to big data technologies (Hadoop, Spark, Kafka) Familiarity with ETL/orchestration tools (e.g. Informatica, Talend, Airflow) Experience with API integration and microservices data flows Familiarity with AI/ML data pipelines or analytics platforms Experience using UML/SysML Security ...

Oracle, SQL Server, PostgreSQL, MySQL). Knowledge of data warehouse and lakehouse architectures Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow). Experience in establishing and maintaining data governance frameworks Experience complying with data security requirements. Experience designing data models. Excellent problem-solving ...

expectations of non-technical stakeholders. Good to Have (As Applicable) Familiar with AWS via hands on experience or certification Familiarity with orchestration tools like Airflow Familiarity with BASEL regulatory reporting framework Familiarity with engineering in a regulatory controlled environmen ...

Technical expertise in designing, building, and maintaining data pipelines and warehouses Experience with ETL/ELT frameworks and Big Data Processing Tools (e.g., Spark, Airflow, Hive) Knowledge of programming languages (e.g., Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in a STEM field; a master ...

Proven Data Engineering experience in operational or live service environments Strong competency with ETL/ELT tools, such as: Azure Data Factory Informatica SSIS Airflow Strong SQL proficiency Experience using monitoring tools, including: Application Insights Splunk Grafana CloudWatch Experience with scheduling systems such as Cron, Autosys, or Control ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

deliver reliable, high-performance data solutions using Databricks and modern cloud technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake Build and optimize ETL/ELT workflows for batch and streaming data Develop data models to support analytics, reporting, and machine … data pipelines, architectures, and best practices Required Qualifications 3+ years of experience in data engineering or a similar role Strong experience with Databricks and Apache Spark Proficiency in Python and SQL Experience with Delta Lake, data modeling, and performance tuning Hands-on experience with cloud platforms (AWS, Azure ...

Gateway, and Kinesis. Integrating third-party APIs into the data platform and transforming data for CRM delivery. Migrating R-based data streams into modern Airflow-managed Python/DBT pipelines. Ensuring observability and reliability using CloudWatch and automated monitoring. Supporting both BAU and new feature development within the data … services including Lambda, API Gateway, S3, Kinesis, and CloudWatch. Strong programming ability in Python and data transformation skills using SQL and DBT. Experience with Airflow for orchestration and scheduling. Familiarity with third-party API integration and scalable data delivery methods. Excellent communication and the ability to work ...

training, validation, deployment, and monitoring of models. Support scalable ML solutions across use cases like recommendations, forecasting, and automation. Work with tools like Airflow, Azure ML, and FastAPI to deliver production-ready systems. Automate model deployment workflows using CI/CD tools (e.g. GitHub Actions, Azure DevOps). Contribute … code (Terraform). Collaborative team player with excellent communication skills. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: MLflow, Azure ML, Airflow, Docker, Fargate, FastAPI Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor Desirable skills/ ...

Skills/Knowledge/Experience (Up to 10, Avoid repetition) Familiar with any Cloud environment and knowledge on Kubernetes Familiarity with orchestration tools like Airflow Familiarity with engineering in a regulatory controlled environment Good to Have (As Applicable) Familiar with AWS via hands on experience or certification Familiarity with … orchestration tools like Airflow Familiarity with BASEL regulatory reporting framework Familiarity with engineering in a regulatory controlled environment Managing expectations of non-technical stakeholders ...

than headcount substitution, aligning with Outside IR35 status . Key responsibilities include: Designing, building, and delivering data pipelines and orchestration workflows using Prefect or Airflow Developing Python-based services and APIs for data ingestion and enrichment Building and deploying containerised workloads using Docker and Kubernetes (or Swarm) Managing dependencies … UKSV DV clearance (minimum 12 months remaining) Strong Python development experience in secure or restricted environments Proven delivery of data pipelines using Prefect or Airflow Hands-on experience with Docker, Kubernetes , and container orchestration Experience operating in air-gapped systems , managing builds without external connectivity Strong understanding of data ...