21 of 21 Contract Apache Airflow Jobs in England

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … highly scalable data engineering solutions that power our client next-generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with our client technical ...

Data Engineer — AWS, Python & Airflow Rate: £550 per day, outside IR35 Location: Remote Duration: Initial 6-month contract, with a further 12-month extension planned Clearance: Active SC clearance, used within the last 12 months Nationality: Sole British nationals only; dual citizenship is not permitted The opportunity … urgently looking for an experienced Data Engineer with strong AWS, Python and Apache Airflow skills to join a programme on an initial six-month contract. You'll help build and maintain reliable data pipelines, develop cloud-based data solutions and make data available for downstream analytics and programme ...

including supporting automated deployments, infrastructure management, monitoring, and scaling Coaching and mentoring junior engineers and promoting engineering best practices Understanding of data processing using Apache Spark and Apache Airflow for task orchestration Use of Python, SQL, and familiarity with PySpark Understanding of EMR and reviewing output logs ...

automated deployments, infrastructure management, monitoring, and scaling Coaching and mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark and Apache Airflow for task orchestration Use of Python, SQL, and familiarity with PySpark Understanding of EMR and reviewing output logs ...

production pipelines Strong SQL and Python, plus hands-on Git Production experience with at least one of Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow or PySpark Cloud-native orchestration and distributed data processing Infrastructure as code (e.g. Terraform) CI/CD and DevOps or DevSecOps practice ...

Python and SQL programming skills, with experience developing and maintaining data pipelines • Familiarity with ETL/ELT technologies and data streaming tools such as Airflow, Azure, SSIS, Spark or Kafka • Experience working with databases and data platforms such as PostgreSQL, SQL Server, Oracle, Snowflake or similar technologies • Knowledge ...

multiple data sources Distributed computing for data intensive work such as Spark and Databricks Cloud platforms including AWS, Azure and GCP Orchestration tools like Airflow Modern DevOps practices including version control, CI/CD, and infrastructure as code using Terraform or Kubernetes Experience building scalable backend applications Understanding ...

operational issues. Understanding of platform security, resilience, scalability and availability. Experience working within Agile and/or DevOps delivery environments . Desirable: Helm, ArgoCD, Airflow, dbt, Bash, Ansible, Packer, Elasticsearch/OpenSearch, Prometheus, Grafana, ELK or Datadog. Experience working within public sector, utilities or other regulated environments is advantageous. ...

SQL. Well-versed in software engineering and systems design principles. Hands-on experience with Azure cloud data platforms is essential. Familiarity with Spark and Apache Airflow is required. Proficient in Linux/Unix/*nix environments. Strong knowledge of ETL tools and data modelling techniques. Candidates will need ...

Data Underpinnings Vector Databases: Experience with high-dimensional data retrieval using Milvus, Weaviate, or Pinecone (essential for RAG architectures). Data Orchestration: Familiarity with Airflow, Prefect, or Dagster for managing complex data movement and model training pipelines. Search & Discovery: Advanced ElasticSearch/OpenSearch for semantic search and large-scale ...

pipelines end to end. Modern data stack Experience with modern ELT, transformation and orchestration technologies. Our environment includes technologies such as dbt, Dagster/Airflow and Snowflake, although experience with comparable technologies is equally relevant. Cloud infrastructure Experience designing and implementing secure, scalable solutions on a major cloud platform. ...

experience with BigQuery . Experience building and supporting production data pipelines. Experience developing transformation layers using dbt . Experience with orchestration tools such as Apache Airflow or Cloud Composer . Strong SQL and Python development skills. Experience designing scalable data models and data warehouse solutions. Strong stakeholder engagement … independently and taking ownership of delivery. Why Join? Be part of a growing Data Engineering team. Work with modern technologies including GCP, BigQuery, dbt, Airflow, Terraform and cloud-native services . Help shape a strategic cloud data platform. Join a business investing heavily in its data capability. Enjoy real ...

pipelines Strong programming skills in Python and SQL or similar Experience with technologies such as Spark, Databricks, AWS, Azure, GCP and/or Airflow Strong focus on data quality, testing and reliability Solid software engineering fundamentals and high engineering standards Strong communication and collaborative skills Comfortable working with ambiguity ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

colleagues Must-have skills: RabbitMQ Java and microservices MongoDB and MySQL Kubernetes Grafana and Prometheus GitLab Job scheduling tooling (eg JS7 Job Scheduler, TWS, Apache Airflow) Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects) Nice to have: Databases and data platforms, ideally Oracle ...

Contract Type: Hybrid/INSIDE IR35 Technologies: Snowflake, SQL, Python, dbt, Snowpipe, Snowpark, Streams, Tasks, Stored Procedures, Data Modelling, ETL, ELT, AWS, Azure, Kafka, Airflow, Matillion, Informatica, Talend, Fivetran, Data Quality, Performance Tuning, SnowPro. Are you a hands-on Snowflake Data Engineer looking for your next contract opportunity … concepts. Experience working in cloud environments, ideally AWS and/or Azure . Knowledge of data integration and orchestration tools such as Kafka , Airflow , Matillion , Informatica , Talend or Fivetran . Experience with data quality, optimisation and performance tuning. Excellent stakeholder management and communication skills. Desirable Snowflake certifications such ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow £600.00/day Talent International UK and it's subsidiaries, Digital Gurus, Infinite Talent and Rethink act as an employment agency for permanent recruitment ...

Build data solutions for complex Defence challenges while broadening your technical experience across varied consultancy projects. Develop your skills across modern data engineering technologies and production environments. What you will need Hands-on data engineering ...

streaming pipelines and deliver scalable Databricks solutions using Delta Live Tables, Workflows, Unity Catalog, Python, PySpark, Terraform and Databricks Asset Bundles. Key requirements: Advanced Apache Spark and Spark SQL optimisation Production Spark Structured Streaming experience Deep Databricks platform knowledge Strong Python and PySpark engineering capability Delta Live Tables, Workflows … Databricks Asset Bundles CI/CD and GitHub Actions Ability to work autonomously in a fast-moving environment Experience with Databricks Apps, Lakebase, Airflow, Azure Data Factory, monitoring platforms or financial services would be beneficial. Location: London Rate: £800–£900 per day IR35: Inside IR35 ...

Preferred Experience PostgreSQL and/or Snowflake. Modern Python web frameworks including Django, Flask or FastAPI. Event-driven architectures and streaming technologies such as Apache Kafka. Data orchestration and pipeline platforms such as Apache Airflow. Java development experience. Financial technology experience, including market data, order management, algorithmic trading ...