AWS Data Engineer
AWS Data Engineer
Contract: 6 Months
Rate: £400 per day
IR35 Status: Outside IR35
Location: Remote / Hybrid
Overview
We are seeking an experienced AWS Data Engineer to join a growing consultancy on an initial 6-month contract. You'll be responsible for designing, building, and optimising modern data platforms and pipelines, enabling analytics, reporting, and AI-driven initiatives across the business.
This role is ideal for someone with strong AWS experience and a background working with modern data tooling including dbt, PySpark, Airflow, Redshift, Iceberg, OpenSearch, Knowledge Graphs, Python, and CircleCI.
Key Responsibilities
- Design and develop scalable ETL/ELT data pipelines using Python and PySpark
- Build and maintain data transformation frameworks using dbt
- Develop and orchestrate workflows using Apache Airflow
- Deliver high-performance data warehouse solutions within Amazon Redshift
- Support the implementation and optimisation of Apache Iceberg lakehouse architectures
- Develop and enhance search capabilities using OpenSearch
- Work with Knowledge Graph technologies to improve data discovery and relationships
- Implement CI/CD best practices using CircleCI
- Collaborate with Data Analysts, Product Teams, and Business Stakeholders to deliver high-quality data solutions
- Ensure data quality, reliability, governance, and scalability across the platform
Required Experience
- Strong commercial experience as a Data Engineer within AWS
- Expertise in Python, SQL, and PySpark
- Hands-on experience with dbt
- Strong experience managing workflows with Apache Airflow
- Experience with Amazon Redshift
- Knowledge of modern data lakehouse architectures, including Apache Iceberg
- Experience with OpenSearch / Elasticsearch technologies
- Exposure to Knowledge Graphs and graph-based data modelling
- Experience implementing CI/CD pipelines using CircleCI
- Strong understanding of data modelling, performance optimisation, and engineering best practices
Desirable
- Experience with AWS services such as S3, Lambda, Glue, ECS, EKS, or EMR
- Experience supporting AI, machine learning, or advanced analytics use cases
- Infrastructure as Code experience (Terraform preferred)
- Experience working within Agile delivery environments