and customise them for different use cases. Develop data models and Data Lake designs around stated use cases to capture KPIs and data transformations. Identify relevant AWS services - on Amazon EMR, Redshift, Athena, Glue, Lambda, to design an architecture that can support client workloads/use-cases; evaluate pros/cons among the identified options to arrive at More ❯
Collaborate with development teams to design and implement automated tests for microservices, emphasizing Spring Boot and Java-based architectures. Implement testing strategies for AWS data lakes (e.g., S3, Glue, Athena) with a focus on schema evolution, data quality rules, and performance benchmarks, prioritizing data lake testing over traditional SQL approaches. Automate data tests within CI/CD workflows to … maintain scalable test automation frameworks, with a focus on backend, API, and data systems using tools like Pytest and Postman. Expertise in Pandas, SQL, and AWS analytics services (Glue, Athena, Redshift) for data profiling, transformation, and validation within data lakes. Solid experience with AWS (S3, Lambda, EMR, ECS/EKS, CloudFormation/Terraform) and understanding of cloud-native architectures More ❯
Collaborate with development teams to design and implement automated tests for microservices, emphasizing Spring Boot and Java-based architectures. Implement testing strategies for AWS data lakes (e.g., S3, Glue, Athena) with a focus on schema evolution, data quality rules, and performance benchmarks, prioritizing data lake testing over traditional SQL approaches. Automate data tests within CI/CD workflows to … maintain scalable test automation frameworks, with a focus on backend, API, and data systems using tools like Pytest and Postman. Expertise in Pandas, SQL, and AWS analytics services (Glue, Athena, Redshift) for data profiling, transformation, and validation within data lakes. Solid experience with AWS (S3, Lambda, EMR, ECS/EKS, CloudFormation/Terraform) and understanding of cloud-native architectures More ❯
ll work across engineering, analytics, and AI functions to ensure data is accessible, reliable, and actionable. ️ Key Responsibilities Build, maintain, and optimise batch and streaming pipelines using AWS Glue, Athena, Redshift, and S3. Use prompt engineering techniques (training provided) to support LLM-based automation and testing. Collaborate with platform teams to embed GenAI tools (e.g., Cursor, Gemini, Claude) into … of experience in data or analytics engineering. Proficiency in Python and SQL, with strong debugging and performance tuning skills. Experience building pipelines with AWS services such as Glue, S3, Athena, Redshift, and Lambda. Familiarity with orchestration tools (e.g., Airflow, Step Functions) and DevOps practices (e.g., CI/CD, Infrastructure as Code). Interest in Generative AI and a willingness More ❯
Glue, etc.) to automate critical data workflows Partner with internal stakeholders-from Product to Leadership-to turn raw data into actionable insights Manage and evolve our AWS infrastructure (S3, Athena, Lambda, RDS/Postgres) Contribute to frontend data tools (React, TypeScript) when needed Build with quality in mind-ensuring data integrity, observability, and scalability across systems Continuously improve engineering … Skills: Strong Python experience for building data pipelines and APIs. 5+ years experience in data engineering and Node.js experience (APIs, backend services, orchestration) Deep AWS experience (S3, Lambda, Glue, Athena, IAM, CloudWatch) Strong with relational databases (Postgres, MySQL, Aurora RDS)-schema design, query tuning, modelling Familiar with common data formats: CSV, JSON, Avro Strong grasp of CI/CD More ❯
GDPR and industry regulations. What We’re Looking For 4+ years of experience in Python development (Flask, Django). Strong expertise in AWS cloud environments (EC2, S3, RDS, Redshift, Athena). Experience with SQL databases (PostgreSQL, SQLAlchemy). Knowledge of ETL pipelines, API development, and front-end technologies (HTML, CSS). Experience with Infrastructure as Code tools (Terraform, Pulumi More ❯
Employment Type: Permanent
Salary: £55000 - £75000/annum + Hybrid Working & Benefits
Leeds, West Yorkshire, United Kingdom Hybrid / WFH Options
Tenth Revolution Group
architectures. You'll collaborate with cross-functional teams to deliver high-impact solutions that drive business value. Requirements: Experience as a Data Engineer using AWS: Glue, Lambda, S3, Redshift, Athena, EMR. Snowflake experience (Not essential) Strong Python SQL & NoSQL databases CI/CD tools : GitHub Actions or CodePipeline Infrastructure as Code (Terraform is preferable) Experience building ETL/ELT More ❯
Data Engineer | Bristol/Hybrid | £65,000 - £80,000 | AWS | Snowflake | Glue | Redshift | Athena | S3 | Lambda | Pyspark | Python | SQL | Kafka | Amazon Web Services | Do you want to work on projects that actually help people? Or maybe you want to work on a modern AWS stack I am currently supporting a brilliant company in Bristol who build software which … of standout benefits.Would you be interested in finding out more? If so apply to the role or send your CV to Sponsorship isn’t available. AWS | Snowflake | Glue | Redshift | Athena | S3 | Lambda | Pyspark | Python | SQL | Kafka | Amazon Web Services More ❯
Market Harborough, Leicestershire, East Midlands, United Kingdom
CDS Global
experience or a related educational qualification Excellent communication skills to collaborate with cross-functional teams and translate business requirements into technical solutions. A very strong command of SQL, both Athena and MSSQL A good understanding of methods for data manipulation, analysis, and automation tasks. Experience with Microsoft SSIS ETL pipelines. Experience with AWS tools such as S3, Glue, Redshift … Lambda, and Athena for building scalable data pipelines and architectures. Experience in developing dashboards and reports using MicroStrategy and\or Amazon QuickSight or similar. Strong analytical skills to troubleshoot data issues and derive actionable insights from complex datasets. Skilled in designing data models and implementing ETL workflows to ensure data quality and readiness for analysis. Ability to create More ❯
Nottingham, Nottinghamshire, East Midlands, United Kingdom Hybrid / WFH Options
Experian Ltd
within the metrics framework to analyze security reporting within PowerBi Environment. Interpret data in SQL, write queries, understand stored procedures. Familiarity with AWS Environments for data storage in S3, Athena, RDS SWL Confer with systems analysts, engineers, and executive partners to update reporting and maintain PowerBi Automation Routines, Templates and internal knowledgebases. Ensure PowerBI and Power Automate Report Development … suitable for users to filter and navigate through. Qualifications Knowledge of MS Data Fabric, Power Automate, MS Forms, PowerBI, MySQL and AWS Environments for data storage in S3 i.e. Athena, RDS SWL An understanding of systems architecture and designing Power Platform Applications and reporting solutions applying business rules and data validations. Experience building both standalone and integrated apps (e.g. More ❯
Associates, Mid-level Engineers, and Senior/Principal engineers. What You'll Do: Design, build, and scale data pipelines and services using AWS Glue, Lambda, Step Functions, S3, EMR, Athena, and more. Lead projects involving serverless, event-driven architectures and CI/CD workflows (GitLab CI). Write clean, production-grade code in Python (Scala is a bonus). … using Datadog or CloudWatch. Actively mentor Data Engineers and Associates, and lead technical discussions and design sessions. Key requirements: Must-Have: Strong experience with AWS services: Glue, Lambda, S3, Athena, Step Functions, EventBridge, EMR, EKS, RDS, Redshift, DynamoDB. Strong Python development skills. Proficient with Docker , containerization, and virtualization. Hands-on experience with CI/CD , especially GitLab CI. Solid More ❯
Crewe, Cheshire, United Kingdom Hybrid / WFH Options
Manchester Digital
AWS certified as SysOps Administrator/Solutions Architect/DevOps Engineer. - Understanding of SEIM setup and administration, e.g. Arctic Wolf - Database architectures, both relational and NoSQL, e.g., RDS, DynamoDB, Athena, Cassandra - Experience with virtualisation platforms such as VMWare, Hyper V - Storage and Data availability systems, e.g., HP Nimble, NetApp. - Networking and Domain Communication, e.g., LDAP, DNS , VPN, Firewalls. - Configuration More ❯
have: Experience Designing and Maintaining Data warehouse in big data solutions on Cloud and on Premises. Experience and knowledge of cloud platforms such as Snowflake, AWS (S3, Glue, EC2, Athena), Databricks and DBT . This includes hands-on knowledge of cloud-based data storage, processing, and analytics solutions. Good understanding of business processes. Strong analytical skills and techniques. Expertise More ❯
have: Experience Designing and Maintaining Data warehouse in big data solutions on Cloud and on Premises. Experience and knowledge of cloud platforms such as Snowflake, AWS (S3, Glue, EC2, Athena), Databricks and DBT . This includes hands-on knowledge of cloud-based data storage, processing, and analytics solutions. Good understanding of business processes. Strong analytical skills and techniques. Expertise More ❯
York, North Yorkshire, Yorkshire, United Kingdom Hybrid / WFH Options
WRK DIGITAL LTD
Proficiency in Terraform, GitHub, and VS Code for pipeline deployment. Proven leadership in developing high-performing data teams. Strong SQL and familiarity with analytics tools such as Alteryx, AWS Athena, Quicksight, Sagemaker, Tableau, and Power BI. Excellent communication and stakeholder management skills. Ability to write clear technical documentation and define user stories with appropriate acceptance criteria. Desirable: Experience in More ❯
Proficiency in Terraform, GitHub, and VS Code for pipeline deployment. Proven leadership in developing high-performing data teams. Strong SQL and familiarity with analytics tools such as Alteryx, AWS Athena, Quicksight, Sagemaker, Tableau, and Power BI. Excellent communication and stakeholder management skills. Ability to write clear technical documentation and define user stories with appropriate acceptance criteria. Desirable: Experience in More ❯
Proficiency in Terraform, GitHub, and VS Code for pipeline deployment. Proven leadership in developing high-performing data teams. Strong SQL and familiarity with analytics tools such as Alteryx, AWS Athena, Quicksight, Sagemaker, Tableau, and Power BI. Excellent communication and stakeholder management skills. Ability to write clear technical documentation and define user stories with appropriate acceptance criteria. Desirable: Experience in More ❯