Test Engineer

About Scrumconnect

Scrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure technology solutions for public and private sector clients. This is a fully remote role based in India, supporting our UK operations and client base. You will work closely with our UK leadership team and must be comfortable operating within UK regulatory and legal frameworks.

About the role

Test Engineer on a large-scale cloud data engineering programme, operating across a modern AWS-native technology stack including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, and DynamoDB. You will own testing across automated pipelines, data workflows, and cloud infrastructure - identifying risks, championing test frameworks, and coaching colleagues in quality engineering practices.

This is a hands-on, technically deep role. You will write code, build and extend test frameworks, and take accountability for test quality across releases - while actively mentoring more junior team members in best practices.

Key responsibilities

Automated testing

Design, implement, and maintain automated test suites across data pipelines, cloud services, and application layers. Ensure reproducibility and reliability across all releases.

Risk identification

Identify and raise awareness of risks arising from automated test results. Translate technical test findings into clear, prioritised risk communication for the team and stakeholders.

Framework development

Research, evaluate, and implement new test frameworks suited to the AWS data stack. Continuously improve existing frameworks to increase coverage and confidence in test outcomes.

Pipeline & data testing

Validate data pipelines built on Apache Airflow, AWS Glue, Athena, and EMR - including data integrity, transformation accuracy, and performance under load.

Mentoring & coaching

Mentor and advise team members in testing practices. Share knowledge actively, contribute to a quality-first culture, and support less experienced colleagues in developing test skills.

CI/CD integration

Integrate and manage tests within GitLab CI/CD pipelines. Ensure tests run automatically on commit, providing fast feedback loops and blocking regressions before deployment.

Technology stack

Apache AirflowAmazon AthenaAWS S3AWS GlueAWS EMRAWS EC2AWS ECRAWS DynamoDBAWS CloudWatchAWS IAM PythonJavaSQLBash GitLab CI/CD Jupyter Notebooks

Skills and experience required

Demonstrable experience as a Test Engineer or QA Engineer working on data pipelines or cloud-native environments
Strong Python proficiency - used for test Scripting, data validation, and automation in a production context
Working knowledge of Java and SQL for test development and data querying
Scripting experience in Bash or equivalent for pipeline and environment automation
Hands-on experience with the AWS ecosystem - particularly S3, Glue, Athena, EMR, CloudWatch, IAM, and DynamoDB
Experience configuring and running tests within GitLab CI/CD pipelines
Proven ability to design and implement automated test frameworks from scratch or extend existing ones
Experience testing data pipelines - including data quality, integrity, transformation logic, and performance
Ability to identify, articulate, and communicate testing risks clearly to both technical and non-technical audiences
SFIA Level 4 capability - works autonomously, takes ownership of test quality, and influences others
Experience mentoring or coaching junior engineers in testing practices
Active SC clearance - mandatory at point of application, no exceptions

Diversity and Inclusion

At Scrumconnect Consulting, we believe that diversity drives innovation. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds and experiences, and we actively encourage applications from women, people with disabilities, under-represented communities, and those seeking flexible working arrangements.

About the role

Key responsibilities