AWS test engineer

Responsibilities:

• Perform end-to-end testing of data ingestion, parsing, aggregation, and schema validation processes in AWS.
• Conduct stress testing to evaluate the performance of the entire data pipeline built on AWS.
• Develop and execute test plans, test cases, and test scripts to ensure data quality and integrity.
• Collaborate with data engineers and developers to identify and resolve data quality issues.
• Monitor data quality metrics and implement improvements to enhance data reliability.
• Validate data transformations and ensure consistency across various data sources.
• Document test results and provide detailed reports on data quality findings.

Required Skills and Experience:

• Proven experience with AWS services such as EMR, Lambda, Redshift, Firehose, S3, Iceberg, Athena, and DynamoDB.
• Strong understanding of data ingestion, parsing, aggregation, and schema validation processes.
• Proficiency in SQL for data querying and validation.
• Experience with Python for scripting and automation.
• Familiarity with data quality tools and frameworks.
• Excellent analytical and problem-solving skills.
• Strong attention to detail and commitment to data accuracy.

Preferred Qualifications:

• Experience with data quality assurance in a cloud environment.
• Knowledge of data governance and best practices.
• Ability to work independently and as part of a team.
• Strong communication skills to effectively collaborate with cross-functional teams.