Proven experience as a Data Architect or Lead Data Engineer in AWS environments* Deep understanding of cloud-native data services: S3, Redshift, Glue, Athena, EMR, Kinesis, Lambda* Strong hands-on expertise in data modelling, distributed systems, and pipeline orchestration (Airflow, Step Functions)* Background in energy, trading, or financial markets is a strong plus* Excellent knowledge of Python, SQL, and More ❯
Milton Keynes, Buckinghamshire, South East, United Kingdom
Upbeat Ideas UK Ltd
and ensure consistency across various data sources. Document test results and provide detailed reports on data quality findings. Required Skills and Experience: Proven experience with AWS services such as EMR, Lambda, Redshift, Firehose, S3, Iceberg, Athena, and DynamoDB. Strong understanding of data ingestion, parsing, aggregation, and schema validation processes. Proficiency in SQL for data querying and validation. Experience with More ❯
London, South East, England, United Kingdom Hybrid / WFH Options
Robert Half
will have strong Python development skills (must be able to design and write clean, maintainable, and testable code). Extensive AWS expertise, particularly across: Lambda Glue Glue Data Catalog EMR services API Gateway Relational database experience with Aurora Postgres (including query performance tuning). Spark experience, including pipelines using Spark on data stored in iceberg table format in S3 More ❯
London, South East, England, United Kingdom Hybrid / WFH Options
Robert Half
and fine-tune query performance on Aurora Postgres and other relational databases. Architect and manage data solutions on AWS using serverless technologies such as Lambda, Glue, Glue Data Catalog, EMR serverless, and API Gateway. Implement and manage large-scale data processing with Spark (Iceberg tables in S3, Gold layer in Aurora Postgres). Collaborate with data scientists, analysts, and … extensible, and testable code. Proven experience with relational databases (Aurora Postgres preferred), including performance optimisation. Extensive AWS experience, particularly with serverless data engineering tools (Lambda, Glue, Glue Data Catalog, EMR serverless, API Gateway, S3). Solid Spark experience with large-scale data pipelines and data lakehouse architectures (Iceberg format a plus). Hands-on experience with data modelling and More ❯
