Hands-on experience with Big Data ecosystems Hadoop, Spark, Kafka, Hive, HBase, etc. Strong experience with Cloud platforms (AWS/Azure/GCP) and services like: AWS: S3, Glue, EMR, Redshift, Lambda, Kinesis Azure: Data Factory, Synapse, Databricks, ADLS GCP: BigQuery, Dataflow, Pub/Sub Experience with Data Warehouse/Data Lake/Lakehouse design and modeling (Kimball, OLAP More ❯
solutions. Strong hands-on experience with at least one major cloud platform (AWS, Azure, or Google Cloud Expertise in cloud-native data services such as AWS Glue, Lambda, S3, EMR, Redshift, Lake Formation, Azure Synapse, Data Factory, Databricks, or BigQuery. Advanced knowledge of SQL, Python, Spark, PySpark, and distributed data frameworks. Proven background in building ELT/ETL systems More ❯
London, South East, England, United Kingdom Hybrid/Remote Options
Involved Solutions
business decisions. Responsibilities for the AWS Data Engineer: Design, build and maintain scalable data pipelines and architectures within the AWS ecosystem Leverage services such as AWS Glue, Lambda, Redshift, EMR and S3 to support data ingestion, transformation and storage Work closely with data analysts, architects and business stakeholders to translate requirements into robust technical solutions Implement and optimise ETL More ❯
the following: Python, SQL, Java Commercial experience in client-facing projects is a plus, especially within multi-disciplinary teams Deep knowledge of database technologies: Distributed systems (e.g., Spark, Hadoop, EMR) RDBMS (e.g., SQL Server, Oracle, PostgreSQL, MySQL) NoSQL (e.g., MongoDB, Cassandra, DynamoDB, Neo4j) Solid understanding of software engineering best practices - code reviews, testing frameworks, CI/CD, and code More ❯
data pipelines. Collaborate with data scientists and analysts to ensure data quality, availability, and consistency for advanced modeling and reporting. Utilize AWS or other cloud services (e.g., S3, Glue, EMR, Snowflake) to architect and maintain cloud-based data ecosystems. Write and optimize complex SQL queries for data extraction, integrity checks, and performance tuning. Required Technical Skills 5+ years of More ❯
Manchester, Lancashire, England, United Kingdom Hybrid/Remote Options
Lorien
years in a technical leadership or management role Strong technical proficiency in data modelling, data warehousing, and distributed systems Hands-on experience with cloud data services (AWS Redshift, Glue, EMR or equivalent) Solid programming skills in Python and SQL Familiarity with DevOps practices (CI/CD, Infrastructure as Code - e.g., Terraform) Excellent communication skills with both technical and non More ❯
Skills • Min of 2 years of experience in data engineering or a similar role. • Hands-on experience with core AWS data services (for example S3, Glue, Athena, Lambda, IAM, EMR). • Strong SQL skills (joins, window functions, optimization). • Solid Python for data processing. • Experience building production ETL/ELT pipelines. • Working knowledge of security and IAM (roles, policies More ❯
seeking to hire a Data Engineering Manager to play a key role in their data operations and business intelligence initiatives. Key Responsibilities: Design & maintain AWS BI infrastructure (Redshift, S3, EMR) using Terraform and IaC best practices. Develop CI/CD pipelines (Jenkins, GitHub Actions) to automate ETL and Power BI code deployments. Manage environments (Dev, QA, UAT) and automate … data refreshes for accuracy and consistency. Oversee data pipelines and big data workflows (EMR, Spark) for high-performance analytics. Optimize code for ETL and Power BI (DAX, data models, refresh scheduling) to enhance performance. Implement observability and logging (CloudWatch, Grafana, ELK) for proactive system monitoring. Collaborate cross-functionally with BI, Platform, and Data teams on releases and issue resolution. … Monitor performance & costs in AWS, driving optimisation and efficiency. Champion automation & innovation through new tools, frameworks, and cloud-native solutions. Key Skills: AWS Cloud: Expert in Redshift, S3, Lambda, EMR, and IaC (Terraform/CloudFormation); strong understanding of big data architecture and performance optimisation. CI/CD & Automation: Skilled in Jenkins, GitHub Actions, and Python scripting for automated ETL More ❯
/cloud architecture, operating systems (e.g., Linux), and storage systems (e.g., AWS, Databricks, Cloudera) Production experience in core data technologies (e.g. Spark, HDFS, Snowflake, Databricks, Redshift, & AmazonEMR) Development of APIs and web server applications (e.g. Flask, Django, Spring) Complete software development lifecycle experience, including design, documentation, implementation, testing, and deployment Excellent communication and presentation skills; previous More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data More ❯
SQL, Scala, or Java 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data More ❯
and delivering production-grade software and data systems. Proficiency in Python, Java, or Scala - comfortable writing robust, testable, and scalable code. Deep experience with AWS (Lambda, ECS/EKS, EMR, Step Functions, S3, IAM, etc.). Strong knowledge of distributed systems and streaming/data pipelines (Kafka, Spark, Delta, Airflow, etc.). Familiarity with infrastructure-as-code (Terraform, CloudFormation More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
SQL, Scala, or Java 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data More ❯
in application development including Python, SQL, Scala, PySpark 2+ years of experience with a public cloud (AWS) 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data More ❯
in application development including Python, SQL, Scala, PySpark 2+ years of experience with a public cloud (AWS) 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data More ❯
in application development including Python, SQL, Scala, PySpark 2+ years of experience with a public cloud (AWS) 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data More ❯
