native distributed PyTorch/Tensorflow, Spark-based distributor libraries, or Horovod.TECH STACK:Python, Flask, Redis, Postgres, React, Plotly, Docker. Temporal; AWS Athena SQL, Athena & EMR Spark, ECS Fargate; Azure Synapse/Data Lake Analytics, HDInsight.KEY RESPONSIBILITIESLead the productionisation of Monolith’s ML models and data processing pipelines, both mid more »
South East London, England, United Kingdom Hybrid / WFH Options
Durlston Partners
a massively distributed cloud-hosted data platform that would be used by the entire firm.Tech Stack: Python/Java, Dremio, Snowflake, Iceberg, PySpark, Glue, EMR, dbt and Airflow/Dagster.Cloud: AWS, Lambdas, ECS servicesThis role would focus on various areas of Data Engineering including:End to End ETL pipeline … quality tooling.Requirements:Strong Python/Java Software Engineering skillsExcellent AWS knowledge, ideally with exposure to Airflow, Glue, Iceberg and SnowflakePrevious experience with Dremio, dbt, EMR or DagsterGood Computer Science fundamentals knowledge with strong knowledge of software and data architecture.If you would like more information on the above, please apply more »
with a strong emphasis on Java and EMR.Implement and optimise data models and algorithms to extract valuable insights from large datasets.Utilise AWS services, including EMR, Glue, and Iceberg, for data storage, processing, and ingestion.Ensure data quality and integrity by implementing robust validation and monitoring processes.Collaborate with cross-functional teams … data engineering, with a focus on managing and processing financial data.Proficiency in Python and Java, with a strong preference for Java, and experience with EMR for data processing.Solid understanding of data modelling concepts and experience with relevant modelling languages.Hands-on experience with AWS services, including but not limited to … EMR, Glue, and S3.Prior experience with Iceberg for data storage and Glue for data ingestion is highly desirable.Strong SQL skills and experience with relational databases.Proven track record of delivering high-quality data solutions in a fast-paced environment.Excellent communication and collaboration skills, with the ability to work effectively in more »
London, England, United Kingdom Hybrid / WFH Options
fierlo
role, you'll spearhead backend and data engineering and mentor team members. Tech stack: Athena; Python, Flask, Redis, Postgres, React, Plotly, Docker, SQL, Athena & EMR Spark, ECS and Temporal. This is a 60/40 split between tech and leadership. Your background: 8 years+ coding experience, 4+ years Python more »
specialise on using the latest frameworks, reference architectures and technologies using AWS, Azure and GCP. Essential Skills and Experience: • AWS (e.g., Athena, Redshift, Glue, EMR) • Strong AWS Data Solution Architect Experience on Data Related Projects • Java, Scala, Python, Spark, SQL • Experience of developing enterprise grade ETL/ELT data … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). • Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. • NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. • BigQuery and Data Studio/Looker. more »
Appomattox, Virginia, United States Hybrid / WFH Options
Maxar Technologies
and the Joint Enterprise Modeling and Analytics (JEMA) framework or Apache NiFi. Experience in AWS services including but not limited to EC2, Sagemaker, S3, EMR, and IAM. Our salary ranges are market-driven and set to allow for flexibility. Individual pay will be competitive based on a candidate's more »
and NoSQL databases, particularly Postgres . · Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. · Experience with AWS cloud services: EC2, EMR, RDS, Redshift. · Experience with multiple data architecture paradigms (relational, non-structured, streaming) · Knowledge of various data communication protocols (Rest API, GraphQL, RPCs, MQTT, AMQP more »
strong emphasis on Java and EMR. Implement and optimise data models and algorithms to extract valuable insights from large datasets. Utilise AWS services, including EMR, Glue, and Iceberg, for data storage, processing, and ingestion. Ensure data quality and integrity by implementing robust validation and monitoring processes. Collaborate with cross … engineering, with a focus on managing and processing financial data. Proficiency in Python and Java, with a strong preference for Java, and experience with EMR for data processing. Solid understanding of data modelling concepts and experience with relevant modelling languages. Hands-on experience with AWS services, including but not … limited to EMR, Glue, and S3. Prior experience with Iceberg for data storage and Glue for data ingestion is highly desirable. Strong SQL skills and experience with relational databases. Proven track record of delivering high-quality data solutions in a fast-paced environment. Excellent communication and collaboration skills, with more »
Senior Data Engineer Hybrid - London £70,000 - £80,000 We're currently partnered with a leading technology-driven company specialising in digital solutions that enhance user experience across various platforms. With over 15,000 5-star reviews and partnerships with more »
estate of cloud native software products and services. The DevOps Engineer will have deep knowledge of the AWS data engineering stack: Kinesis, Redshift, Glue, EMR and be passionate about writing high-quality code that not only solves problems but improves the overall estate. The role will involve operating and … Required DevOps Engineer experience. AWS skills. Docker, Kubernetes skills. knowledge across the AWS data engineering stack including but not limited to: Kinesis, Redshift, Glue, EMR, Databricks. of the software development lifecycle and the role DevOps plays within it. working in autonomous agile teams and environments. to degree level or more »
AWS CLI and IAM etc. (required) Experience with distributed message brokers using Kafka (required) Experience with high speed distributed computing frameworks such as AWS EMR, Hadoop, HDFS, S3, MapReduce, Apache Spark, Apache Hive, Kafka Streams, Apache Flink etc. (required) Working knowledge of DevOps tools. Eg Terraform, Ansible, Jenkins, Kubernetes more »
AWS CLI and IAM etc. (required) Experience with distributed message brokers using Kafka (required) Experience with high speed distributed computing frameworks such as AWS EMR, Hadoop, HDFS, S3, MapReduce, Apache Spark, Apache Hive, Kafka Streams, Apache Flink etc. (required) Working knowledge of DevOps tools. Eg Terraform, Ansible, Jenkins, Kubernetes more »
Apache Iceberg & Spark. Exposure to Apache Airflow, Prefect, Dagster, DBT. Expertise in data analysis with exposure to data services (such as Glue, Lake Formation, EMR, EventBridge, Athena, etc.) & metadata management tools (such as Amundsen, Atlas, DataHub, OpenDataDiscovery, Marquez, etc.), Experience on RDBMS like PostgreSQL would be a plus. Experience more »
wrangling, visualization, and reporting. Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. Capable of processing large volumes of structured and unstructured data on AWS. Familiarity with AWS best practices in data more »
boundaries through multiple data centers and AWS regions. Your Profile Key skills/knowledge/experience: Experience of AWS Data platform experience including AWS-EMR Cluster Platform, Glue, EC2, S3, VPC, IAM, RDS, Redshift, Sagemaker, Lake formation and Glue catalogue. Data Storage Fundamentals. Cloud specific patterns and technologies. AWS … DevSecOps pipelines. Experience in data security areas in AWS like IAM, Cloudwatch, Cloudtrail, Inspector. Hands on experience in architecture of the data workloads in EMR\Clusters. Designs system architecture to integrate easily with other AWS services. Strong background in technologies like Spark, Hive and Pyspark. Key Experience: Experience of more »
the adoption of best practices (agile methodologies, continuous improvement) in software engineering and product development. Design and lead software integration with external vendors (PACS, EMR) and business partners. Manage translation, user interface, and user documentation, adhering to human factors engineering principles and usability testing. Collaborate with leadership to align … performing product development teams across the entire lifecycle. Experience managing software development, UI/UX/workflow design, testing, SaaS security, and PACS/EMR integration. Worked with Cloud based SaaS (ideally AWS) Ideally some familiarity with Ai/ML for diagnosis Benefits Competitive salary, with share options and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
data processing. Technologies We Use: Development languages/frameworks : Java/Scala, Apache Spark, Kafka, Vertica, JavaScript (React/Redux), MicroStrategy Amazon : EMR, Step Functions, SQS, LaMDA and AWS cloud-native architectures DevOps Tools : Terraform or Cloud Formation, NewRelic, Jenkins, Grafana, PagerDuty, GitHub, GitHub Actions Database : MySQL … or Scala skills and UI skills Experience with designing and implementing high volume data processing jobs is a required. Working knowledge of Spark on EMR is a preferred. Strong database development skills, including advanced SQL, relational and NoSQL database technologies. Experience with AWS technologies are required. Strong analytical and more »
