20 of 20 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

scale in government or similarly complex domains, including performance tuning and data‐quality management · Hands‐on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large‐scale batch processing. · CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing ...

/Hybrid Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

Code Reviews QA/Testing methodologies Strong experience (At least 3+, prefers 6+ years) with the following technologies and platforms: AWS platform: S3, EC2, EMR, EKS, Glue, Lambda, AppFlow, Cloudwatch etc. AWS certification is big plus Snowflake Data Cloud Account Administration Virtual warehouse strategies Snowflake feature implementation: Data Sharing ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Develop lakehouse/data warehousing solutions Improve data quality, performance & scalability Partner with Data Science & Analytics teams Tech stack: AWS (Glue, Lambda, S3, Redshift, EMR) Python, SQL, PySpark Modern ETL/orchestration tools What we’re looking for: Strong AWS data engineering experience Solid Python & SQL skills Experience with ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

standard office hours. Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines using AWS services such as AWS Glue, Lambda, Step Functions, and EMR Build and manage data workflows orchestrated with tools like AWS Step Functions, or similar Implement data transformations using Python, PySpark, or SQL Design … Requirements: Must possess an active Security Clearance Experience in data engineering or ETL development Strong hands-on experience with AWS services (Glue, S3, Lambda, EMR, IAM) Proficiency in Python and SQL Familiarity with CI/CD pipelines and Infrastructure as Code Knowledge of data security, encryption, and IAM best ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data lakes and Glue Data Catalog. Strong understanding … Strong hands-on expertise with: AWS DMS (Database Migration Service). AWS SCT (Schema Conversion Tool). AWS Glue (ETL & data integration). Amazon Redshift (data warehousing). Other Qualifications (OQs): Exposure to real-time replication tools (e.g., GoldenGate or equivalent). Database Administration (DBA) expertise, including backup ...

deliver end-to-end solutio nsKey Skills & Experienc e:Strong expertise in AWS cloud services, includin g:CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formati onExperience with Infrastructure as Code (Terrafor m)Proficiency in Python, SQL, and familiarity with PySpa rkExperience with Apache Spark and data processing … Apache Airflow for workflow orchestrati onExperience using Jupyter Notebooks and/or Amazon Athena for data querying and validati onStrong understanding of EMR and log analys isExperience with GitLab for CI/CD pipelines, release tagging, and version contr olKnowledge of Docker and containerisati onUnderstanding of encryption ...

Data Engineer Manchester, 2 days per week Millions of customers. One proprietary platform. Infinite possibilities. This global technology business has just completed a massive migration to their own platform, and now, the stage is set ...

Senior Data Engineer Manchester, 2 days per week Millions of customers. One proprietary platform. Infinite possibilities. This global technology business has just completed a massive migration to their own platform, and now, the stage is ...

causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats to ensure integrity and correctness of new functionality Support CI/CD practices using Git-based workflows … slowly changing dimensions Ability to generate realistic test data for various scenarios Additional Knowledge Understanding of distributed data platforms and big data environments (e.g., EMR) Knowledge of encryption principles (client-side and server-side) Awareness of production system monitoring and incident analysis Soft Skills Strong analytical and problem-solving ...

role leads all network architecture, firewall design, and connectivity solutions across clinics and head office. The role is critical in supporting NHS collaboration, DaVita EMR integrations, and domain/infrastructure migration programmes, ensuring all solutions are secure, compliant, and aligned with enterprise standards. Key Objectives Core IT Systems & Infrastructure … disruption Define and maintain network standards for existing and future sites Integration & NHS Collaboration Act as network lead for all integration projects, including DaVita EMR connectivity Work closely with NHS IT teams to implement secure connectivity (e.g. HSCN, VPNs) Lead domain migration and infrastructure integration activities Ensure compliance with ...