18 of 18 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

/Hybrid Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

technical leadership and mentoring within Agile delivery teams. Key Responsibilities: Design and develop data pipelines using PySpark, Python, SQL, Informatica , and AWS services including EMR, S3, and Lambda . Lead the migration of legacy ETL and reporting solutions to modern cloud-based platforms. Support reporting and MI requirements, including ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Remote (occasional expensed travel to Leeds/Newcastle) Clearance: Active SC Clearance required Rate: Up to £425/day (Inside IR35) Tech Stack: AWS (EMR, S3, Lambda), PySpark, Python, SQL, CI/CD Focus: Building scalable pipelines, modernising ETL, supporting MI/Power BI Level: Mid to Senior - mentoring ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

takes pride in seeing solutions through to production. Essential experience: AWS data stack ownership in a production environment (S3, Glue, Kinesis, Redshift, EMR) Strong data modelling background (dimensional modelling, CDM, LDM, star schema) ETL/ELT pipeline design and delivery across real-time and batch Data governance, security ...

standard office hours. Key Responsibilities: Design, develop, and maintain ETL/ELT pipelines using AWS services such as AWS Glue, Lambda, Step Functions, and EMR Build and manage data workflows orchestrated with tools like AWS Step Functions, or similar Implement data transformations using Python, PySpark, or SQL Design … Requirements: Must possess an active Security Clearance Experience in data engineering or ETL development Strong hands-on experience with AWS services (Glue, S3, Lambda, EMR, IAM) Proficiency in Python and SQL Familiarity with CI/CD pipelines and Infrastructure as Code Knowledge of data security, encryption, and IAM best ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

governance frameworks - Data modeling and schema design for analytics workloads - Modern data platform design and implementation - ETL/ELT pipeline development and optimization (Glue, EMR) - Data warehousing solutions (Redshift) - Data catalog management and metadata governance PREFERRED QUALIFICATIONS - Experience in IT implementation - Data quality and lineage frameworks - Data security, compliance … access controls - Performance optimization for data processing workloads - Big data processing platforms (EMR, Glue) Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion ...

governance frameworks - Data modeling and schema design for analytics workloads - Modern data platform design and implementation - ETL/ELT pipeline development and optimization (Glue, EMR) - Data warehousing solutions (Redshift) - Data catalog management and metadata governance - Data quality and lineage frameworks PREFERRED QUALIFICATIONS - Experience and technical expertise (design and implementation … data center or data center to cloud infrastructure - Data security, compliance, and access controls - Performance optimization for data processing workloads - Big data processing platforms (EMR, Glue) Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make ...

causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats to ensure integrity and correctness of new functionality Support CI/CD practices using Git-based workflows … slowly changing dimensions Ability to generate realistic test data for various scenarios Additional Knowledge Understanding of distributed data platforms and big data environments (e.g., EMR) Knowledge of encryption principles (client-side and server-side) Awareness of production system monitoring and incident analysis Soft Skills Strong analytical and problem-solving ...

want to own driving the adoption of Amazon Web Services (AWS) in UKIs largest and most innovative companies? Want to be part of a cross-functional team focused on increasing technology adoption of cloud computing by engaging directly with C-level executives and influencers at all levels … Data & Analytics, you will drive customer adoption of Amazon Web Services (AWS) Data and Analytics services (i.e. Redshift, Search (ElasticSearch, CloudSearch), EMR, Athena, Kinesis, Glue,, and other Data Services) across UK and Ireland. Customer applications that are built to operate at scale need services with high throughput ...

want to own driving the adoption of Amazon Web Services (AWS) in UKI's largest and most innovative companies? Want to be part of a cross-functional team focused on increasing technology adoption of cloud computing by engaging directly with C-level executives and influencers at all levels … Read all the information about this opportunity carefully, then use the application button below to send your CV and application. Redshift, Search (ElasticSearch, CloudSearch), EMR, Athena, Kinesis, Glue,, and other Data Services) across UK and Ireland. Customer applications that are built to operate at scale need services with high ...

want to own driving the adoption of Amazon Web Services (AWS) in UKI's largest and most innovative companies? Want to be part of a cross-functional team focused on increasing technology adoption of cloud computing by engaging directly with C-level executives and influencers at all levels … Read all the information about this opportunity carefully, then use the application button below to send your CV and application. Redshift, Search (ElasticSearch, CloudSearch), EMR, Athena, Kinesis, Glue,, and other Data Services) across UK and Ireland. Customer applications that are built to operate at scale need services with high ...