1 to 25 of 72 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

/Hybrid Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Code Reviews QA/Testing methodologies Strong experience (At least 3+, prefers 6+ years) with the following technologies and platforms: AWS platform: S3, EC2, EMR, EKS, Glue, Lambda, AppFlow, Cloudwatch etc. AWS certification is big plus Snowflake Data Cloud Account Administration Virtual warehouse strategies Snowflake feature implementation: Data Sharing ...

Develop lakehouse/data warehousing solutions Improve data quality, performance & scalability Partner with Data Science & Analytics teams Tech stack: AWS (Glue, Lambda, S3, Redshift, EMR) Python, SQL, PySpark Modern ETL/orchestration tools What we’re looking for: Strong AWS data engineering experience Solid Python & SQL skills Experience with ...

Enterprise Data Architect within complex environments Strong expertise across the AWS data ecosystem , including: S3, Redshift, Glue, Lambda Kinesis/streaming architectures Athena, EMR Experience delivering cloud-native data platforms at enterprise scale Strong background within Banking/Financial Services (essential) Experience working in a consultancy or client-facing ...

hands-on Snowflake experience Solid understanding of modern cloud data architectures and Lakehouse concepts Familiarity with AWS services including S3, Glue, Athena, RDS, and EMR Confident communicator able to work with both technical and non-technical stakeholders Reference Number: BBBH(phone number removed) Rise Technical Recruitment Ltd acts ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...