1 to 25 of 140 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

Java a plus). Proven experience with Apache Spark (PySpark) for large-scale processing. Hands-on experience with AWS Glue, S3, Redshift, Athena, EMR, Lake Formation. Strong debugging and performance optimization skills in distributed systems. Hands-on experience with Iceberg, Delta Lake, or other OTF table formats. Experience with ...

/Hybrid Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

technologies, S3, Athena, Redshift Experience in snowflake is an added bonus. Familiarity with the following technologies: Hadoop, Kafka, Airflow, Hive, Presto, Athena, S3, Aurora, EMR, Spark Ability to drive, contribute to, and communicate solutions to technical product challenges Ability to roll-up your sleeves and work in process ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Spark, Kafka, etc. Relational SQL and NoSQL databases and experience working with relational databases AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena Data Platforms: Databricks Data streaming systems: Batch,Kafka, Storm, Spark-Streaming, etc. Languages: Python, R, Scala, Go Ability ...

Develop lakehouse/data warehousing solutions Improve data quality, performance & scalability Partner with Data Science & Analytics teams Tech stack: AWS (Glue, Lambda, S3, Redshift, EMR) Python, SQL, PySpark Modern ETL/orchestration tools What we’re looking for: Strong AWS data engineering experience Solid Python & SQL skills Experience with ...

focus on Databricks. Proficiency in Databricks, including the ability to build and optimize Spark applications. Extensive experience with AWS services such as S3, Redshift, EMR, Lambda, Glue, and others. Strong knowledge of Unity Catalog and its application in managing and securing data assets. Strong knowledge of data modeling ...

Enterprise Data Architect within complex environments Strong expertise across the AWS data ecosystem , including: S3, Redshift, Glue, Lambda Kinesis/streaming architectures Athena, EMR Experience delivering cloud-native data platforms at enterprise scale Strong background within Banking/Financial Services (essential) Experience working in a consultancy or client-facing ...

hands-on Snowflake experience Solid understanding of modern cloud data architectures and Lakehouse concepts Familiarity with AWS services including S3, Glue, Athena, RDS, and EMR Confident communicator able to work with both technical and non-technical stakeholders Reference Number: BBBH(phone number removed) Rise Technical Recruitment Ltd acts ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

Job Description Job Description Who We Are: Oasys International LLC (Oasys) is a fast-growing federal government contractor delivering high-quality technology consulting and professional services to civilian, defense, and homeland security agencies. We have ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...

external systems, including standardized integrations with T-MSIS for compliance and Data Quality Assessment status Building data processing workflows on AWS (S3, Glue, EMR, Athena, RDS/Redshift) and, where applicable, Databricks, Spark, and Hive for big-data processing Creating data products that feed Power BI dashboards and Salesforce … Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline Strong understanding of AWS data services including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Athena, QuickSight, and Amazon SNS Experience with Spark and Hive for big-data processing Experience building job workflows ...