1 to 25 of 32 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such as S3, Glue, Lambda, EMR, IAM, KMS, CloudWatch, Secrets Manager, Athena, and related services. Implement lakehouse architecture using Databricks, Delta Lake, and AWS S3. Build data ingestion pipelines from ...

GitLab , and CI/CD pipelines. Understanding of AWS security best practices, IAM, and RBAC. Desirable Skills Experience with Athena, SQS, CloudWatch, CloudTrail, or EMR . Knowledge of Terraform or CloudFormation . Experience in consultancy or client-facing environments. Personal Attributes Strong problem-solving and troubleshooting skills. Excellent communication ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

data analysis, profiling and insight delivery Cloud & Platforms Azure: Data Factory, Synapse, Databricks, ADLS Gen2, Event Hub, Purview AWS: S3, Glue, Athena, Kinesis, Lambda, EMR, Lake Formation Databricks: Delta Lake, Unity Catalog, Structured Streaming, MLflow SQL Server stack (SSIS, SSRS) Engineering Practices CI/CD for data platforms ...

expertise with Infrastructure as Code (Terraform) Experience working with GitLab and implementing CI/CD pipelines Practical experience building data pipelines in AWS (e.g. EMR, PySpark) Solid understanding of ETL tooling (e.g. Informatica or similar) Strong background in data processing and transformation Proficiency in Python and PySpark Essential Requirements ...

experience designing and building distributed data processing systems and large-scale data platforms on AWS Strong knowledge of modern AWS data technologies including Glue, EMR, Lambda, DynamoDB, S3, EventBridge, Step Functions and related services Proven experience building Data Lakehouse platforms, Data Products and ELT/ETL frameworks supporting analytics ...

pipeline, data warehouse and data lake design/building and data movement. AWS data and analytics services (or open-source equivalent) such as EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. What you can expect Work to agile best practices and cross-functionally with multiple teams and stakeholders. ...

Remote (occasional expensed travel to Leeds/Newcastle) Clearance: Active SC Clearance required Rate: Up to £425/day (Inside IR35) Tech Stack: AWS (EMR, S3, Lambda), PySpark, Python, SQL, CI/CD Focus: Building scalable pipelines, modernising ETL, supporting MI/Power BI Level: Mid to Senior - mentoring ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

/CD tooling including Git Desirable Skills Hands-on experience with Apache NiFi for data pipeline development and orchestration AWS experience including familiarity with EMR, S3 and SQS Awareness of domain-driven design approaches and architecture Linux awareness and Scripting skills PostgreSQL and PL/SQL knowledge ...

metrics and KPIs to verify and automate management of data quality. \n Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). \n\n \n Desirable Skills \n \n \n Working experience ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality.\ NGood working experience of AWS (e.G. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.G. ADF, Synapse, Fabric, Azure Functions).\ N\n \nDesirable Skills\ N \n\nWorking experience in Data Lakehouse ...

metrics and KPIs to verify and automate management of data quality. \n Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). \n\n Desirable Skills \n \n Working experience in Data Lakehouse ...

metrics and KPIs to verify and automate management of data quality. \n Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). \n\n Desirable Skills \n \n Working experience in Data Lakehouse ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. \ NGood working experience of AWS (e.G. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.G. ADF, Synapse, Fabric, Azure Functions). \ N\nDesirable Skills\ N\nWorking experience in Data Lakehouse environments such ...

designing metrics and KPIs to verify and automate management of data quality. \ NGood working experience of AWS (e.G. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.G. ADF, Synapse, Fabric, Azure Functions). \ N\nDesirable Skills\ N\nWorking experience in Data Lakehouse environments such ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake Good understanding of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns, with demonstrated experience architecting end-to-end data solutions Experience navigating complex data ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake Good understanding of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns, with demonstrated experience architecting end-to-end data solutions Experience navigating complex data ...

enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake\ N\n\nGood understanding of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns, with demonstrated experience architecting end-to-end data solutions\ N\n\nExperience navigating ...