1 to 25 of 27 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Code Reviews QA/Testing methodologies Strong experience (At least 3+, prefers 6+ years) with the following technologies and platforms: AWS platform: S3, EC2, EMR, EKS, Glue, Lambda, AppFlow, Cloudwatch etc. AWS certification is big plus Snowflake Data Cloud Account Administration Virtual warehouse strategies Snowflake feature implementation: Data Sharing ...

Enterprise Data Architect within complex environments Strong expertise across the AWS data ecosystem , including: S3, Redshift, Glue, Lambda Kinesis/streaming architectures Athena, EMR Experience delivering cloud-native data platforms at enterprise scale Strong background within Banking/Financial Services (essential) Experience working in a consultancy or client-facing ...

Glue). Extensive experience working on AWS data stack i.e., Data engineering, Warehousing and AI/ML. Advocate best practices with AWS including Glue, EMR, Airflow and etc., Strong leadership and project management skills, with experience leading cross-functional teams and managing multiple projects simultaneously. Expertise in cloud data ...

hands-on Snowflake experience Solid understanding of modern cloud data architectures and Lakehouse concepts Familiarity with AWS services including S3, Glue, Athena, RDS, and EMR Confident communicator able to work with both technical and non-technical stakeholders Reference Number: BBBH(phone number removed) Rise Technical Recruitment Ltd acts ...

with one or more of the following would also be beneficial. Scala. AWS services like Kinesis, RDS, Elasticache, S3, Athena, Data pipeline, Glue, Lambda, EMR, Spark, EC2, ECS, CloudWatch or Elastic Beanstalk. Familiarity with Jenkins, GitHub Actions, and Terraform similar tools would also be a plus. Experience with software ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Experience with IaC deployment of AKS/EKS/GKE architecture Experience ...

technology stack 💻 Python and associated ML/DS libraries (scikit-learn, numpy, LightlGBM, Pandas, LangChain/LangGraph, TensorFlow, etc..) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake. Strong knowledge of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns. Preferred Qualifications, Capabilities, and Skills Experience navigating complex data governance, security, and compliance ...

experience with ML libraries such as scikit‐learn, XGBoost, TensorFlow or PyTorch. Extensive experience with Spark or PySpark and distributed data systems (e.g., AWS EMR, Databricks) working on very large, messy datasets. Deep understanding of supervised and unsupervised learning, feature engineering, model evaluation, and experiment design (A/ ...

technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

Ansible and/or Python Skilled in AWS services such as EC2, SSM, CloudFormation, CloudWatch, and Lambda Experience with Spark and Amazon EMR/Hadoop, PySpark and AWS Glue is a plus Experience with Terraform Module creation and usage is a plus Must have hands on experience ...

personal and professional impact. Clear communication of objectives and rationale to foster commitment from teammates. Experience with data‐driven workflows using tools like Airflow, EMR, Glue, or Spark. Core Value Embodiment: Embody our core values to foster a culture of excellence that drives meaningful impact and collective success. ...

cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data lakes and Glue Data Catalog. Strong understanding … Strong hands-on expertise with: AWS DMS (Database Migration Service). AWS SCT (Schema Conversion Tool). AWS Glue (ETL & data integration). Amazon Redshift (data warehousing). Other Qualifications (OQs): Exposure to real-time replication tools (e.g., GoldenGate or equivalent). Database Administration (DBA) expertise, including backup ...

based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance ...

causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats to ensure integrity and correctness of new functionality Support CI/CD practices using Git-based workflows … slowly changing dimensions Ability to generate realistic test data for various scenarios Additional Knowledge Understanding of distributed data platforms and big data environments (e.g., EMR) Knowledge of encryption principles (client-side and server-side) Awareness of production system monitoring and incident analysis Soft Skills Strong analytical and problem-solving ...

governance frameworks Data modeling and schema design for analytics workloads Modern data platform design and implementation ETL/ELT pipeline development and optimization (Glue, EMR) Data warehousing solutions (Redshift) Data catalog management and metadata governance Data quality and lineage frameworks Preferred Qualifications Experience and technical expertise (design and implementation … data center or data center to cloud infrastructure Data security, compliance, and access controls Performance optimization for data processing workloads Big data processing platforms (EMR, Glue) Amazon is an equal opportunities employer and does not discriminate on the basis of protected veteran status, disability, or other legally ...