1 to 25 of 29 Amazon EMR Jobs

SageMaker JumpStart, prompt engineering, RAG architectures, vector databases, and 2) **Data Platforms****:** AWS Glue, SageMaker Unified Studio, Databricks on AWS, Snowflake on AWS, Redshift, EMR, Apache Iceberg, Delta Lake.• Position AWS solutions effectively against other cloud platforms' offerings in both generative AI (Azure OpenAI, Vertex AI) and data platforms … Q, LLM architectures, RAG implementations.* Proven track record delivering data modernization: Lakehouse architectures, Databricks and/or Snowflake implementations, AWS Glue/EMR deployments* A bachelor's degree in computer science, Data Science, Engineering, Mathematics, or a related technical field is required. At the manager’s discretion ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Glue. p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Hands-on experience with AWS data services including Redshift, Athena, Glue, EMR, and Kinesis. p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Solid understanding of data modelling, warehousing, and schema design. ...

Proactive, collaborative, and delivery-focused mindset Desirable Skills Knowledge of open table formats (Iceberg, Delta) Experience with additional AWS services (CloudWatch, SNS, Athena, DynamoDB, EMR, Kinesis) Data modelling Job scheduling and orchestration tools Data virtualisation tools (e.g. Denodo) ALM tools (Jira, Confluence) CI/CD tooling (GitLab, Terraform) Reporting ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

pipelines Implement data quality checks, monitoring, and logging Support analytics, reporting, and downstream data products Tech Stack Cloud: AWS (S3, Glue, Lambda, Redshift, Athena, EMR) Languages: Python Data Tools: Glue, Airflow, dbt, CDK or custom ETL frameworks Databases: SQL (Redshift, Snowflake, Postgres) What We're Looking For Strong experience ...

standards Essential Skills & Experience Databricks engineering - strong hands-on experience Azure data stack - Data Lake, ADF, Functions, DevOps AWS data services - Glue, Lambda, S3, EMR (or equivalent) Python & SQL - production-grade coding CI/CD - Git-based workflows, automated deployments Stakeholder communication - able to operate autonomously and influence senior ...

large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . You will be responsible for defining … transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis for production data issues and platform defects Risk Management & Reporting ...

with one or more of the following would also be beneficial. Scala. AWS services like Kinesis, RDS, Elasticache, S3, Athena, Data pipeline, Glue, Lambda, EMR, Spark, EC2, ECS, CloudWatch or Elastic Beanstalk. Familiarity with Jenkins, GitHub Actions, and Terraform similar tools would also be a plus. Experience with software ...

hands-on Snowflake experience Solid understanding of modern cloud data architectures and Lakehouse concepts Familiarity with AWS services including S3, Glue, Athena, RDS, and EMR Confident communicator able to work with both technical and non-technical stakeholders Reference Number: BBBH(phone number removed) Rise Technical Recruitment Ltd acts ...

technology stack 💻 Python and associated ML/DS libraries (scikit-learn, numpy, LightlGBM, Pandas, LangChain/LangGraph, TensorFlow, etc..) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake. Strong knowledge of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns. Preferred Qualifications, Capabilities, and Skills Experience navigating complex data governance, security, and compliance ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

experience with ML libraries such as scikit‐learn, XGBoost, TensorFlow or PyTorch. Extensive experience with Spark or PySpark and distributed data systems (e.g., AWS EMR, Databricks) working on very large, messy datasets. Deep understanding of supervised and unsupervised learning, feature engineering, model evaluation, and experiment design (A/ ...

Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena to query and validate data Data analysis to identify … changing dimensions/historic data capture Use of AWS console and services such as, but not limited to; CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, Dynamo DB, LakeFormation Familiarity with Amazon Textract and Comprehend Understanding of both server-side and client-side encryption Use of GitLab ...

based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance ...

Production data analysis to identify root cause of production environment issues. Using Apache Airflow for task orchestration and reviewing output logs. Basic understanding of EMR and reviewing output logs. Basic understanding of data processing using Apache Spark. Use of Jupyter notebooks to query and validate data. Use of Amazon … data in multiple formats to ascertain validity of new functionality. Use of AWS console and services such as CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB. Understanding of both server‐side and client‐side encryption. Basic understanding of dimensional data models and slowly changing dimensions/historic data capture. ...

causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats to ensure integrity and correctness of new functionality Support CI/CD practices using Git-based workflows … slowly changing dimensions Ability to generate realistic test data for various scenarios Additional Knowledge Understanding of distributed data platforms and big data environments (e.g., EMR) Knowledge of encryption principles (client-side and server-side) Awareness of production system monitoring and incident analysis Soft Skills Strong analytical and problem-solving ...