1 to 25 of 79 Amazon EMR Jobs

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Proactive, collaborative, and delivery-focused mindset Desirable Skills Knowledge of open table formats (Iceberg, Delta) Experience with additional AWS services (CloudWatch, SNS, Athena, DynamoDB, EMR, Kinesis) Data modelling Job scheduling and orchestration tools Data virtualisation tools (e.g. Denodo) ALM tools (Jira, Confluence) CI/CD tooling (GitLab, Terraform) Reporting ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

Agile, multidisciplinary teams Strong analytical and problem-solving skills Clear communicator and collaborative team player Desirable Python, Docker, Airflow Big data tools (e.g. Spark, EMR) Experience with data platforms or risk/fraud systems How You'll Work Deliver value iteratively and early Collaborate closely across disciplines Write simple ...

large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . You will be responsible for defining … transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis for production data issues and platform defects Risk Management & Reporting ...

high-performance data pipelines in Snowflake and Databricks (PySpark, Delta Lake, Unity Catalog) for batch and streaming workloads. Orchestration using Apache Airflow and Amazon tools. Engineer scalable data ingestion pipelines into AWS (DBT, SQL, PythonS3, Kinesis, Glue, Lambda, Step Functions) with strong monitoring and fault tolerance. Ensure observability … advanced macros, Jinja, testing, exposures, dbt Cloud deployment. Databricks: Spark (PySpark, SQL), Delta Lake, Unity Catalog. Snowflake AWS: S3, Glue, Lambda, Step Functions, Datasync, EMR, Redshift, IAM and networking/security fundamentals. Data Visualization: Power BI or Tableau. Strong programming background in Python and SQL (including query optimization). ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

mapping. Data quality governance and performance optimization. A solid understanding of end-to-end data ingestion and transformation processes. AWS data stack (S3, Glue, EMR, PySpark, Iceberg - nice to have). Preferred Skills Demonstrated commitment, ownership, and accountability. Strong attention to detail with a proactive approach to identifying ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake. Strong knowledge of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns. Preferred Qualifications, Capabilities, and Skills Experience navigating complex data governance, security, and compliance ...

required to evidence your clearance) Required Skills & Experience Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure ...

required to evidence your clearance) Required Skills & Experience Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure ...

loss Enable CDC-based replication and minimal downtime migrations across heterogeneous systems Design and optimize ETL/data pipelines using AWS Glue, PySpark, or EMR Architect and implement scalable, secure, and highly available data platforms on AWS Drive schema conversion and modernization strategies (heterogeneous migrations where applicable) Optimize performance … Redshift (data warehousing) Experience designing and implementing data validation and reconciliation frameworks Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent) Deep proficiency in SQL and data modeling concepts Experience working with S3-based data lakes and Glue Data Catalog Strong understanding ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

experience with ML libraries such as scikit‐learn, XGBoost, TensorFlow or PyTorch. Extensive experience with Spark or PySpark and distributed data systems (e.g., AWS EMR, Databricks) working on very large, messy datasets. Deep understanding of supervised and unsupervised learning, feature engineering, model evaluation, and experiment design (A/ ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

usability, and scalability. Proficiency in front-end technologies such as HTML, CSS, JavaScript (ES6+), and related libraries. Experience with AWS services like Lambda, EC2, EMR, Redshift, Glue, S3, IAM, RDS, Aurora, DynamoDB etc. Hands-on experience in relational databases (Oracle, PostgreSQL, RDS, Amazon Aurora, SQL Server … Orchestration services including Docker and Kubernetes, and a variety of AWS tools and services. Expert understanding of microservices architecture and cloud implementations, using Amazon Web Services, 3+ years of experience setting up and working with Amazon Web Services (AWS) environments, especially AWS Lambda and other serverless ...

Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena to query and validate data Data analysis to identify … changing dimensions/historic data capture Use of AWS console and services such as, but not limited to; CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, Dynamo DB, LakeFormation Familiarity with Amazon Textract and Comprehend Understanding of both server-side and client-side encryption Use of GitLab ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...

Description Job Description Key Responsibilities: Develop and integrate custom software for EMR interoperability. Automate data extraction, transformation, and loading processes using cloud-based tools. Design and implement healthcare data ingestion pipelines using HL7v2, CCDA, and FHIR standards. Develop SMART on FHIR applications with authentication and authorization (UDAP Security Flow ...