1 to 25 of 31 Amazon EMR Jobs

Engineer (AWS) - Manchester (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client … Noir Consulting. (Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) NOIRUKTECHREC NOIRUKREC ...

Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab ...

/Hybrid Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including ...

notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such as S3, Glue, Lambda, EMR, IAM, KMS, CloudWatch, Secrets Manager, Athena, and related services. Implement lakehouse architecture using Databricks, Delta Lake, and AWS S3. Build data ingestion pipelines from ...

data solutions that meet complex business needs and support informed decision-making. Experience Required: Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions. Skilled in modern programming, particularly Python, Java, Scala and PySpark. Solid knowledge of data storage ...

technical leadership and mentoring within Agile delivery teams. Key Responsibilities: Design and develop data pipelines using PySpark, Python, SQL, Informatica , and AWS services including EMR, S3, and Lambda . Lead the migration of legacy ETL and reporting solutions to modern cloud-based platforms. Support reporting and MI requirements, including ...

extended services. - 2+ years experience working with at least 3 of the following: ECS, EKS, Lambda, DynamoDB, Kinesis, AWS Batch, ElasticSearch/OpenSearch, EMR, Athena, Docker/Kubernetes - Proficient with Python and SQL, and with good experience with data modelling - Experience with a modern orchestration tools (Airflow/Prefect ...

extraction solutions using RESTful APIs and modern integration techniques * Work extensively with AWS-native data and analytics services including Glue, Athena, DataBrew, Lake Formation, EMR, MSK and Kinesis * Implement data quality, profiling and validation processes to ensure integrity, accuracy and completeness of datasets * Contribute within an Agile delivery environment ...

Commercial experience in client-facing projects is a plus, especially within multi-disciplinary teams Deep knowledge of database technologies: Distributed systems (e.g., Spark, Hadoop, EMR) RDBMS (e.g., SQL Server, Oracle, PostgreSQL, MySQL) NoSQL (e.g., MongoDB, Cassandra, DynamoDB, Neo4j) Solid understanding of software engineering best practices code reviews, testing frameworks ...

Remote (occasional expensed travel to Leeds/Newcastle) Clearance: Active SC Clearance required Rate: Up to £425/day (Inside IR35) Tech Stack: AWS (EMR, S3, Lambda), PySpark, Python, SQL, CI/CD Focus: Building scalable pipelines, modernising ETL, supporting MI/Power BI Level: Mid to Senior - mentoring ...

technology stack 💻 Python and associated ML/DS libraries (scikit-learn, numpy, LightlGBM, Pandas, LangChain/LangGraph, TensorFlow, etc..) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy ...

functional teams to deliver end-to-end solutions Key Skills & Experience: Strong expertise in AWS cloud services, including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing … Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging, and version control Knowledge of Docker and containerisation Understanding of encryption ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory, or Apache Hive/Spark/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/ ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

designing metrics and KPIs to verify and automate management of data quality. Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) and/or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions). Desirable Skills Working experience in Data Lakehouse environments such as Databricks ...

takes pride in seeing solutions through to production. Essential experience: AWS data stack ownership in a production environment (S3, Glue, Kinesis, Redshift, EMR) Strong data modelling background (dimensional modelling, CDM, LDM, star schema) ETL/ELT pipeline design and delivery across real-time and batch Data governance, security ...

technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like ...

data analytics and AI-driven solutions. Skilled in designing and automating data quality metrics and KPIs. Proficient with AWS (S3, Kinesis, Glue, Redshift, Lambda, EMR) and/or Azure (ADF, Synapse, Fabric, Functions). To be Considered.... Please either apply by clicking online or emailing me directly to henry.clay ...

Experience with enterprise-scale implementations of cloud-native data platforms such as Databricks and Snowflake. Strong knowledge of AWS data analytics services, including Redshift, EMR, Glue, Athena, Kinesis, Lake Formation, MSK, and S3 data lake patterns. Preferred Qualifications, Capabilities, and Skills Experience navigating complex data governance, security, and compliance ...

experience with ML libraries such as scikit‐learn, XGBoost, TensorFlow or PyTorch. Extensive experience with Spark or PySpark and distributed data systems (e.g., AWS EMR, Databricks) working on very large, messy datasets. Deep understanding of supervised and unsupervised learning, feature engineering, model evaluation, and experiment design (A/ ...

personal and professional impact. Clear communication of objectives and rationale to foster commitment from teammates. Experience with data‐driven workflows using tools like Airflow, EMR, Glue, or Spark. Core Value Embodiment Embody our core values to foster a culture of excellence that drives meaningful impact and collective success. ...

cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data lakes and Glue Data Catalog. Strong understanding … Strong hands-on expertise with: AWS DMS (Database Migration Service). AWS SCT (Schema Conversion Tool). AWS Glue (ETL & data integration). Amazon Redshift (data warehousing). Other Qualifications (OQs): Exposure to real-time replication tools (e.g., GoldenGate or equivalent). Database Administration (DBA) expertise, including backup ...

based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance ...

governance frameworks - Data modeling and schema design for analytics workloads - Modern data platform design and implementation - ETL/ELT pipeline development and optimization (Glue, EMR) - Data warehousing solutions (Redshift) - Data catalog management and metadata governance PREFERRED QUALIFICATIONS - Experience in IT implementation - Data quality and lineage frameworks - Data security, compliance … access controls - Performance optimization for data processing workloads - Big data processing platforms (EMR, Glue) Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion ...