251 to 261 of 261 Amazon EMR Jobs

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

that are secure, scalable, and production-ready. What You'll Do: Implementing data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR Creating and maintaining data extraction, transformation, and loading processes Configuring and optimizing AWS database services including RDS, Aurora, Redshift, and DynamoDB Implementing data lakes … using S3 and related AWS services Designing and building production-ready Generative AI applications using Amazon Bedrock and foundation models such as Anthropic Claude Building and optimizing RAG (Retrieval-Augmented Generation) pipelines with vector databases Developing AI agents and multi-agent orchestration systems using frameworks like LangChain ...

Ensure high availability and reliability of data pipelines across enterprise systems AWS Cloud & Data Services Leverage AWS services such as S3, Glue, Lambda, Redshift, EMR, and Kinesis to support data engineering solutions Design and manage cloud-based data architectures that are secure, scalable, and cost-efficient Implement best practices … technical field (or equivalent experience) Demonstrated experience in data engineering and pipeline development Strong experience with AWS data services (e.g., S3, Glue, Redshift, Lambda, EMR) Proficiency in SQL and Python for data processing and transformation Experience working with large-scale data sets and big data technologies Strong analytical ...

data experts to unlock actionable insights from diverse datasets. Requirements: Engineer robust, secure, and scalable data pipelines using Apache Spark , Apache Hudi , AWS EMR , and Kubernetes Maintain data provenance and access controls to ensure full lineage and auditability of mission-critical datasets Clean, transform, and condition data using tools … Proficiency in Python , SQL , and one or more of: Java , Scala Strong understanding of cloud services (especially AWS and GovCloud ), including S3 , EC2 , Lambda , EMR , Glue , Redshift , or Snowflake Hands-on experience with streaming frameworks such as Apache Kafka , Kafka Connect , or Flink Familiarity with data lakehouse formats (e.g. ...

Senior Data Engineer Manchester, 2 days per week Millions of customers. One proprietary platform. Infinite possibilities. This global technology business has just completed a massive migration to their own platform, and now, the stage is ...

causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats to ensure integrity and correctness of new functionality Support CI/CD practices using Git-based workflows … slowly changing dimensions Ability to generate realistic test data for various scenarios Additional Knowledge Understanding of distributed data platforms and big data environments (e.g., EMR) Knowledge of encryption principles (client-side and server-side) Awareness of production system monitoring and incident analysis Soft Skills Strong analytical and problem-solving ...

other developers, architects, and our clients. Specific responsibilities for the Data Engineer - Cloud position include: Developing data pipelines and other data products using Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) Advising clients on specific technologies and methodologies for utilizing cloud resources to efficiently ingest … Database: SQL Server, PostgreSQL, Snowflake, Redshift, Aurora, Presto, BigQuery, Oracle DevOps: git, docker, subversion, Kubernetes, Jenkins Additional Technologies: Spark, Databricks, Kafka, Kinesis, Hadoop, Lambda, EMR Popular Certifications: AWS Cloud Practitioner, Microsoft Azure Data Fundamentals, Google Associate Cloud Engineer Additional Information We want everyone at CapTech to be able ...