2 of 2 HDFS Jobs

contributor Preferred/Nice-to-Have Skills: Agentic AI frameworks: LangGraph, LangChain, A2A Programming Languages: Java, Scala, SQL, HiveQL Big Data Technologies: Hadoop, Spark, HDFS, Hive, Cloudera, Hortonworks Cloud Platforms: AWS (Glue, Lambda, Redshift, S3, CloudWatch) ETL/ELT Tools: AWS Glue, PySpark, Databricks Data Integration & Migration: Hadoop ecosystems, Kafka ...

with strong expertise in designing and implementing scalable big data solutions using Scala and Apache Spark. Experienced in working with the Hadoop ecosystem, including HDFS, Hive, and YARN, to process and analyze large datasets efficiently. Skilled in building robust ETL pipelines, real-time data processing, and optimizing distributed systems … processes using tools like Hive Pig and Spark. Develop and maintain data ingestion processes using Sqoop Flume or Kafka. Optimize MapReduce jobs and manage HDFS storage. Collaborate with data scientists and analysts to support data needs. Ensure data security and compliance with organizational policies. Create and maintain technical documentation ...