Hadoop Developer (Spark/Scala)
- Hiring Organisation
- Capgemini
- Location
- City of London, London, United Kingdom
troubleshoot Hadoop clusters. Develop robust ETL workflows using Spark, Hive, and Pig. Create and manage data ingestion pipelines using Sqoop, Flume, or Kafka. Optimize MapReduce jobs and efficiently manage HDFS storage. Collaborate with data scientists and analysts to meet data requirements. Ensure data security, compliance, and maintain technical documentation. Your … Java Big Data development experience. Good knowledge of Python for data processing and scripting (nice to have). Solid understanding of Hadoop ecosystem: HDFS, MapReduce, YARN. Experience with Hive, Pig, and HBase. Knowledge of data ingestion tools such as Sqoop, Flume, and Kafka. Ability to optimize distributed data processing jobs. ...