3 of 3 HDFS Jobs in the UK

Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion for mentoring ...

technologies including Kafka, AWS SQS or equivalent cloud-native services. Strong understanding of SQL and NoSQL databases. Experience with modern data storage technologies including HDFS, Iceberg, Elasticsearch, S3 and Data Lake architectures. Containerisation and orchestration experience using Docker, Kubernetes and/or OpenShift. Knowledge of testing frameworks, engineering standards ...

with strong expertise in designing and implementing scalable big data solutions using Scala and Apache Spark. Experienced in working with the Hadoop ecosystem, including HDFS, Hive, and YARN, to process and analyze large datasets efficiently. Skilled in building robust ETL pipelines, real-time data processing, and optimizing distributed systems … processes using tools like Hive Pig and Spark. Develop and maintain data ingestion processes using Sqoop Flume or Kafka. Optimize MapReduce jobs and manage HDFS storage. Collaborate with data scientists and analysts to support data needs. Ensure data security and compliance with organizational policies. Create and maintain technical documentation ...