8 of 8 HDFS Jobs in the UK

including mappings, workflows, applications, transformations, and reusable components. Support the migration of data assets, ETL pipelines, and workloads from the Cloudera Hadoop platform (HDFS, Hive, Impala, Spark) to AWS Cloud. Design, develop, and optimise data ingestion, transformation, and loading processes into Amazon S3 Data Lake and Amazon Redshift. Develop ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

between different test types (unit test, integration test) and can cite specific examples of what they have written themselves Good understanding of CDP 7.1, HDFS filesystems, Unix, Unix Shell Scripting, Elasticsearch Experience in understanding and analyzing complex business requirements and carry out the system design accordingly Quantexa Data Engineering Certification ...

cloud-based risk platforms and pricing engines Azure Kubernetes Service (AKS) Azure Data Lake Microservices and event-driven architectures Kafka, Redis, Spark, HBase, and HDFS Market Risk platforms covering VaR, Stress Testing, and FRTB Risk exposure reconciliation and reporting platforms Enterprise scheduling tools such as Control-M What ...

incident retrospectives, and iterative delivery—making pragmatic trade‐offs and documenting them clearly. Technology Stack Languages: Predominantly Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira … improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code Degree in Computer Science or numerical degree ...