11 of 11 HDFS Jobs

executive audiences. Drive innovation and adoption of emerging technologies aligned with business strategy. Required Technical Skills Data Platforms Databricks Lakehouse Cloudera Snowflake Hadoop Ecosystem - HDFS, Hive, Spark, YARN Data Lakes & Data Warehouses Delta Lake Data Mesh/Data Fabric Data Engineering ETL/ELT Architecture Enterprise Data Pipelines Apache Spark ...

Python/Scala, Hadoop, Hive, Kafka, SQL, and cloud-based data platforms. Key Skills Required Apache Spark & PySpark Python and/or Scala Hadoop, HDFS, Hive & MapReduce Apache Kafka & Spark Streaming SQL & Relational Databases ETL/ELT Pipeline Development Batch & Real-Time Data Processing Data Lakes & Data Warehousing ...

including mappings, workflows, applications, transformations, and reusable components. Support the migration of data assets, ETL pipelines, and workloads from the Cloudera Hadoop platform (HDFS, Hive, Impala, Spark) to AWS Cloud. Design, develop, and optimise data ingestion, transformation, and loading processes into Amazon S3 Data Lake and Amazon Redshift. Develop ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching ...

between different test types (unit test, integration test) and can cite specific examples of what they have written themselves Good understanding of CDP 7.1, HDFS filesystems, Unix, Unix Shell Scripting, Elasticsearch Experience in understanding and analyzing complex business requirements and carry out the system design accordingly Quantexa Data Engineering Certification ...

cloud-based risk platforms and pricing engines Azure Kubernetes Service (AKS) Azure Data Lake Microservices and event-driven architectures Kafka, Redis, Spark, HBase, and HDFS Market Risk platforms covering VaR, Stress Testing, and FRTB Risk exposure reconciliation and reporting platforms Enterprise scheduling tools such as Control-M What ...

incident retrospectives, and iterative delivery—making pragmatic trade‐offs and documenting them clearly. Technology Stack Languages: Predominantly Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira … improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code Degree in Computer Science or numerical degree ...

deployment automation. Partner with business, risk, compliance, and architecture teams to deliver data solutions. Develop on Big Data platforms using Hadoop ecosystem components including HDFS, Hive, Spark, and YARN. Design and operate streaming pipelines using Kafka or similar technologies. Translate business requirements into scalable data solutions and technical implementations. Troubleshoot … like Splunk to improve reliability and efficiency. Required Skills & Qualifications: Hands-on experience with Big Data technologies and distributed systems. Proficiency with Hadoop ecosystem: HDFS, Hive, Spark, and YARN. Experience building and optimizing batch and streaming data pipelines, preferably with Kafka. Experience implementing CI/CD and automated deployments using ...