26 to 31 of 31 Remote/Hybrid Hadoop Jobs

enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment and reconciliation services Create canonical data models spanning multiple source systems … strong expertise across several of the following areas: Core Technical Skills Python development in enterprise environments Apache Spark/PySpark Apache Kafka Apache NiFi Hadoop and Hive Advanced SQL REST API integration and development Data modelling and data lineage Distributed systems engineering CI/CD and modern software delivery ...

reconciliation data platform built on Scala Spark. We are currently on an ambitious journey to migrate and modernize our platform from a legacy Cloudera Hadoop environment to Databricks on AWS — adopting cloud-native patterns, simplifying pipelines, and unlocking new capabilities along the way. The work is technically complex, high … scalable engineering solutionsWhat Makes Our Team Interesting:We are working on a high-impact, technically challenging modernization — moving a production-grade reconciliation platform from Hadoop to Databricks on AWSThe domain — data reconciliation at scale — is inherently complex and intellectually rewarding, involving large datasets, performance-critical pipelines, and precision engineeringWe ...

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience developing … GDPR. Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. ...

ready to support advanced analytics and business insights. Responsibilities Design, develop, and implement large-scale data pipelines using tools such as Apache Spark, Hadoop, and Apache Hive to process vast amounts of structured and unstructured data. Collaborate with cross-functional teams to understand data requirements and translate them into … best practices in database design and management. Lead efforts in integrating diverse data sources through tools like Informatica and manage big data ecosystems involving Hadoop clusters. Conduct analysis of complex datasets to uncover insights that drive strategic business decisions. Experience Proven experience as a Data Engineer ...

related technologies. You will ingest structured and unstructured data into our data lake and collaborate with Machine Learning Engineers and Data Scientists in Hadoop, Spark, Teradata, GCP, and Azure environments. You should be passionate about developing new solutions, evaluating new tech, and translating data into analytics opportunities. What … Develop high-quality, reusable data pipelines at scale and ingest data into the data lake Collaborate with Machine Learning Engineers and Data Scientists in Hadoop, Spark, Teradata, GCP, and Azure Evaluate new technologies and translate data into analytics-based opportunities Essential experience Good coding/scripting experience using Python ...

Java development Deep expertise in Kafka Commitment to writing high-quality, maintainable code Experience or strong interest in low-latency systems Bonus: Familiarity with Hadoop Client: FinTech Firm Salary: Up to £130,000 + Bonus Location: Hybrid (London) Experience: 6-30+ Years #J-18808-Ljbffr ...