126 to 133 of 133 Apache Spark Jobs in London

modeling and the creation of type-safe and easily testable batch jobs. The team also orchestrates a compute environment that simplifies the execution of Spark jobs, facilitating the analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot … Identifying and implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge ...

ability to mentor and develop others. You'll lead platform engineering initiatives while coaching a team of BI Developers in Fabric, Python and Spark, helping drive the organisation's data engineering capability forward. Key Responsibilities Development, support and enhancement of EDAP on Microsoft Fabric. Design and build scalable data … Python and Spark. Work with stakeholders to deliver robust, business-focused data solutions. Essential Skills Microsoft Fabric (hands-on platform engineering experience). Python. Apache Spark. Azure Data Factory, Databricks and Data Lake. Enterprise-scale data platform design and delivery. Experience coaching, mentoring and leading technical teams. Important This ...

replication tool, and Waggle Dance , a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg , a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures.We guarantee you’ll learn … focus on scalability, reliability, and observability.Strong knowledge of the JVM and server-side Kotlin/Java programming.Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive.Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue.Experience working in Agile environments and contributing through code ...

downstream systems, including trader-facing tools. Your experience 5+ years software engineering experience in building distributed systems. Excellent Java skills Experience with streaming technologies (Apache Flink preferred; Kafka Streams, Spark Streaming, etc.). Solid understanding of event-driven architecture and asynchronous processing patterns. Experience handling data consistency challenges ...

scalability Build and integrate RESTful microservices within a complex distributed architecture Work with large-scale data sets, leveraging SQL and big data tools (e.g., Spark, Kafka) Influence system design and contribute to the technical roadmap Collaborate closely with a highly skilled engineering team to deliver innovative solutions Requirements Strong … experience building and consuming RESTful services Solid SQL skills and familiarity with large-scale data processing Knowledge of big data technologies such as Spark and Kafka Experience in finance is preferred, but not essential Strong academic background, ideally in Computer Science or a related discipline #J-18808-Ljbffr ...

tuned foundational Large Language Models (LLMs), along with sub‐agents, tools, and MCP integrations Architect end‐to‐end ML pipelines, integrating data processing (e.g., Spark, Airflow) with model training, evaluation, and deployment workflows Drive experimentation frameworks, including A/B testing and offline evaluation, to continuously improve model performance … fine‐tuning Large Language Models (LLMs) and building production AI Agents. Proven experience designing scalable data and ML pipelines using tools such as Spark, Airflow, or similar distributed systems. Demonstrated ability to operate as a senior individual contributor, influencing technical strategy and decision‐making without direct authority. Experience partnering ...

Engineer requirig strong GCP experience. The role will focus on building scalable data platforms and pipelines on GCP, working with technologies including Kafka, Python, Spark and Iceberg , while implementing robust data governance, lineage and auditability. Key Requirements: Active SC Clearance - (SC Elapsed Candidates will also be considered!) Strong … experience Kafka, Python, Spark & Iceberg Data Governance and Lineage expertise Please note that rate and IR35 determination are still being finalised and are expected next week . Given the project timelines, immediately available contractors are strongly preferred .Please apply now to avoid missing out on being shorlisted for this. ...

Privacy Legal Advisor - 12 Month Contract Location: UK London Employment Type: Employee - Fixed Term Contract Work Arrangement: Hybrid At Bumble, the Love Company, trust is the heart of every connection we help spark. As a ...