176 to 188 of 188 Apache Spark Jobs in England

strategy and promote best practices amongst engineers and data scientists. How to make an Impact Architect and Implement: Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development … deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

Applying advanced techniques in LLMs, agentic systems, vector databases, fine‐tuning, and deployment tools (e.g., HuggingFace, Langchain). Hands‐on experience working with Distributed Spark based systems Experience with data engineering concepts or a good understanding of data engineering concepts Pre‐sales or post‐sales experience working with external … clients across a variety of industry markets. Minimum of 5+ years of customer‐facing experience would be preferred [Preferred] Experience working with Apache SparkTM to process large‐scale distributed datasets Communication: Proven ability to communicate and teach complex technical concepts to both technical and non‐technical audiences. Core Traits ...

experimentation frameworks that support business decision-making. Data Engineering & Model Productionisation: Develop robust analytical datasets and machine learning pipelines using Databricks, Python, SQL and Spark, ensuring solutions are scalable, reliable and production-ready. Stakeholder Partnership & Commercial Impact: Partner with business stakeholders to define hypotheses, success measures and value metrics … analytics solutions within commercial environments. Python, SQL & Data Platform Expertise: Advanced Python and SQL skills, with experience building reproducible machine learning workflows using Databricks, Spark or similar large‐scale data processing technologies. Machine Learning & Statistical Expertise: Deep understanding of supervised and unsupervised machine learning, feature engineering, model evaluation, experimentation ...

modeling and the creation of type-safe and easily testable batch jobs. The team also orchestrates a compute environment that simplifies the execution of Spark jobs, facilitating the analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot … Identifying and implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge ...

replication tool, and Waggle Dance , a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg , a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you’ll learn … scalability, reliability, and observability. Strong knowledge of the JVM and server-side Kotlin/Java programming. Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive. Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue. Experience working in Agile environments and contributing ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you’ll learn … scalability, reliability, and observability. Strong knowledge of the JVM and server-side Kotlin/Java programming. Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive. Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue. Experience working in Agile environments and contributing ...

business use cases and/or to improve product/system performance, quality and accuracy. Near Real-Time and Batch Inferencing: Use infrastructure like Spark and Ray to stand up inferencing services that integrate with operational/analytics workloads. ML Infrastructure: Help build a first-class machine learning platform … learning on real recommendations use cases (brownie points for productionised sequential learning use cases!). Experience with ML/distributed ML frameworks like Ray, Spark-MLlib, TensorFlow etc. Experience with real-time scoring/evaluation of models with low latency constraints. Great coding skills and strong software development experience ...

with downstream systems, including trader‐facing tools. Experience 5+ years software engineering experience in building distributed systems. Excellent Java skills. Experience with streaming technologies (Apache Flink preferred; Kafka Streams, Spark Streaming, etc.). Solid understanding of event‐driven architecture and asynchronous processing patterns. Experience handling data consistency challenges ...

Engineer requirig strong GCP experience. The role will focus on building scalable data platforms and pipelines on GCP, working with technologies including Kafka, Python, Spark and Iceberg , while implementing robust data governance, lineage and auditability. Key Requirements: Active SC Clearance - (SC Elapsed Candidates will also be considered!) Strong … experience Kafka, Python, Spark & Iceberg Data Governance and Lineage expertise Please note that rate and IR35 determination are still being finalised and are expected next week . Given the project timelines, immediately available contractors are strongly preferred .Please apply now to avoid missing out on being shorlisted for this. ...

scalability Build and integrate RESTful microservices within a complex distributed architecture Work with large-scale data sets, leveraging SQL and big data tools (e.g., Spark, Kafka) Influence system design and contribute to the technical roadmap Collaborate closely with a highly skilled engineering team to deliver innovative solutions Requirements Strong … experience building and consuming RESTful services Solid SQL skills and familiarity with large-scale data processing Knowledge of big data technologies such as Spark and Kafka Experience in finance is preferred, but not essential Strong academic background, ideally in Computer Science or a related discipline #J-18808-Ljbffr ...

business use cases and/or to improve product/system performance, quality and accuracy. Near Real‐Time and Batch Inferencing: Use infrastructure like Spark and Ray to stand up inferencing services that integrate with operational/analytics workloads. ML Infrastructure: Help build a first‐class machine learning platform … learning on real recommendations use cases (brownie points for productionised sequential learning use cases). Experience with ML/distributed ML frameworks like Ray, Spark‐MLlib, TensorFlow etc. Experience with real‐time scoring/evaluation of models with low latency constraints. Great coding skills and strong software development experience ...

Privacy Legal Advisor - 12 Month Contract Location: UK London Employment Type: Employee - Fixed Term Contract Work Arrangement: Hybrid At Bumble, the Love Company, trust is the heart of every connection we help spark. As a ...