276 to 300 of 305 Apache Spark Jobs in London

needs of the department. This is an exciting opportunity to work on cutting edge technologies including Big Data and NoSQL databases (Hadoop, HBase, Hive, Spark and MongoDB) to allow the business to gain advanced insight into their portfolios and valuation metrics. Key responsibilities include:* Lead business requirements elicitation, analysis … documentation efforts* Assist technology team in Azure cloud solution implementation; write Spark SQL queries to analyze and transform data; provide feedback to User Interface team; testing and documentation* Analyze solution requirements; understand project scope; determine project priorities; ensure efficient and on-time delivery of project tasks and milestones; follow ...

improvements in tooling and processes. Ideal candidates will have 5+ years in backend engineering for data systems, strong experience with MongoDB, Postgres, Kinesis, Flink, Spark, ElasticSearch, Redis, and query engines; plus #J-18808-Ljbffr ...

equivalent platforms, with the ability to partner with technology teams to shape architecture and drive enterprise adoption.Proficiency across Python, q/KDB, SQL, Spark, C#, and related engineering toolsets, with sufficient depth to guide technical decisions, challenge design options, and work effectively with engineering teams to deliver scalable market … Science, Engineering, Data Science, or related discipline.Nice to HaveExperience integrating real-time market data into Align, Endur, and Front Office trading platforms.Familiarity with Flink, Spark Streaming, and modern stream-processing technologies.Knowledge of execution algorithms, quantitative trading strategies, and best execution frameworks.Experience with MDSL, TRG Screen, DACS, or other market ...

REQ ID: FEQ327R160Location: London, UKRecruiter: Dina HussainAt Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified ...

REQ ID: FEQ327R160Location:London, UKRecruiter: Dina HussainAt Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified ...

JOB ID: FEQ227R117Location: London, United KingdomIndustry:Professional Business Services (Consultancy, Legal, etc)At Databricks, our core values are at the heart of everything we do. A culture of proactiveness and a customer-centric mindset guides ...

At Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified platform that makes data science and ...

JOB ID: FEQ327R451Recruiter: Dina HussainLocation: London, UKIndustry:Manufacturing, Defence and AutomotiveAt Databricks, our core values are at the heart of everything we do. A culture of proactiveness and a customer-centric mindset guides us in ...

Location: London, UK At Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer‐centric mindset guides us to create a unified platform that makes ...

engine that provides cross-asset coverage to calculate various Credit Risk measures. FRA platform fraimWRX is distributed as AKS images. FraimWRX relies on Redis, Apache Kafka, Spark, HBASE and HDFS. Risk technology are using Kubernetes to deploy fraimWRX with a Python based Coordinator and Job Runner REST components. ...

operate data workloads across cloud environments is preferred. Knowledge of Golang is a strong advantage. Hands‐on experience with data transformation tools such as Spark and DBT, with a focus on building well‐tested, maintainable data products is a strong advantage. Familiarity with Apache Iceberg table format ...

Applying advanced techniques in LLMs, agentic systems, vector databases, fine‐tuning, and deployment tools (e.g., HuggingFace, Langchain). Hands‐on experience working with Distributed Spark based systems. Experience with data engineering concepts or a good understanding of data engineering concepts. Pre‐sales or post‐sales experience working with external … clients across a variety of industry markets. Minimum of 5+ years of customer‐facing experience would be preferred. [Preferred] Experience working with Apache SparkTM to process large‐scale distributed datasets. Communication: Proven ability to communicate and teach complex technical concepts to both technical and non‐technical audiences. Core Traits ...

strategy and promote best practices amongst engineers and data scientists. How to make an Impact Architect and Implement : Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development … deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

engineering teams. You’ll deliver highly available, secure and high-performance analytics services across on-premises and AWS environments, using technologies such as Spark, Trino, Kafka, ClickHouse and Airflow.This is a highly collaborative position, working directly with researchers and engineers to evolve platform capabilities, automate operations and explore emerging … support critical systemsWho are we looking for? Core skills and technologiesExperience running distributed data and analytics systems at scale using tools such as Spark, Kafka, Trino or AirflowStrong Linux skills and proficiency in Python for automation and integrationFamiliarity with infrastructure as code, using Terraform or AnsibleDeep understanding ...

retraining, and optimise models for reliability and cost efficiency in AWS.Collaboration & Enablement: Partner with data engineers to integrate ML workflows into wider platforms (Spark, Databricks), and share best practice and mentor other technical professionals.What You’ll Love About This RoleThink Big: Build ML and AI solutions that shape products … models underperform across data, features and architecture, and make reasoned trade-offs.Real-time & distributed ML: A strong grasp of production inference patterns, plus Spark and distributed data processing.Reproducibility & tooling: Reproducible environments (UV/Docker) and MLflow or equivalent, on AWS with Spark, Databricks and Snowflake.Engineering mindset: A focus ...

processing, including vector databases, fine-tuning LLMs, and deploying LLMs with tools such as HuggingFace, Langchain, and OpenAI. Hands-on experience working with Distributed Spark based systems. Graduate degree in a quantitative discipline (Computer Science, Engineering, Statistics, Operations Research, etc.) or equivalent practical experience. Experience communicating and teaching technical … driving our values through ML. [Preferred] 2+ years customer-facing experience in a pre-sales or post-sales role. [Preferred] Experience working with Apache SparkTM to process large-scale distributed datasets. Can meet expectations for technical training and role-specific outcomes within 3 months of hire. Can travel ...

have:·Demonstrable hands-on experience in deploying and maintaining machine learning models in production environments·Processing massive amounts of structured and unstructured data using Spark/SQL/Hive·Familiar with offline (batch) and online (live/stream) data pipelines·Writing production code in Python·Understanding the full software … state-of-the-art data analytics products and solutions·Carrying out A/B test experiments·Experience in Big Data technologies (Hadoop, Spark)·Experience in data pre-processing, Machine Learning, and data visualisation·Experience with large relational and NoSQL databases·Communicating findings to non-technical audience·Mentoring and supporting ...

interest or experience in data/network security would be a nice-to-have Datadog/Grafana/Prometheus Data related products (Airflow, Jupyter, Spark, dbt etc) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve … tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data modelling SparkSQL for analytics Streamlit for data applications Are you ready for a career with us? We want ...

modeling and the creation of type-safe and easily testable batch jobs. The team also orchestrates a compute environment that simplifies the execution of Spark jobs, facilitating the analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot … Identifying and implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you'll learn … scalability, reliability, and observability. Strong knowledge of the JVM and server-side Kotlin/Java programming. Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive. Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue. Experience working in Agile environments and contributing ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures.We guarantee you’ll learn … focus on scalability, reliability, and observability.Strong knowledge of the JVM and server-side Kotlin/Java programming.Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive.Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue.Experience working in Agile environments and contributing through code ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you'll learn … scalability, reliability, and observability. Strong knowledge of the JVM and server-side Kotlin/Java programming. Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive. Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue. Experience working in Agile environments and contributing ...

with downstream systems, including trader‐facing tools. Experience 5+ years software engineering experience in building distributed systems. Excellent Java skills. Experience with streaming technologies (Apache Flink preferred; Kafka Streams, Spark Streaming, etc.). Solid understanding of event‐driven architecture and asynchronous processing patterns. Experience handling data consistency challenges ...

customers achieve more with the AWS cloud. Basic qualifications- Experience as technical specialist in design and architecture- Experience in database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis)- Experience in consulting, design and implementation of serverless distributed solutions- Experience in cloud based solution (AWS or equivalent), system, network and operating … Computer Science or related field, or experience working with or evaluating AI systems- Experience in data applications using large scale distributed systems (e.g., EMR, Spark, Elasticsearch, Hadoop, Pig, and Hive)Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. ...

apply them on client data.Perform exploratory data analysis to characterise and visualise datasets Extend and develop programming skills, in languages such as Python and Spark, to develop efficient science code for science modules.Help identify new opportunities within the Data Science space for future dunnhumby solutions.Implement advice from colleagues … programming, ideally Python, and the ability to quickly pick up handling large data volumes with modern data processing tools, e.g. by using Hadoop/Spark/SQLExperience with or ability to quickly learn open-source software including machine learning packages, such as Pandas and scikit-learn, along with data ...