151 to 175 of 254 Apache Spark Jobs in London

orchestration, standardize data across real-world teleoperation and synthetic datasets, and collaborate with a team building general-purpose Physical AI. Strong Python/Go, Spark, Kafka, Kubernetes, and Terraform experience required. #J-18808-Ljbffr ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

understanding of statistics, machine learning methodologies, and model evaluation. Experience delivering AI solutions from proof-of-concept through to production. Exposure to Databricks, MLflow, Spark, or similar cloud-based data platforms. Ability to influence senior stakeholders and lead technical strategy Desirable Experience Experience building multi-agent AI systems ...

diagnosing and fixing performance bottlenecks in live systems through profiling and measurement. Nice to Have: Experience with PostgreSQL/pgvector, OpenSearch (or Elasticsearch), Spark/Delta Lake, AWS, NoSQL databases, or Rust/C++ is useful. The Founders You'll partner with a founding team of AI PhDs ...

data mining, information retrieval, and statistics Understanding of machine learning frameworks: Tensorflow, Pytorch, pyG, Keras, MXNet, Scikit-Learn Strong programming knowledge of Python and Spark; proficiency in vector operations using numpy, scipy; experience with distributed computation using multithreading, multi GPUs, Dask, Ray, Polars Familiarity with AWS Cloud services such ...

essential: Background in financial services or regulated environments. Exposure to financial crime domains such as fraud, AML or sanctions. Experience with Spark or PySpark. Familiarity with AWS cloud services and data platforms. You will be assessed on key capabilities relevant to the role, including risk and controls, change ...

essential: Background in financial services or regulated environments. Exposure to financial crime domains such as fraud, AML or sanctions. Experience with Spark or PySpark. Familiarity with AWS cloud services and data platforms. You will be assessed on key capabilities relevant to the role, including risk and controls, change ...

essential: Background in financial services or regulated environments. Exposure to financial crime domains such as fraud, AML or sanctions. Experience with Spark or PySpark. Familiarity with AWS cloud services and data platforms. You will be assessed on key capabilities relevant to the role, including risk and controls, change ...

investors Databricks Ventures, ServiceNow Ventures, and Workday Ventures participating alongside returning investors Altimeter Capital, Avenir Growth Capital, D1 Capital Partners, K5 Global, NewView Capital, Spark Capital, Sutter Hill Ventures, and XN. This milestone follows Sigma reaching $200M in annual recurring revenue in April 2026, with more than 100% year ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with an ethos of commitment ...

retrieval, and multi‐modal representation learning. Technical Stack: Mastery of Python and deep learning frameworks (TensorFlow, PyTorch) alongside hands‐on experience with distributed computing (Spark, Ray) and cloud infrastructure (AWS/GCP). Graph Neural Networks (GNNs): Strong experience applying GNNs, knowledge graphs, or graph embeddings to map complex ...

retrieval, and multi-modal representation learning. Technical Stack: Mastery of Python and deep learning frameworks (TensorFlow, PyTorch) alongside hands-on experience with distributed computing (Spark, Ray) and cloud infrastructure (AWS/GCP). Graph Neural Networks (GNNs): Strong experience applying GNNs, knowledge graphs, or graph embeddings to map complex ...

Engineer, maintaining and further developing a well-established cloud and CI/CD setup to implement and evolve reliable, well-tested data pipelines in Spark/Scala and to extend the dimensional data models. You will be part of the technical team collaborating closely with platform engineering, data science … ingesting from various sources into Databricks Collaborate closely with software engineers to evolve our software platform and address technical challenges Design, develop, and optimize Spark/Scala jobs for ingestion and transformation, using the existing GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests ...

improve reliability, maintainability, scalability and reuse.Your experience is likely to include several of the following: Microsoft Fabric Snowflake Azure OneLake Lakehouse architecture Data Factory Spark Python SQL ETL/ELT dimensional modelling Data Vault orchestration Git Azure DevOps/GitHub CI/CD Infrastructure as Code data quality MLOps … data they receive. Optimise Microsoft Fabric, Snowflake and Azure environments improve performance, scalability, resilience and resource utilisation across OneLake, Lakehouse, Data Factory, Spark/Notebooks, Snowflake and associated cloud services. Person Specification Qualifications Essential Educated to master's level or equivalent level of experience working at a senior level ...

components . Build and maintain platform capabilities supporting batch and real-time data workloads . Develop and operate distributed data services using Kafka, Kinesis, Spark, Flink or equivalent technologies . Build and maintain cloud-native data platform capabilities using AWS and/or Azure . Support data ingestion, processing … Hands-on experience with real-time streaming and event-driven architectures , including Kafka, Kinesis or equivalent. Experience with distributed data processing technologies such as Spark, Flink, Kafka Streams or equivalent. Strong experience with AWS and/or Azure cloud environments. Experience with cloud services such as S3, Glue, Lambda ...

exciting future and play a vital role in growth. You will lead the design and development of scalable data pipelines using Python and Apache Spark, architect modern data platforms on Azure and Fabric, and own end-to-end data delivery with a focus on quality and reliability. ...

ensure data availability, quality, and efficiency across markets. This role is onsite in London and requires strong SQL and programming skills, plus experience with Spark, Hadoop, and cloud platforms. #J-18808-Ljbffr ...

collaborating with data teams to deliver robust data solutions. The role requires strong SQL, Python and experience with distributed data tools like Presto/Spark/Hive, plus workflow scheduling with Airflow. Excellent communication and teamwork are essential. #J-18808-Ljbffr ...

data pipelines. The ideal candidate has 3+ years in data platform or infrastructure roles, strong AWS and Terraform skills, and experience with Kafka, Flink, Spark, and related lakehouse #J-18808-Ljbffr ...

software engineering skills (Python, Go, or similar) Extensive experience designing, building, and maintaining large-scale data pipelines (8+ years) Deep understanding of distributed systems (Spark, Kafka, or similar) Extensive experience with data storage technologies (data lakes, warehouses, object stores like S3) Experience running and maintaining production-grade infrastructure (Kubernetes ...

data mining, information retrieval, and statistics Understanding of machine learning frameworks: Tensorflow, Pytorch, pyG, Keras, MXNet, Scikit-Learn Strong programming knowledge of Python and Spark; proficiency in vector operations using numpy, scipy; experience with distributed computation using multithreading, multi GPUs, Dask, Ray, Polars Familiarity with AWS Cloud services such ...

experience with additional cloud providers is a plus. Demonstrated success leading initiatives involving complex data pipelines and analytics platforms utilizing technologies such as Hive, Spark, Presto, Cassandra, Hadoop, and ML/NLP capabilities. IQVIA is a leading global provider of clinical research services, commercial insights and healthcare intelligence ...

Python, C++; and Go. Deploy and manage applications using Kubernetes (K8s) and HPC/Grid Computing/Big Data technologies such as Slurm, LSF, Spark, Ray and Symphony in cloud environments. Required Qualifications, Capabilities, And Skills Formal training or certification on Software Engineering concepts and 5+ years applied experience. ...

Desirable Experience with cloud-based risk platforms and pricing engines Azure Kubernetes Service (AKS) Azure Data Lake Microservices and event-driven architectures Kafka, Redis, Spark, HBase, and HDFS Market Risk platforms covering VaR, Stress Testing, and FRTB Risk exposure reconciliation and reporting platforms Enterprise scheduling tools such as Control ...

business domains. What would Be Useful: Experience engineering systems for the cloud-based storage and processing of large datasets or using frameworks such as Spark and open standards like Parquet and Arrow. Experience working with other cloud and related technologies, such as Docker and Kubernetes. Experience working ...