176 to 200 of 312 Apache Spark Jobs in London

platforms (AWS, Azure, or GCP).Distributed Systems Mastery: Deep technical capability in structuring data platforms, data pipelines, and high-throughput distributed systems (like Spark or Kafka) optimized for performance and scalability.Cross-Functional Technical Leadership: Exceptional ability to bridge the gap between deep technical engineering and strategic business needs, guiding ...

time founders who previously built Segovia (acquired) and GiveDirectly (>$300M revenue, fastest-growing NGO in the world)Backed by top investors including Spark Capital, Canaan, Reid Hoffman, Breyer Capital, Unbound, Wamda, and moreBuilt by global top performers from Bain, Google, McKinsey, Uber, Checkout, Bolt, and beyond - we move fast ...

automation tasks.Key Responsibilities:Azure Platform Support & MonitoringSupport and maintain Azure-based data solutions, including:Azure Data Factory (ADF) pipelines, dataset, linked service, triggerAzure Databricks (Spark jobs, notebooks, clusters)Azure Machine Learning models and endpointsPower BI dashboards and dataset refreshesMonitor and troubleshoot failures in pipelines, jobs, and ML workflows using … incident response.Maintain detailed SOPs, runbooks, knowledge base articles, and client documentation.Required Skills and Qualifications:Azure ServicesAzure Data Factory (ADF): pipelines, triggers, parameterization, monitoringAzure Databricks: Spark, notebooks, job orchestrationAzure Machine Learning: pipelines, model deployment, monitoringPower BI Service: dataset refreshes, access control, report diagnosticsDevOps & AutomationCI/CD: Azure DevOps, GitHub Actions ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

Databricks. Familiarity with cloud‐based application development (AWS & Azure). Familiarity with one or more ML frameworks and technologies: scikit‐learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, Triton. Strong communication skills, able to express ideas clearly and collaborate across teams. #J-18808-Ljbffr ...

Python and MLflow Strong AWS/cloud-native experience ( ideally with SageMaker exposure) React frontend proficiency Docker, Kafka, and large-scale data systems (e.g. Spark) experience Strong leadership presence and senior stakeholder communication skills If you have the technical expertise, leadership experience and ambition to make an impact ...

Python and MLflow Strong AWS/cloud-native experience ( ideally with SageMaker exposure) React frontend proficiency Docker, Kafka, and large-scale data systems (e.g. Spark) experience Strong leadership presence and senior stakeholder communication skills If you have the technical expertise, leadership experience and ambition to make an impact ...

statistics, machine learning, and mathematical modelling. Experience designing evaluation frameworks for AI systems, including model performance and monitoring. Hands-on experience with Databricks, MLflow, Spark, or similar cloud-based data platforms. Knowledge of responsible AI, model governance, and explainability. Strong communication skills with the ability to explain complex technical ...

/or IFRS9/CECL provisioning models. Hands-on experience with programming languages and tools commonly used in data science, such as Python, SQL, Spark, and AWS. Excellent analytical, problem-solving, and decision-making abilities. A passion for innovation and staying at the forefront of data science and risk ...

including Feast or AWS Feature Store. Docker, Kubernetes, or Amazon ECS. Infrastructure as Code using Terraform or CloudFormation. Big data processing technologies such as Spark or Dask. Experience working within regulated or highly secure environments. What's On Offer Work on meaningful AI programmes supporting critical national security missions. ...

Qlik). Knowledge of Python or R for advanced analytics and automation is highly desirable. Familiarity with cloud/big data platforms (Snowflake, Databricks, Spark) is preferred. Key Requirements Minimum 5+ years of hands-on experience in Data Analytics or Data Science (experience within Customer Success, Operations, or Digital ...

Data; prior experience with Tier-1 Cloud Service Providers (CSPs) is highly preferred. Expertise in the Hadoop ecosystem and deep understanding of components like Spark, Flink, Elasticsearch, Doris, ClickHouse, and Trino., Strong knowledge of big data architecture, including Data Lake (Iceberg, Hudi) and Serverless data concepts. Familiarity with mainstream ...

powered services into production. Strong Python and Java, both of which are in active use across the team's production services. Experience with Snowflake, Spark, Databricks or similar technologies, CI/CD pipelines, and modern DevOps tooling. Solid understanding of SecOps practices and security‐conscious system design. Demonstrable track ...

data libraries such as Pandas, NumPy and Scikit-learnA solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniquesExperience with Spark, agile delivery methods and CI/CD practicesFamiliarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlowConfidence influencing and communicating ...

such as Pandas, NumPy and Scikit-learn A solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniques Experience with Spark, agile delivery methods and CI/CD practices Familiarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlow Confidence influencing ...

agentic systems. Experience designing external-facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

implementing process improvements and automation Strong Python development skills (including Pandas and related data-processing libraries). Experience with big data technologies such as Spark, PySpark, Hadoop, and Hive Exposure to quantitative modelling or financial modelling is a plus but not required. Skills that will help: Global Risk Management ...

JavaScript etc.Experience implementing process improvements and automationStrong Python development skills (including Pandas and related data-processing libraries).Experience with big data technologies such as Spark, PySpark, Hadoop, and Hive.Exposure to quantitative modelling or financial modelling is a plus but not required.Skills that will help:Global Risk Management experienceBenefits ...

least 2 years in a customer-facing, embedded, or solutions engineering capacityStrong hands-on experience with streaming and event-driven architectures — Kafka, Flink, Kinesis, Spark Streaming, or equivalent (DynamoDB and MemoryDB/Redis)AWS Cloud Fluency - understand how to design scalable end-to-end architectures and pipelines on large ...

with the technical workStrong software engineering practice: Python, version control, reproducible pipelines and production-ready codeExperience with large-scale data processing (we use Kafka, Spark and Flink)The ability to explain complex technical work clearly to senior leadership, commercial teams and partners, and a bias for connecting research ...

have a background)Testing and deployment pipelines, software and productivity tools, monitoring and middleware, data infrastructure.Integrating best-of-breed tools and technologies such as Spark, Kafka, Zookeeper, Mesos, Docker, ElasticsearchAssembling platforms and frameworks to automate business processesCollaborating with internal technology, research and portfolio management teamsPython Developers MUST have ...

startup, or equivalent experience navigating fast-paced startup lifecycles. Strong understanding of modern data warehouse architectures, real-time analytics, and data governance (e.g., BigQuery, Spark, Kafka). Deep technical expertise in Generative AI development (e.g., prompt engineering, model tuning, vector databases like Vertex AI Vector Search, and AI frameworks ...

startup, or equivalent experience navigating fast-paced startup lifecycles. Strong understanding of modern data warehouse architectures, real-time analytics, and data governance (e.g., BigQuery, Spark, Kafka). Deep technical expertise in Generative AI development (e.g., prompt engineering, model tuning, vector databases like Vertex AI Vector Search, and AI frameworks ...

Candidates Must Have: • 4 years commercial experience with Linux CentOS , bare metal and virtualisation • Puppet configuration management or equivalent automation experience • Big data (Hadoop, Spark, Kafka) • Containers and scheduling experience (Docker, LXC, Kubernetes, Mesos) • Strong scripting You will be joining a progressive and exciting company committed to excellence. They ...

with the broader internal/external ecosystem.On day one you we will expect you to have:Experience working with modern data technology (e.g. dbt, spark, containers, devops tooling, orchestration tools, git, etc.)Solution engineering/solution architect experience in a Saas environment or relevant industry experience (analytics, data science ...