151 to 175 of 312 Apache Spark Jobs in London

commercial forecasting processes. Familiarity with web-analytics tools & optimizing user interfaces. Experience with survey exchange platforms or online market research products. Experience with Databricks, Spark, or PySpark for scalable data processing. Additional Information#LI-Remote#LI-JC1Our ValuesCollaboration is our superpowerWe uncover rich perspectives across the worldSuccess happens togetherWe deliver across ...

data quality programs, or analytics transformationStrong technical proficiency in data profiling, analysis, and data management using modern tools and environments, including Python, R, SQL, Spark, and cloud platformsExperience with data quality frameworks, including profiling, rule development, issue remediation, and the design of preventative controlsAbility to translate complex technical findings ...

regulated sectors such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments.Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial.About YouYou will be a technically credible leader who is still comfortable getting into the detail.You will understand ...

both OnPrem and Cloud* Experience with processing large datasets as well as streaming data and unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java ...

MLOps/LLMOps principles, including CI/CD for ML, model monitoring, and governance frameworks.Proficiency with large-scale data processing and storage technologies (SQL, Spark, Hadoop) is a plus.Excellent stakeholder management and communication skills, with proven ability to translate complex AI concepts for diverse audiences.Connect to your business - Technology ...

with experiment tracking and orchestration tools such as clearml, Weights & Biases, MLflow. Experience with distributed computing frameworks and large-scale data processing (e.g., Ray, Spark, Dask)Excellent communication skills to collaborate with researchers and translate research ideas into robust implementationsSelf-driven attitude with genuine curiosity about ML research developmentsComfortable ...

incremental data pipelines leveraging Azure Data Factory for orchestration and Azure Data Lake Storage Gen2 as the primary data store.Independently implement SQL- and Spark‐based transformations to produce curated datasets that support enterprise reporting, analytics, and downstream consumption.Take ownership of assigned data pipelines and datasets, including monitoring, troubleshooting … jobs to ingest, transform, and curate data within the enterprise data platform.Build and modify Azure Data Factory pipelines for batch and incremental data ingestion.Implement Spark‐based transformations that write curated datasets to Azure Data Lake Storage Gen2 using established folder structures and naming conventions.Create and maintain SQL views ...

release-oriented delivery.Able to demonstrate evidence of applying technology to solve business problems.Essential SkillsStrong working knowledge of modern cloud data platforms (Azure Databricks/Spark using Python and associated frameworks, or Snowflake equivalent).Familiarity with how AI/ML and agentic systems are integrated with enterprise data platforms.Strong working … React, Node) using Webpack, Typescript, and GitHub.Experience with Docker and/or Kubernetes and containerized deployments (AKS).Experience with workflow orchestration tools such as Apache Airflow or n8n.Experience of code quality, test, CI/CD and package managers such as Jest, Lint, Brigade, and Helm.Experience with Kafka and event ...

Able to demonstrate evidence of applying technology to solve business problems. Essential Skills Strong working knowledge of modern cloud data platforms (Azure Databricks/Spark using Python and associated frameworks, or Snowflake equivalent). Familiarity with how AI/ML and agentic systems are integrated with enterprise data platforms. … using Webpack, Typescript, and GitHub. Experience with Docker and/or Kubernetes and containerised deployments (AKS). Experience with workflow orchestration tools such as Apache Airflow or n8n. Experience of code quality, test, CI/CD and package managers such as Jest, Lint, Brigade, and Helm. Experience with Kafka ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA’s and repeatable deploymentImplementing metadata, lineage and catalogue capabilities and implementing policy-based ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA's and repeatable deployment Implementing metadata, lineage and catalogue capabilities and implementing policy ...

processing, including vector databases, fine‐tuning LLMs, and deploying LLMs with tools such as HuggingFace, Langchain, and OpenAI. Hands‐on experience working with Distributed Spark based systems. Experience communicating and teaching technical concepts to non‐technical and technical audiences alike. Passion for collaboration, life‐long learning, and driving … values through ML. [Preferred] 2+ years customer‐facing experience in a pre‐sales or post‐sales role. [Preferred] Experience working with Apache Spark to process large‐scale distributed datasets. Can meet expectations for technical training and role‐specific outcomes within 3 months of hire. Can travel ...

Applying advanced techniques in LLMs, agentic systems, vector databases, fine‐tuning, and deployment tools (e.g., HuggingFace, Langchain). Hands‐on experience working with Distributed Spark based systems Experience with data engineering concepts or a good understanding of data engineering concepts Pre-sales or post-sales experience working with external … clients across a variety of industry markets. Minimum of 5+ years of customer‐facing experience would be preferred Preferred: Experience working with Apache Spark (TM) to process large‐scale distributed datasets Communication: Proven ability to communicate and teach complex technical concepts to both technical and non‐technical audiences. ...

support data pipelines, ensuring they are production-ready and meet data quality standards. You will work with technologies such as Python, SQL, and Apache Spark, helping drive analytics and operational decision-making. Ideal candidates possess strong programming skills, a relevant degree, and a collaborative spirit, as you will ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

years' experience delivering enterprise data platforms or analytics solutions. Strong commercial experience delivering Databricks solutions. Consulting or customer-facing delivery experience. Strong understanding of Apache Spark, modern Lakehouse architecture and Unity Catalog. Experience with Azure, AWS or GCP. Experience implementing CI/CD and deployment automation. Excellent communication ...

solving complex problems will be invaluable. You are proficient in SQL and Python, with experience in AWS services such as Athena, S3, Glue, and Spark, and you are adept at handling big data technologies and orchestration tools. If this sounds like you and you're keen to learn more ...

machine learning and data science frameworks. Hands-on experience with large language models, generative AI, prompt engineering, and RAG architectures. Experience with Databricks, MLflow, Spark, and cloud-based AI platforms (Azure preferred). Strong understanding of MLOps, software engineering best practices, and production deployment pipelines. Experience leading technical teams ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

Python or Go.Expertise in constructing highly available and fault-tolerant big data environments in the cloud, utilizing NoSQL and RDMS technologies like Hadoop, Spark, PostgreSQL, and cloud-based management tools such as EMR, Athena, and RDS.Ability to implement systems with a security-first mindset, adhering to firm-wide security ...

Python or Go. Expertise in constructing highly available and fault-tolerant big data environments in the cloud, utilizing NoSQL and RDMS technologies like Hadoop, Spark, PostgreSQL, and cloud-based management tools such as EMR, Athena, and RDS.Ability to implement systems with a security-first mindset, adhering to firm-wide ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing Experience with containerisation (Docker) and scalable data systems (e.g. Spark, Kafka) Nice-to-Have Experience with AWS SageMaker or similar managed ML platforms Background in safety-critical or regulated industries (aerospace, aviation, or similar ...

possibility of making serious money doing something you are passionate about.Follow me on Twitter for more roles and news @SebFintechSytems ENgineer – Security, Hadoop, SPark, debian, SELinux, Debian, puppet, ruby , python, kerberos ...

pgvector.Feature stores including Feast or AWS Feature Store.Docker, Kubernetes, or Amazon ECS.Infrastructure as Code using Terraform or CloudFormation.Big data processing technologies such as Spark or Dask.Experience working within regulated or highly secure environments.What's On OfferWork on meaningful AI programmes supporting critical national security missions.Exposure to cutting-edge Machine ...

such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. ...