26 to 50 of 69 Apache Spark Jobs in Central London

be technically skilled in most or all of the below: - Expert knowledge of Python and SQL, inc. the following libraries: Numpy, Pandas, PySpark and Spark SQL - Expert knowledge of ML Ops frameworks in the following categories: a) experiment tracking and model metadata management (e.g. MLflow) b) orchestration of ML More ❯

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew of More ❯

for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy … throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages such as SQL, Python, R … 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure More ❯

systems Strong knowledge of Kubernetes and Kafka Experience with Git, and Deployment Pipelines Having worked with at least one of the following stacks: Hadoop, Apache Spark, Presto AWS Redshift, Azure Synapse or Google BigQuery Experience profiling performance issues in database systems Ability to learn and/or adapt More ❯

Kibana/Grafana/Prometheus). Write software using either Java/Scala/Python. The following are nice to have, but not required - Apache Spark jobs and pipelines. Experience with any functional programming language. Writing and analysing SQL queries. Application overVIOOH Our recruitment team will work hard More ❯

of data governance, security, and regulatory compliance Strong leadership skills with the ability to influence senior stakeholders Hands-on coding experience with SQL, Python, Spark, or similar technologies Experience working in a large-scale enterprise environment with highly distributed data ecosystems Desirable skills/knowledge/experience: Azure/ More ❯

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning More ❯

proficiency - from writing queries to basic database operations Exposure to cloud infrastructure, ideally AWS Bonus: Experience with streaming/data tools such as Kafka, Spark, or Delta Lake Technology Stack & Environment: You'll be working with: AWS, Kubernetes, Kafka, Argo Java 17, Python 3, HDF5 Distributed systems processing large More ❯

CD processes and infrastructure-as-code. • Eligible for SC clearance (active clearance highly advantageous). Desirable Skills • Exposure to large data processing frameworks (e.g., Spark, Hadoop). • Experience deploying data via APIs and microservices. • AWS certifications (Solution Architect Associate, Data Analytics Speciality, etc.). • Experience in public sector programmes More ❯

proficiency - from writing queries to basic database operations Exposure to cloud infrastructure, ideally AWS Bonus: Experience with streaming/data tools such as Kafka, Spark, or Delta Lake Technology Stack & Environment You'll be working with: AWS, Kubernetes, Kafka, Argo Java 17, Python 3, HDF5 Distributed systems processing large More ❯

outcomes. Strong coding ability with SQL and Python, as well as experience working with data orchestration tools like Airflow or Dataform. Commercial experience with Spark and Databricks. Familiarity with leading integration and data platforms such as Mulesoft, Talend, or Alteryx. A natural ability to mentor others and provide technical More ❯

UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with clients across industries. What You’ll … in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, data lakes, and data warehouses. Hands-on experience with Apache Spark and bonus points for Microsoft Fabric Any certifications are a bonus. Benefits: Competitive base salary Hybrid work once a week into their More ❯

UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with clients across industries. What You’ll … in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, data lakes, and data warehouses. Hands-on experience with Apache Spark and bonus points for Microsoft Fabric Any certifications are a bonus. Hybrid work once a week into their Central London office Learning More ❯

frameworks, and performance monitoring Desirable Skills: Background in serverless technologies (e.g. Lambda, Step Functions, API Gateway) Experience with data tools like EMR, Glue, or Apache Spark Understanding of event-driven architecture (EventBridge, SNS, SQS) Knowledge of AWS database offerings including DynamoDB and RDS Familiarity with multi-region deployments More ❯

IoT or related fields. Proficient in programming languages such as Python and SQL. Experience with big data tools and platforms (e.g. Hadoop/Snowflake, Spark, Azure IoT Hub, AWS IoT). Please apply to the advert for more information. More ❯

Deep expertise in machine learning, NLP, and predictive modelling. Proficient in Python or R, cloud platforms (AWS, GCP, Azure), and big data tools (e.g. Spark). Strong business acumen, communication skills, and stakeholder engagement. If this role looks of interest, please apply here. Please note - this role cannot offer More ❯

. Experience in using cloud-native services for data engineering and analytics. Experience with distributed systems, serverless data pipelines, and big data technologies (e.g., Spark, Kafka). Ability to define and enforce data governance standards. Experience in providing architectural guidance, mentorship and leading cross-functional discussions to align on More ❯

GCP, AWS or Azure), especially managed ML services like SageMaker or Vertex AI Proficiency in SQL and working knowledge of distributed computing tools like Spark or Dask Nice to Have: Experience with natural language processing (NLP) e.g., using LLMs, transformers, text classification Familiarity with Graph ML (e.g., for customer More ❯

. Experience in using cloud-native services for data engineering and analytics. Experience with distributed systems, serverless data pipelines, and big data technologies (e.g., Spark, Kafka). Ability to define and enforce data governance standards. Experience in providing architectural guidance, mentorship and leading cross-functional discussions to align on More ❯

of experience • Working experience in Palantir Foundry platform is must • Experience designing and implementing data analytics solutions on enterprise data platforms and distributed computing (Spark/Hive/Hadoop preferred). • Proven track record of understanding and transforming customer requirements into a best-fit design and architecture. • Demonstrated experience … in end-to-end data management, data modelling, and data transformation for analytical use cases. • Proficient in SQL (Spark SQL preferred). • Experience with JavaScript/HTML/CSS a plus. Experience working in a Cloud environment such as Azure or AWS is a plus. • Experience with Scrum/ More ❯

cloud integration Guide and mentor technical teams on best practices Requirements: Strong experience as a Solution Architect in Databricks and cloud platforms Expertise in Apache Spark, Delta Lake, and data lakehouse architectures Hands-on with Azure, AWS, or GCP (Azure preferred) Excellent client-facing and stakeholder management skills More ❯

Proven experience in recommender systems, behavioural AI, and/or reinforcement learning. Building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). PhD in Computer Science, Machine Learning, or a closely related field What We Offer: Opportunity to build technology More ❯

track record in networks, telecom, or customer experience domains (preferred). Proficiency in cloud platforms like GCP, AWS, or Azure; plus tools like Kafka, Spark, Snowflake, Databricks. Skilled collaborator with excellent stakeholder and vendor management capabilities. Confident communicator with the ability to bridge technical and business audiences. London – Hybrid More ❯

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew of More ❯

system full-link data fortification implementation experience. Familiarity with at least one language, such as Java, Go, or Python, and knowledge of Hadoop/Spark ecosystems in the big data domain. Preference for candidates with multinational corporate experience in data security and over 3 years of experience in international More ❯