25 of 25 Apache Spark Jobs in the South East

prem data platform aligned to the metals strategy while developing and maintaining scalable data pipelines and analytics infrastructure. Using Hadoop, Big Data, and Spark technologies, you will ensure data quality through automated validation, monitoring, and testing. You will also enable seamless integration across data warehouses and data lakes, contributing … detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such ...

about other roles, please contact Nick Mandella at Harnham. KEYWORDS Python, SQL, AWS, GCP, Azure, Cloud, Databricks, Docker, Kubernetes, CI/CD, Terraform, Pyspark, Spark, Kafka, machine learning, statistics, Data Science, Data Scientist, Big Data, Artificial Intelligence, private equity, finance. ...

experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best practices ...

experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best practices ...

data engineering, data platforms and big data technologies Evaluate and improved existing data processes Essential Experiences: Advanced SQL skills Proficient in Databricks, including Spark, Delta Lake and MLflow Experience migrating SSIS to Databricks In depth understanding of ETL/ELT processes Knowledge of data warehousing concepts and architectures Able ...

transformation. Proficiency in Python and PySpark for advanced data engineering, machine learning, and large-scale distributed processing within Databricks. Hands-on experience with Spark optimizations (Adaptive Query Execution, caching, partitioning, broadcast joins) to deliver performant solutions for big data workloads. Advanced knowledge of BI and visualization tools such ...

teams Strong background in cloud data platforms and modern data architectures Hands-on expertise with Python , data modelling, ETL, and big data technologies (e.g. Spark) Experience operating in regulated or complex enterprise environments Excellent stakeholder management and communication skills Comfortable working with ambiguity and driving change Nice to have ...

DevOps and broader software engineering standards. Cloud experience (any platform). Previous mentoring experience. Nice to have: Snowflake or Databricks Spark, PySpark, Hadoop or similar big data tooling BI exposure (PowerBI, Tableau, etc.) Interview Process The process is fully structured, transparent, and efficient: Video call - high-level overview ...

ServiceNow Automation, Automation Anywhere, Camunda, or similar). Proven experience in AI/ML, data modeling, data lakes, event streaming, and analytics (e.g., Kafka, Spark, Python ML stack). Deep exposure to telecom protocols and domains Experience with cloud-native architectures (Azure/AWS/GCP), containerization, and DevSecOps ...

architectures using Delta Lake principles Ingest, transform, and curate data from multiple sources (APIs, databases, files, streaming) Develop scalable data transformations using PySpark and Spark SQL Implement data models optimized for analytics and reporting (e.g. star schemas) Monitor, troubleshoot, and optimize performance and cost of data workloads Apply data … version control (Git) and collaborative development workflows Nice to Have Experience with Power BI and semantic models Exposure to streaming technologies (Kafka, Event Hubs, Spark Structured Streaming) Infrastructure-as-code experience (Bicep, Terraform) CI/CD tooling (Azure DevOps, GitHub Actions) Familiarity with data governance and cataloging tools Experience ...

engineers, setting technical standards and driving best practices across the organisation. The role demands hands-on expertise with Azure, big data technologies like Spark and Hadoop, and programming languages including Python. You will also oversee database architecture, data modelling, and ETL processes to ensure seamless integration across multiple platforms ...

Azure based Production Data & Analytics ETL/integration solutions (Fabric, Databricks, Synapse, Data Factory, Function Apps, Power BI etc) against SLAs Python, PySpark, Scala, Spark SQL Ability to identify and investigate complex problems and implement resolutions Direct contact with customers/end-users on the phone and by email ...

Azure Services Understanding of software architecture, design patterns, and agile delivery Experience with geospatial data, REST APIs, JSON and spatial query engines (e.g. Apache Spark) Experience with Generative AI technologies including Large Language Models (LLMs), Retrieval Augmented Generation (RAG) and Prompt/Context Engineering Passion for continuous improvement ...

record designing, scaling, and maintaining distributed systems and high-volume data pipelines. Strong background in cloud-native architectures and modern data stacks such as Spark or Databricks. Experience working closely with Data Science teams, including delivering production-grade ML models and pipelines. Solid understanding of modern frontend engineering practices ...

complex, data-driven platform used by enterprise customers. You will set technical direction and guide architecture across distributed systems, large-scale data processing (Scala, Spark, Databricks), machine learning pipelines, and cloud-native infrastructure on AWS. A core focus is driving high-quality execution through strong engineering practices including ...

innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation … design, build, and maintain scalable data pipelines for batch and streaming data using modern frameworks. Experience working with Databricks for big data processing, Spark-based transformations, and collaborative analytics workflows. Familiarity with Azure Data Factory (ADF) for data orchestration and integration across cloud environments. Skilled in version control using ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

starter approach, and strong communication skills Geographical flexibility to move to any Parker location both during and upon completion of the program. Did we spark your interest? Then put your career in motion as soon as possible! Simply click Apply Now and attach a copy of your CV. NextSteps ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

Documentation. Your Profile Essential Skills/Knowledge/Experience Strong working and technical knowledge of GCP (GCS Buckets, BigQuery, Dataproc, Pub/Sub), Airflow, Spark, PySpark, Spark SQL, Hive. Good knowledge of Python (Pandas, NumPy, Wiresafe), SQL, Linux CLI, BigQuery. Experience with DBMS, Delta Tables, Oracle, MS Access ...

team engineering organisations Strong background in distributed systems, large-scale data pipelines and ML model deployment Deep understanding of modern data stacks such as Spark, Databricks or cloud-native architectures Solid understanding of modern frontend engineering, DevOps knowledge and experience delivering machine learning systems Strong communication, organisation and stakeholder ...

team engineering organisations Strong background in distributed systems, large-scale data pipelines and ML model deployment Deep understanding of modern data stacks such as Spark, Databricks or cloud-native architectures Solid understanding of modern frontend engineering, DevOps knowledge and experience delivering machine learning systems Strong communication, organisation and stakeholder ...

scale Hadoop clusters across on-prem environments Supporting and optimising our HDFS-based Data Lake and adjacent platform components Working with technologies including Hadoop, Spark, HDFS, YARN, Hive, Oozie , and related tooling Partnering with engineering teams to deliver scalable, high-quality data solutions Contributing to our cloud migration strategy … objectives Ensuring stability, performance, and security of critical data services What We're Looking For Strong hands-on experience with Hadoop ecosystem tools (HDFS, Spark, Hive, Oozie, YARN, etc.) Experience operating and supporting large, complex, distributed clusters Understanding of data engineering principles and data platform design Experience with ...

Head of Engineering - Remote - UK - £80,000-£150,000 GenAI, ML, Spark Why Join This Company? This is a leadership role for someone who wants to work in a truly AI-first engineering environment, not a business experimenting at the edges. The company is already well ahead … solving rather than legacy ways of working. You'll Work With Cross-functional teams spanning: Web Engineering (React/TypeScript) Data Engineering (distributed systems, Spark-based platforms) Data Science & Machine Learning A highly technical CTO who is deeply invested in AI adoption and modern engineering Product leadership following significant ...