24 of 24 Apache Spark Jobs in the South East

capabilities across the organization. Key Responsibilities Design and implement data architectures on the Databricks Lakehouse Platform Build and optimize ETL/ELT pipelines using Apache Spark and Databricks Implement Delta Lake and medallion architecture (Bronze, Silver, Gold) Ensure data governance, security, and performance optimization Integrate Databricks with cloud … stakeholders to deliver scalable data solutions Required Skills & Qualifications Databricks Certification (e.g., Databricks Certified Data Engineer Associate/Professional) Strong experience with Databricks, Apache Spark, and Delta Lake Proficiency in Python, SQL, and data pipeline development Experience with cloud platforms (Azure, AWS, or GCP) Solid understanding of modern ...

quality and reliable deliverables. Optionally contribute to on-call platform support where required. What We’re Looking For Technical Expertise Strong experience with Databricks, Apache Spark, and cloud-based data architectures (preferably Azure). Excellent programming capability in Python and SQL. Experience orchestrating pipelines with Azure Data Factory ...

applications. Knowledge of MLOps, DevOps, CI/CD, and automated model pipelines within Foundry and cloud-based environments. Familiarity with big data ecosystems (Foundry, Spark, Hadoop) and commercial data source integration. ...

Vault . Hands-on experience exposing Lakehouse solutions to BI tools (e.g. Power BI , Tableau ). Proficiency in Databricks Workflows , Delta Live Tables , and Spark for batch and streaming pipelines. Strong programming capability in Python , Scala , and/or SQL . Practical experience with at least one major cloud ...

Skills & Experience* Proven experience delivering enterprise-scale Databricks solutions end-to-end.* Strong background in Lakehouse Architecture, including structured and unstructured data.* Expertise in Spark, PySpark, Delta Lake, and Databricks workflows.* Experience building scalable ETL/ELT pipelines, including Delta Live Tables.* Strong programming skills in Python, Scala ...

Salesforce data ingestion experience Exposure to NoSQL databases such as MongoDB, Cassandra, or DynamoDB Experience supporting Power BI and Tableau reporting Proficiency in Python, Spark, or similar data engineering tools Understanding of AI and ML trends Strong communication and stakeholder management skills ...

explain technical concepts to varied audiences. Nice to Have Cloud experience: AWS/Azure/GCP . Containerisation: Docker, Kubernetes . Distributed processing: Apache Spark/Flink . Please apply within for further details - Matthew Holmes, Harvey Nash Always use these settings ...

Kafka Exposure to cloud platforms (AWS, Azure, or GCP) Familiarity with containerisation technologies (Docker, Kubernetes) Experience with distributed data processing tools such as Spark or Flink Additional Information Daily rate: £850 - £900 Hybrid working model in London Initial contract with strong likelihood of extension McGregor Boyall is an equal ...

common ML frameworks such as PyTorch, TensorFlow, or scikit-learn * Exposure to ML/MLOps tooling such as SageMaker, MLflow, or TFServing * Experience with Spark, Databricks, CI/CD tools, and streaming or orchestration systems like Kafka or Airflow How to Apply If you are interested in this Machine ...

influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). ...

Kubernetes Strong Git-based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities … React frontend Build and maintain Dockerised microservices deployed to AWS (EKS) Troubleshoot and optimise performance across the stack Support data ingestion and transformation pipelines (Spark-based) Work cross-functionally with product and data specialists Help shape new features and influence engineering standards This role is 100% hands-on engineering ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation … design, build, and maintain scalable data pipelines for batch and streaming data using modern frameworks. Experience working with Databricks for big data processing, Spark-based transformations, and collaborative analytics workflows. Familiarity with Azure Data Factory (ADF) for data orchestration and integration across cloud environments. Skilled in version control using ...

will be working across different client setups. Typically, that includes: AWS, Azure and GCP depending on the project Tools like Databricks, Snowflake and Synapse Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or Kinesis You will use your expertise to choose the right ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

/ELT) pulling from core platforms, cloud storage, databases and APIs Turn raw, messy, disparate data into curated, analysis-ready datasets Work with Python, Spark and SQL every day Develop models and schemas using a medallion architecture approach Help shape our data architecture with the Data Architect Implement governance … solve a problem you'll fit right in. What you'll bring You'll likely have: Hands-on expertise with Databricks Confidence in Python, Spark and SQL Experience building pipelines in Azure A solid understanding of data modelling and scalable architectures But just as important: You're inclusive ...

data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark. Very useful - Python, SQL, JSON, LakeFS. Reasonable ...

experience building APIs at scale ⚙️ You’ll need at least 2 of the below: ☁️ AWS (3+ years) – S3, Lambda, DynamoDB 📊 Big Data (3+ years) – Spark, Snowflake, Kafka 📦 Containers (3+ years) – Kubernetes/OpenShift 💳 Payments experience (2–3 years) – card payments/fintech/transaction systems 👉 We’re being strict ...

candidates with experience of one or more of the major cloud platforms. AWS, Azure and GCP depending on the project Databricks, Snowflake and Synapse Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or Kinesis You're expected to pick the right tools ...

with EKS) Proficiency in Git-based development workflows Experience working independently and communicating effectively in distributed teams Desirable : Familiarity with (or happy to learn): Apache Spark (EMR) and Delta Lake PostgreSQL (Aurora) CI/CD (Github Actions) and IaC (Helm, CloudFormation) Monitoring (Sentry) Knowledge/interest in research … improve the TypeScript/React-based frontend application. Build and manage containerised services deployed on AWS, utilising Kubernetes orchestration. Support data and transformation workflows (Spark) Diagnose, troubleshoot, and improve overall application performance. Handle and resolve live production issues efficiently and under time pressure. Collaborate closely with product and data ...

data engineering projects. Responsibilities Design, build and deliver data solutions using Databricks Develop and optimise ETL/ELT data pipelines using Spark, Python and SQL Build and maintain data lakes and data warehousing solutions Collaborate with stakeholders to deliver scalable and insight-driven data platforms Contribute to projects involving … Snowflake, Azure Data Factory, Synapse or Microsoft Fabric Skills and Experience Strong experience with Databricks with either Azure or AWS Strong SQL, Python and Spark expertise Proven track record delivering data engineering solutions across full project lifecycles Strong understanding of data modelling and modern data architectures Ability to engage ...