6 of 6 Apache Spark Jobs in Newcastle upon Tyne

deliver reliable, high-performance data solutions using Databricks and modern cloud technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake Build and optimize ETL/ELT workflows for batch and streaming data Develop data models to support analytics, reporting … data pipelines, architectures, and best practices Required Qualifications 3+ years of experience in data engineering or a similar role Strong experience with Databricks and Apache Spark Proficiency in Python and SQL Experience with Delta Lake, data modelling, and performance tuning Hands-on experience with cloud platforms (AWS, Azure ...

+ 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform, who are hiring … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such ...

PyTorch) into real customer value Working closely with engineering teams using TypeScript, Node.js, React, and Next.js Partnering with data teams working across AWS, Kafka, Spark, and modern data pipelines Shaping products built on secure, scalable fintech architecture (Event-driven, microservices, serverless) Running discovery experiments and validating concepts at speed ...

best practices. Required Skills and Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data Factory … Data Engineer Associate/Professional, Azure Data Engineer Associate). * Experience with Azure Synapse SQL or serverless SQL pools. * Familiarity with streaming technologies (e.g., Spark Structured Streaming, Kafka, Event Hub). * Experience with infrastructure-as-code (Terraform or Bicep). * Background in BI or analytics engineering (Power ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

production systems and deliver measurable business impact. Develop APIs and services that enable ML-driven advertising solutions across Expedia Group's global platform. Optimize Spark-based applications for large-scale data processing and ensure system reliability and performance. Implement strategies for training models on massive datasets using distributed computing … deploying ML pipelines in production environments. Strong programming skills in Python and at least one other language (e.g., Scala or Java). Expertise in Spark and distributed data processing frameworks. Proficiency with ML libraries such as PyTorch and TensorFlow. Experience with cloud platforms (AWS, EMR, Kubernetes, Docker ...