1 to 25 of 46 Permanent Apache Spark Jobs in the South East

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services (AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud … Storage). Exposure to workflow orchestration tools (Apache Airflow, Prefect, or Dagster). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best practices ...

experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best practices ...

data engineering, data platforms and big data technologies Evaluate and improved existing data processes Essential Experiences: Advanced SQL skills Proficient in Databricks, including Spark, Delta Lake and MLflow Experience migrating SSIS to Databricks In depth understanding of ETL/ELT processes Knowledge of data warehousing concepts and architectures Able ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

TypeScript, JavaScript Machine Learning: TensorFlow, PyTorch, Scikit-learn GenAI/LLMs: OpenAI API, Hugging Face Transformers, LangChain, Langraph, RAG pipelines Data & Analytics: Pandas, NumPy, Spark, Databricks, Hadoop, Vector Databases, Snowflake Model Deployment: MLflow, FastAPI, Flask, Docker, Kubernetes Cloud & Infrastructure: AWS, Azure, GCP CI/CD pipelines (GitLab, Jenkins) Desirable ...

optimise Cypher queries for performance and scalability • Integrate Neo4j Aura with cloud-native applications (REST, GraphQL, event-driven systems) • Manage data ingestion pipelines (Kafka, Spark, ETL tools, APIs) • Monitor, tune, and troubleshoot Neo4j Aura performance and availability • Implement security, access control, and data governance best practices • Collaborate with product ...

teams Strong background in cloud data platforms and modern data architectures Hands-on expertise with Python , data modelling, ETL, and big data technologies (e.g. Spark) Experience operating in regulated or complex enterprise environments Excellent stakeholder management and communication skills Comfortable working with ambiguity and driving change Nice to have ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

Data Analytics Sales Executive - Data Sales, Talend, Snowflake, Apache Spark, Kafka, Dataops, AWS, Azure, Data visualisation, SQL, ETL, Pre-Sales, Account Manager, SaaS Sales, Cloud Solutions, Digital Transformation, Business Development, Sales Consultant, CXO Engagement, Strategic Accounts, Deal Negotiation, Remote Working, UK-wide, Permanent Role, Sales Targets, New Business ...

engineers, setting technical standards and driving best practices across the organisation. The role demands hands-on expertise with Azure, big data technologies like Spark and Hadoop, and programming languages including Python. You will also oversee database architecture, data modelling, and ETL processes to ensure seamless integration across multiple platforms ...

models, transformers, retrieval models). Hands-on experience with LLMs and GenAI technologies. Strong programming and problem-solving skills with proficiency in Python, SQL, Spark, and Hive. Deep understanding of classical and modern ML techniques, A/B testing methodologies, and experiment design. Solid background in ranking, recommendation ...

extracting, manipulating, and preparing data from relational databases. Experience with NoSQL databases is a plus. Big Data/Cloud: Experience with big data tools (Spark, Hadoop) and cloud computing platforms (AWS, Azure, or GCP) for scalable ML workflows. Visualization: Ability to create clear, compelling data visualizations using tools like ...

/CD, Git-based workflows, and automation Strong problem-solving skills and a collaborative mindset Bonus Terraform or other IaC tools Slurm, Kueue, Ray, Spark, or similar systems GPU tooling (CUDA, Nvidia operators, schedulers) Experience supporting ML training or data science teams ...

record designing, scaling, and maintaining distributed systems and high-volume data pipelines. Strong background in cloud-native architectures and modern data stacks such as Spark or Databricks. Experience working closely with Data Science teams, including delivering production-grade ML models and pipelines. Solid understanding of modern frontend engineering practices ...

containerised workloads (Docker, Kubernetes, Kubeflow, etc.) and integrating with CI/CD tools (Jenkins, GitHub Actions, GitLab CI). Familiarity with distributed computing frameworks (Spark, Ray, TensorFlow Distributed, PyTorch Distributed). Strong understanding of monitoring, logging, and observability for large-scale ML systems. Experience in cost optimisation for compute ...

/multi-region patterns, meeting defined RPO/RTO targets. Hands-On Engineering Write and review production-grade code using: Python, Scala, SQL Spark/AWS Glue AWS Lambda & Step Functions Build infrastructure using Terraform (IaC). Implement CI/CD pipelines (GitLab, Jenkins). Enforce automated testing, performance ...