17 of 17 Permanent Apache Spark Jobs in the South East

Experience using Git/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package ...

domain-driven approaches CI/CD for data pipelines, version control discipline, infrastructure-as-code basics (Terraform) Streaming and big data exposure (Kafka, Kinesis, Spark, Flink) is a plus Data quality and observability tooling (Great Expectations, Monte Carlo, Soda, dbt tests) is a plus Engineering & Soft Skills: Strong software ...

consultancy or client delivery setting. Proven experience across data analysis, extraction, transformation, and loading, with hands-on knowledge of relevant technologies such as Spark, Python, Kafka, SQL, Airflow, and cloud-based ETL services. Experience designing schemas for data warehouses and data marts, assessing data quality issues, and integrating quality ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly ...

work closely with stakeholders and translate business requirements into effective AI solutions Desirable Generative AI experience Large Language Models (LLMs) LangChain or similar frameworks Spark, Hadoop or other big data technologies Experience with distributed computing, streaming platforms or workflow orchestration tools Experience monitoring and optimising model performance in production ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

What we’re looking for: Strong Python development and software engineering experience Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments ...

workflows. Key Requirements Passion for building scalable, resilient cloud data platforms. Strong experience with Databricks or Snowflake on AWS. Proven Python skills, including Spark and Airflow expertise. Advanced SQL skills and end-to-end data modelling experience. Experience building batch and real-time data integrations. Hands-on CICD skills ...

backend roles Strong Python; experience with distributed data systems Relational DB experience (PostgreSQL preferred) NoSQLExposure to vector databases (Qdrant, Milvus, pg vector) Bonus: Spark/Airflow/Kafka, cloud (GCP/AWS/Azure), AI/ML platform experience, early-stage startup bac kground Why Join Build the data ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure) is a plus Why join High impact and ownership ...

source models or integrating with managed foundation model platforms. Hands-on experience building data pipelines and model workflows using tools such as Python, SQL, Spark or similar data processing technologies. Experience deploying and operating AI systems in cloud environments using containerisation, managed ML services or serverless architectures. Understanding ...

partner management and ability to influence across teams and organisations. Continuous learning and improvement mentality. Desired Experience with big data technologies (e.g. Hadoop, Hive, Spark). Experience with generative AI, LLMs, or retrieval-augmented generation (RAG). Exposure to Agentic AI concepts, including autonomous agents, tool use, and orchestration ...

platforms (AWS, Azure, or GCP). Distributed Systems Mastery: Deep technical capability in structuring data platforms, data pipelines, and high‐throughput distributed systems (like Spark or Kafka) optimized for performance and scalability. Cross‐Functional Technical Leadership: Exceptional ability to bridge the gap between deep technical engineering and strategic business ...

Strong experience with MLOps, production ML systems, model lifecycle management, and monitoring. Experience working with large-scale data systems and distributed computing frameworks (e.g. Spark, Hadoop). Knowledge of experimental design, scientific methodology, and analysis. Strong partner management and confirmed ability to influence large organisations without direct authority. Demonstrated ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

Data Engineer Location: London (Hybrid) Salary: Up to £75,000 + Bonus + Benefits Technology: SQL, Python, Spark - Azure, AWS, GCP - ETL/ELT - Data Governance The Opportunity Are you a Data Engineer looking for an opportunity to work on large-scale data transformation programmes whilst helping shape enterprise … data lineage. Support Data Governance initiatives, ensuring data security, privacy and compliance standards are maintained. Work with large and complex datasets using SQL, Python, Spark, Hadoop and modern data technologies. Collaborate with technical and business stakeholders within Agile delivery environments. Support data architecture and data modelling activities across enterprise ...