101 to 125 of 138 Apache Spark Jobs in England

requires a candidate with 8+ years of software and data engineering experience, focusing on API development using Spring Boot and big data technologies like Apache Spark and Airflow. Responsibilities include designing scalable systems, collaborating with cross-functional teams, and mentorship. Candidates should have strong programming skills, SQL expertise ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

Understanding of common data structures and algorithms. Experience working with open-source Data‐Science environments. Knowledge of open source big‐data technologies such as Apache Spark. Experience building solutions that run in the cloud, ideally Azure. Experience with software development methodologies including Scrum & Kanban. Working patterns ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

looking for engineers and leaders across: AI Engineering (LLMs, RAG, Applied AI systems) Machine Learning Engineering (model development, optimisation, deployment) Data Engineering (Spark, Databricks, cloud data platforms) Full Stack Software Engineering (end-to-end system ownership) DevOps/Platform Engineering (CI/CD, Kubernetes, cloud infrastructure) Low-Level/ ...

Strong SQL and database knowledge Experience working with large-scale or high-volume datasets Exposure to big data or distributed systems technologies such as Spark or Kafka Previous experience within financial services is beneficial but not essential Strong academic background, ideally in Computer Science What ...

with predictive and diagnostic analytics in CLTV, risk scoring, financial crime or marketing Technical proficiency in SQL databases and tools like Python, R, or Spark Skilled in visualisation and dashboard development using Power BI, Qlik or Tableau Solid background in Financial Services, with experience in Capital Markets, Insurance ...

patterns and contribute to shared accelerators. What we’re looking for Hands‐on LLMs, embeddings, vector databases; retrieval strategies and grounding. Data engineering fundamentals (Spark/Databricks; ETL/ELT) and operational handover. Consulting behaviours: stakeholder engagement and clear, client‐ready communication. Nice to have Document/graph stores ...

Analytics Engineer London, hybrid 3 times a week Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilise programming expertise … experience in data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue ...

e.g., Grafana, Humio) and chaos engineering. Familiarity with leveraging Generative AI (e.g., GitHub Copilot, Gemini) to accelerate development. Experience with big data technologies like Apache Spark or Amazon S3. If indicated, please note that years of experience are a guide; we will consider applications from all candidates ...

owning the complete machine learning lifecycle. Ideal candidates will have an advanced degree in Computer Science or similar, proficiency in Python, and experience with Spark and DevOps technologies. Benefits include private health insurance and a flexible hybrid work model. #J-18808-Ljbffr ...

software engineer to be responsible for owning and maintaining their production machine learning and delivery platform. The role involves designing large-scale Databricks/Spark pipelines and leading technical improvements. Applicants should have extensive experience in software engineering, particularly with Python and data pipeline integration. A passion for digital ...

design and develop APIs and Big Data pipelines. The role demands deep expertise in API development via Spring Boot and knowledge of Apache Spark. You will work closely with cross-functional teams to deliver robust data solutions. The ideal candidate has over 10 years of professional experience in software ...

tooling and working with large transactional datasets • Understanding of OTC products including CDS, Interest Rate Swaps and Variance Swaps Highly Desirable • Experience with C++, Spark, Kafka or other distributed compute tools • Exposure to position keeping, risk, or PnL systems • Strong debugging, profiling and optimisation capability across data pipelines ...

What Others Only Talk About London (Hybrid) We have partnered with a market-leading, global energy software business that’s assembling an elite AI “Spark Team” . The Mission Design and deliver cutting-edge AI solutions that actually move the needle – not just prototypes that sit on a shelf. ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

Data Engineer Salary up to £70,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via ABOUT THE CLIENT Due to continued growth, we're seeking a skilled Data Engineer ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

governed and used across the organisation. Key Responsibilities: Maintain, improve and optimise an existing Databricks data platform. Build and enhance scalable data pipelines using Spark, SQL and Python or Scala. Develop and mature Bronze, Silver and Gold layers to support analytics, reporting and downstream data use. Ingest and integrate … What We're Looking For: Proven experience as a Data Engineer within a cloud-based data environment. Strong hands-on experience with Databricks and Apache Spark. Experience building or improving medallion architecture, including Bronze, Silver and Gold data layers. Strong SQL skills for transformation, validation and data analysis. Python ...

Deep experience with cloud platforms (Azure, AWS, or GCP) and cloud-native architecture Strong background in data engineering and large-scale data platforms (e.g., Spark, Databricks) Experience implementing MLOps frameworks, including CI/CD and model governance Experience integrating AI into microservices, APIs, and enterprise systems Exposure to advanced … similar programming languages) TensorFlow, PyTorch, or similar ML frameworks Cloud AI services (Azure AI, AWS SageMaker, GCP AI Platform) Data technologies (SQL/NoSQL, Spark, Databricks) API and microservices architecture DevOps/MLOps tooling Desirable Experience Experience with Generative AI solutions (e.g. Azure OpenAI, GPT-based models) Microsoft stack ...

opportunities for remote or on-site engagement, diverse training, and contributions to a sustainable open-source product. Strong hands-on experience with tools like Apache Spark, Kubernetes, and Terraform is essential. #J-18808-Ljbffr ...

Strong hands-on experience wi th AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience wi th real-time and batch data process ing — Apache Flink, Kafka, Spark, or similar Proficiency in one or more o f: Python, Java, Scala Experience wi th data modell ing, ETL processes ...