151 to 175 of 197 Apache Spark Jobs in the UK excluding London

secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high … experience as a Data Engineer in secure, regulated, or mission-critical environments Proven expertise with the Elastic Stack (Elasticsearch, Logstash, Kibana) Solid experience with Apache NiFi Strong understanding of data security, governance, and compliance requirements Experience building real-time, large-scale data pipelines Working knowledge of cloud platforms (AWS … with a strong focus on data accuracy, quality, and reliability Desirable (Nice to Have): Background in defence, government, or highly regulated sectors Familiarity with Apache Kafka, Spark, or Hadoop Experience with Docker and Kubernetes Use of monitoring/alerting tools such as Prometheus, Grafana, or ELK Understanding of More ❯

s Degree Required technical and professional expertise Design, develop, and maintain Java-based applications for processing and analyzing large datasets, utilizing frameworks such as Apache Hadoop, Spark, and Kafka. Collaborate with cross-functional teams to define, design, and ship data-intensive features and services. Optimize existing data processing … Technology, or a related field, or equivalent experience. Experience in Big Data Java development. In-depth knowledge of Big Data frameworks, such as Hadoop, Spark, and Kafka, with a strong emphasis on Java development. Proficiency in data modeling, ETL processes, and data warehousing concepts. Experience with data processing languages More ❯

Microservice based architecture - Experience with database Oracle and SQL knowledge - Preferable to Have - exposure to kafka/Hadoop/React JS/ElasticSearch/Spark - Good to Have - have exposure to Fabric/Kubernetes/Dockers/helm More ❯

opportunities and enhancing the platform’s capabilities SKILLS AND EXPERIENCE The successful Senior Data Engineer will have the following skills and experience: Python SQL Spark Snowflake AWS Ideally you will also have: DBT Jenkins Git Tableau BENEFITS The successful Lead Data Engineer will receive the following benefits: Salary between More ❯

generative AI applications in media (e.g., content summarisation, auto-tagging, script generation). Expertise in Python, with knowledge of data engineering tools (e.g., SQL, Spark, Airflow) a plus. Understanding of media workflows, digital publishing, recommendation systems, and audience analytics. Strong stakeholder management and communication skills to translate technical insights More ❯

Experience in building machine learning models for business applications Experience in applied research PREFERRED QUALIFICATIONS Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy, etc. Experience in implementing Computer Vision algorithms Amazon is an equal opportunities employer. We believe passionately that employing a More ❯

Our systems are self-healing, responding gracefully to extreme loads or unexpected input. We use modern languages like Kotlin and Scala, Data technologies Kafka, Spark, MLflow, Kubeflow, VastStorage, StarRocks and agile development practices. Most importantly, we hire great people from around the world and empower them to be successful. More ❯

exposure to Agile methodologies or similar technologies (training provided if necessary). • Advanced Technologies: Knowledge of big data technologies (HBase, Hadoop), machine learning frameworks (Spark), or orbit dynamics is of interest. Why CGI Secure Space Systems? Join a team that's at the forefront of space technology innovation. At More ❯

exposure to Agile methodologies or similar technologies (training provided if necessary). • Advanced Technologies: Knowledge of big data technologies (HBase, Hadoop), machine learning frameworks (Spark), or orbit dynamics is of interest. Why CGI Secure Space Systems? Join a team that's at the forefront of space technology innovation. At More ❯

is ideal for someone who enjoys blending technical precision with innovation. You’ll: Build and manage ML pipelines in Databricks using MLflow, Delta Lake, Spark, and Mosaic AI. Train and deploy generative models (LLMs, GANs, VAEs) for NLP, content generation, and synthetic data. Architect scalable solutions using Azure, AWS More ❯

Experience in building machine learning models for business application - Experience in applied research PREFERRED QUALIFICATIONS - Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc. - Experience in implementing Computer Vision algorithms. Amazon is an equal opportunities employer. We believe passionately that employing a More ❯

and equity financing to mid-market and late-stage companies. Liquidity is backed by leading global financial institutions including Japan’s largest bank, MUFG, Spark Capital, and KeyBank Asset Management. Liquidity offers a dynamic and fast-paced work environment. With an open-door policy and a commitment to high More ❯

TensorFlow, PyTorch, or Scikit-learn for ML, and Pandas, PySpark, or similar for data processing. Experience designing and orchestrating data pipelines with tools like Apache Airflow, Spark, or Kafka. Strong understanding of SQL, NoSQL, and data modeling. Familiarity with cloud platforms (AWS, Azure, GCP) for deploying ML and More ❯

MiFID II or EMIR Experience building real-time applications based on a messaging paradigm Experience building large-scale data processing pipelines (for e.g. using Apache Spark, etc) Experience building Highly Available and High Performance applications Experience of FIX messaging protocol FX Options/Derivatives experience Senior Python Developer More ❯

any environment. On average every day 10,000 new app downloads from the app stores contain our code. Our innovative solutions create meaningful connections, spark real change, and drive success for businesses around the world. If you’re ready to be part of a team shaping the future of More ❯

Data Engineering: Proficiency in developing and maintaining real-time data pipelines. Experience with ETL processes, Python, and SQL. Familiarity with big data technologies like Apache Hadoop and Apache Spark. MLOps & Deployment: Experience deploying and maintaining ML inference pipelines. Proficiency with Docker and Kubernetes. Familiarity with AWS cloud platform. More ❯

quality data solutions. Automation: Implement automation processes and best practices to streamline data workflows and reduce manual interventions. Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python. Good to have: Cloudera - Spark, Hive, Impala, HDFS, Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin. Qualifications: Bachelor's More ❯

ECOM are pleased to be exclusively recruiting for a Senior Data Engineer here in Manchester. You'll join a team where your work reaches millions. This role is within a forward-thinking company leading, offering a dynamic environment where you More ❯

of Data Science Lead the data science strategy and team to deliver data science solutions e.g. retention, acquisitions and customer management using Python and Spark Lead the hiring to build a great pool of Data Scientists and Engineers for the team and support the recruitment activities of other data More ❯

Job Title: Gen AI Architect Duration: Full time Job Description: We are seeking a highly skilled Generative AI Architect to join our dynamic team and drive the development of advanced AI systems that have capabilities to learn, reason and make More ❯

As a Senior BI Developer, you will be at the forefront of creating Analytical Solutions and insights into a wide range of business processes throughout the organisation and playing a core role in our strategic initiatives to enhance data-driven More ❯

and infrastructure. Skills and Experience Required: Proficiency in SQL, Python, Scala, and R. Experience with Big Data technologies such as Microsoft Fabric, Microsoft Synapse, Spark, and Kafka. Familiarity with database management systems including SQL Server and PostgreSQL. Knowledge of data integration and ETL tools (e.g., Talend, Informatica, Apache More ❯

efficient data models for real-time analytics. Proven experience in managing real-time data pipelines across multiple initiatives. Expertise in distributed streaming platforms (Kafka, Spark Streaming, Flink). Experience with GCP (preferred), AWS, or Azure for real-time data ingestion and storage. Strong programming skills in Python, Java, or … Scala . Proficiency in SQL, NoSQL, and time-series databases . Knowledge of orchestration tools (Apache Airflow, Kubernetes). If you are a passionate and experienced Senior Data Engineer seeking a Lead role, or a Lead Data Engineer aiming to make an impact like in a previous position, we More ❯

Informatics, Information Systems, or another quantitative field. They should also have experience using the following software/tools: Experience with big data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow … etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. Salary: 30000 per annum + benefits Apply For This Job If you would like to apply More ❯

With us, you'll do meaningful work from Day 1. Our collaborative culture is built on three core behaviors: We Play to Win, We Get Better Every Day & We Succeed Together. And we mean it - we want you to grow More ❯