1 to 25 of 389 Apache Spark Jobs in the UK

will be working on complex data problems in a challenging and fun environment using some of the latest Big Data open-source technologies like Apache Spark as well as Amazon Web Service technologies including Elastic MapReduce Athena and Lambda to develop scalable data solutions. • Adhering to Company Policies … we're looking for: The ability to problem-solve. Knowledge of AWS or equivalent cloud technologies. Knowledge of Serverless technologies frameworks and best practices. Apache Spark (Scala or Pyspark) Experience using AWS CloudFormation or Terraform for infrastructure automation. Knowledge of Scala or 00 language such as Java or … skills positive attitude willing to help other members of the team. Experience debugging and dealing with failures on business-critical systems. Preferred: Exposure to Apache Spark Apache Trino or another big data processing system. Knowledge of streaming data principles and best practices. Understanding of database technologies and More ❯

Location: Remote-first (UK-based) 💰 Rate: Up to £550 p/d 📆 Contract: 6 - 12 months (Outside IR35) 🛠 Tech Stack: Python, FastAPI, GCP, BigQuery, Apache Spark, Apache Beam, Google Cloud Dataflow We're working with a forward-thinking consultancy that helps top companies build and scale high … You’ll Be Doing: 🔹 Building data pipelines and ETL workflows that process huge datasets 🔹 Designing, optimizing, and maintaining high-throughput reporting solutions 🔹 Working with Apache Spark for large-scale data processing 🔹 Using Apache Beam and Google Cloud Dataflow to manage complex data workflows 🔹 Developing and improving backend … writing clean, efficient, and scalable code ✔ Experience with BigQuery, PostgreSQL, and Elasticsearch ✔ Hands-on experience with Google Cloud, Kubernetes, and Terraform ✔ Deep understanding of Apache Spark for large-scale data processing ✔ Knowledge of Apache Beam & Google Cloud Dataflow for data pipeline orchestration ✔ A team-first mindset with More ❯

performance and responsiveness. Stay Up to Date with Technology: Keep yourself and the team updated on the latest Python technologies, frameworks, and tools like Apache Spark , Databricks , Apache Pulsar , Apache Airflow , Temporal , and Apache Flink , sharing knowledge and suggesting improvements. Documentation: Contribute to clear and … or Azure . DevOps Tools: Familiarity with containerization ( Docker ) and infrastructure automation tools like Terraform or Ansible . Real-time Data Streaming: Experience with Apache Pulsar or similar systems for real-time messaging and stream processing is a plus. Data Engineering: Experience with Apache Spark , Databricks , or … similar big data platforms for processing large datasets, building data pipelines, and machine learning workflows. Workflow Orchestration: Familiarity with tools like Apache Airflow or Temporal for managing workflows and scheduling jobs in distributed systems. Stream Processing: Experience with Apache Flink or other stream processing frameworks is a plus. More ❯

experienced Head of Data Engineering to lead data strategy, architecture, and team management in a fast-paced fintech environment. This role involves designing scalable Apache Spark , Databricks , and Snowflake solutions on Azure , optimizing ETL/ELT pipelines , ensuring data security and compliance, and driving innovation in big data … team of data engineers, fostering a culture of innovation, collaboration, and best practices. Big Data Processing: Architect, optimize, and manage big data solutions leveraging Apache Spark, Databricks, and Snowflake to enable real-time and batch data processing. Cloud Data Infrastructure: Oversee the deployment and maintenance of Azure-based … 8+ years in data engineering, with at least 3+ years in a leadership role within fintech or financial services. Strong hands-on experience with Apache Spark, Databricks, Snowflake, and Azure Data Services (Azure Data Lake, Azure Synapse, etc.). Deep understanding of distributed computing, data warehousing, and data More ❯

Location: London Hybrid working Full Time Contract Long term engagement Job Summary We are seeking a highly skilled Data Engineer with expertise in Databricks, Apache Spark, and Scala/Python to join our dynamic team in London on a contingent worker basis. The ideal candidate will have hands … implementing robust data pipelines and ensuring compliance with data governance best practices. Key Responsibilities: Develop, optimize, and maintain big data pipelines using Databricks and Apache Spark . Write efficient, scalable, and maintainable Scala/Python code for data processing and transformation. Collaborate with data architects, analysts, and business … issues. Document technical processes, best practices, and governance policies. Key Requirements: 5+ years of experience in data engineering with a focus on Databricks and Apache Spark . Strong programming skills in Scala and/or Python . Experience or a strong understanding of data governance, metadata management, and More ❯

Data Platform Engineer - SSIS & T-SQL, Data Factory - Hybrid Data Platform Engineer SSIS & T-SQL, Data Factory, Databricks/Apache Spark London Insurance Market City, London/Hybrid (3 days per week in the office) Permanent £85,000 per annum + benefits + bonus PLEASE ONLY APPLY IF … data function in a London Market Insurance setting Sound understanding of data warehousing concepts ETL/ELTs - SSIS & T-SQL, Data Factory, Databricks/Apache Spark Data modelling Strong communication skills and able to build relationships and trust with stakeholders Data Platform Engineer SSIS & T-SQL, Data Factory … Databricks/Apache Spark London Insurance Market City, London/Hybrid (3 days per week in the office) Permanent £85,000 per annum + benefits + bonus More ❯

Our team values continuous learning, knowledge sharing, and creating inclusive solutions that make a difference. Key Responsibilities Support customers with big data services including Apache Spark, Hive, Presto, and other Hadoop ecosystem components Develop and share technical solutions through various communication channels Contribute to improving support processes and … work week schedule, which may include weekends on rotation. BASIC QUALIFICATIONS - Good depth of understanding in Hadoop Administration, support and troubleshooting (Any two applications: Apache Spark, Apache Hive, Presto, Map-Reduce, Zookeeper, HBASE, HDFS and Pig.) - Good understanding of Linux and Networking concepts - Intermediate programming/scripting More ❯

culture. Key Responsibilities Design, build, and maintain scalable data solutions to support business objectives. Work with Microsoft Fabric to develop robust data pipelines. Utilise Apache Spark and the Spark API to handle large-scale data processing. Contribute to data strategy, governance, and architecture best practices. Identify and … approaches. Collaborate with cross-functional teams to deliver projects on time . Key Requirements ? Hands-on experience with Microsoft Fabric . ? Strong expertise in Apache Spark and Spark API . ? Knowledge of data architecture, engineering best practices, and governance . ? DP-600 & DP-700 certifications are highly More ❯

learning libraries in one or more programming languages. Keen interest in some of the following areas: Big Data Analytics (e.g. Google BigQuery/BigTable, Apache Spark), Parallel Computing (e.g. Apache Spark, Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Data Storytelling with (Jupyter) Notebooks More ❯

learning libraries in one or more programming languages. Keen interest in some of the following areas: Big Data Analytics (e.g. Google BigQuery/BigTable, Apache Spark), Parallel Computing (e.g. Apache Spark, Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Data Storytelling with (Jupyter) Notebooks More ❯

learning libraries in one or more programming languages. Keen interest in some of the following areas: Big Data Analytics (e.g. Google BigQuery/BigTable, Apache Spark), Parallel Computing (e.g. Apache Spark, Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Data Storytelling with (Jupyter) Notebooks More ❯

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one. Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals. Familiarity with CI/CD for production deployments. Working knowledge of MLOps. Design and deployment … data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake, and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of More ❯

companies where years-long behemoth projects are the norm, our projects are fast-paced, typically 2 to 4 months long. Most are delivered using Apache Spark/Databricks on AWS/Azure and require you to directly manage the customer relationship alone or in collaboration with a Project … at DATAPAO, meaning that you'll get access to Databricks' public and internal courses to learn all the tricks of Distributed Data Processing, MLOps, Apache Spark, Databricks, and Cloud Migration from the best. Additionally, we'll pay for various data & cloud certifications, you'll get dedicated time for … seniority level during the selection process. About DATAPAO At DATAPAO, we are delivery partners and the preferred training provider for Databricks, the creators of Apache Spark. Additionally, we are Microsoft Gold Partners in delivering cloud migration and data architecture on Azure. Our delivery partnerships enable us to work in More ❯

high-performance data pipelines, enable advanced analytics, and drive business value through data Key Responsibilities: Build and maintain scalable data pipelines using Databricks and Apache Spark . Develop efficient data models and implement Lakehouse architecture (bronze/silver/gold layers). Optimise data processing workflows for performance … and CI/CD practices. Monitor, troubleshoot, and continuously improve data workflows in production environments. Essential Skills & Experience: Hands-on experience with Databricks and Apache Spark . Proficiency in PySpark and SQL . Strong understanding of data lakehouse concepts , data modelling, and data warehousing. Familiarity with one or More ❯

is made up of a series of components providing the next generation valuation and risk management services. Responsibilities Development of big data technologies like Apache Spark and Azure Databricks Programming complex production systems in Scala, Java, or Python Experience in a platform engineering role on a major cloud … development, build, and runtime environments including experience of Kubernetes Salary - 140-150K/year SKILLS Must have Development of big data technologies like Apache Spark and Azure Databricks Programming complex production systems in Scala, Java, or Python Experience in a platform engineering role on a major cloud More ❯

to non-technical and technical audiences alike. Passion for collaboration, life-long learning, and driving business value through ML. Preferred Experience working with Databricks & Apache Spark to process large-scale distributed datasets. About Databricks Databricks is the data and AI company. More than 10,000 organizations worldwide - including … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive More ❯

driving business value through ML Company first focus and collaborative individuals - we work better when we work together. Preferred Experience working with Databricks and Apache Spark Preferred Experience working in a customer-facing role About Databricks Databricks is the data and AI company. More than 10,000 organizations … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of More ❯

driving business value through ML Company first focus and collaborative individuals - we work better when we work together. Preferred Experience working with Databricks and Apache Spark Preferred Experience working in a customer-facing role About Databricks Databricks is the data and AI company. More than 10,000 organizations … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive More ❯

Greetings, We currently have an urgent opening for a Senior Java Developer (Spark) with more than 8 Years of experience at Synechron in Belfast, UK. Job Role: - Senior Java Developer (Spark) Job Location- Belfast, UK About Company: At Synechron, we believe in the power of digital to transform … with 8-12 years of hands-on commercial development experience. Proficient in understanding and working with distributed systems, specifically technologies such as Hadoop and Apache Hive. Mandatory experience with Apache Spark for big data processing. Familiarity with streaming technologies, particularly Kafka, is a plus. Experience in the … are interested in this opportunity, kindly send your updated profile along with the required details listed below. Total Experience Experience in Java Experience in Spark, Hadoop/Apache Hive, Kafka: - Current CTC:- Expected CTC:- Notice period:- Current Location:- Ready to relocate to Belfast UK: - Visa Status:- Passport Validity More ❯

years of hands-on experience with big data tools and frameworks. Technical Skills: Proficiency in SQL, Python, and data pipeline tools such as Apache Kafka, Apache Spark, or AWS Glue. Problem-Solving: Strong analytical skills with the ability to troubleshoot and resolve data issues. Communication: Excellent communication More ❯

or product feature use cases. Experience in building and deploying live software services in production. Exposure to some of the following technologies (or equivalent): Apache Spark, AWS Redshift, AWS S3, Cassandra (and other NoSQL systems), AWS Athena, Apache Kafka, Apache Flink, AWS, and service-oriented architecture. More ❯

AWS Certified Data Analytics - Specialty or AWS Certified Solutions Architect - Associate. Experience with Airflow for workflow orchestration. Exposure to big data frameworks such as Apache Spark, Hadoop, or Presto. Hands-on experience with machine learning pipelines and AI/ML data engineering on AWS. Benefits: Competitive salary and More ❯

experience working with relational and non-relational databases (e.g. Snowflake, BigQuery, PostgreSQL, MySQL, MongoDB). Hands-on experience with big data technologies such as Apache Spark, Kafka, Hive, or Hadoop. Proficient in at least one programming language (e.g. Python, Scala, Java, R). Experience deploying and maintaining cloud More ❯

Databricks. Solid understanding of ETL processes , data modeling, and data warehousing. Familiarity with SQL and relational databases. Knowledge of big data technologies , such as Spark, Hadoop, or Kafka, is a plus. Strong problem-solving skills and the ability to work in a collaborative team environment. Excellent verbal and written More ❯

to the experimentation and development of new ad products at Yelp. Design, build, and maintain efficient data pipelines using large-scale processing tools like Apache Spark to transform ad-related data. Manage high-volume, real-time data streams using Apache Kafka and process them with frameworks like … Apache Flink. Estimate timelines for projects, feature enhancements, and bug fixes. Work with large-scale data storage solutions, including Apache Cassandra and various data lake systems. Collaborate with cross-functional teams, including engineers, product managers and data scientists, to understand business requirements and translate them into effective system … a proactive approach to identifying opportunities and recommending scalable, creative solutions. Exposure to some of the following technologies: Python, AWS Redshift, AWS Athena/Apache Presto, Big Data technologies (e.g S3, Hadoop, Hive, Spark, Flink, Kafka etc), NoSQL systems like Cassandra, DBT is nice to have. What you More ❯