76 to 99 of 99 Apache Spark Jobs in Yorkshire

to present data insights to diverse audiences. Passionate about sports, bookmaking, and betting, with experience in pricing/optimization and Big Data technologies (e.g. Apache Spark, AWS). What we offer Financial: Competitive salary and bonus schemes. Hybrid working: Our employees can work from home up to More ❯

track record in networks, telecom, or customer experience domains (preferred). Proficiency in cloud platforms like GCP, AWS, or Azure; plus tools like Kafka, Spark, Snowflake, Databricks. Skilled collaborator with excellent stakeholder and vendor management capabilities. Confident communicator with the ability to bridge technical and business audiences. #J More ❯

track record in networks, telecom, or customer experience domains (preferred). Proficiency in cloud platforms like GCP, AWS, or Azure; plus tools like Kafka, Spark, Snowflake, Databricks. Skilled collaborator with excellent stakeholder and vendor management capabilities. Confident communicator with the ability to bridge technical and business audiences. #J More ❯

secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high … experience as a Data Engineer in secure, regulated, or mission-critical environments Proven expertise with the Elastic Stack (Elasticsearch, Logstash, Kibana) Solid experience with Apache NiFi Strong understanding of data security, governance, and compliance requirements Working knowledge of cloud platforms (AWS, Azure, or GCP), particularly in secure deployments Experience … with a strong focus on data accuracy, quality, and reliability Desirable (Nice to Have): Background in defence, government, or highly regulated sectors Familiarity with Apache Kafka, Spark, or Hadoop Experience with Docker and Kubernetes Use of monitoring/alerting tools such as Prometheus, Grafana, or ELK Understanding of More ❯

secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high … experience as a Data Engineer in secure, regulated, or mission-critical environments Proven expertise with the Elastic Stack (Elasticsearch, Logstash, Kibana) Solid experience with Apache NiFi Strong understanding of data security, governance, and compliance requirements Working knowledge of cloud platforms (AWS, Azure, or GCP), particularly in secure deployments Experience … with a strong focus on data accuracy, quality, and reliability Desirable (Nice to Have): Background in defence, government, or highly regulated sectors Familiarity with Apache Kafka, Spark, or Hadoop Experience with Docker and Kubernetes Use of monitoring/alerting tools such as Prometheus, Grafana, or ELK Understanding of More ❯

secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high … experience as a Data Engineer in secure, regulated, or mission-critical environments Proven expertise with the Elastic Stack (Elasticsearch, Logstash, Kibana) Solid experience with Apache NiFi Strong understanding of data security, governance, and compliance requirements Working knowledge of cloud platforms (AWS, Azure, or GCP), particularly in secure deployments Experience … with a strong focus on data accuracy, quality, and reliability Desirable (Nice to Have): Background in defence, government, or highly regulated sectors Familiarity with Apache Kafka, Spark, or Hadoop Experience with Docker and Kubernetes Use of monitoring/alerting tools such as Prometheus, Grafana, or ELK Understanding of More ❯

career progression opportunities across Group, which includes several high profile household name What you'll bring: Experience with Cloud and big data technologies (e.g. Spark/Databricks/Delta Lake/BigQuery). Familiarity with eventing technologies (e.g. Event Hubs/Kafka) and file formats such as Parquet/ More ❯

progression opportunities across the Group, which includes several high-profile household names What you'll bring: Experience with cloud and big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more More ❯

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. You will gain experience in a diverse and exciting set of technologies and clients and have a real impact on Pendo More ❯

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. You will gain experience in a diverse and exciting set of technologies and clients and have a real impact on Pendo More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting … scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and … manage data warehousing solutions, ensuring data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting … scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and … manage data warehousing solutions, ensuring data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong More ❯

Role: Data Architect-GCP Responsibilities include working with Data Products, Data Mesh, ETL, EDW, and technologies such as Spark, Hive, Kafka, and Pub-Sub. The role involves defining Data Product Definition, Architecture, and Roadmap, and leading architectural design delivery. The candidate should support gathering information requirements from IT and More ❯

narrow-right Job Views: 2 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Role: Data Architect-GCP Data Products, Data Mesh, ETL, EDW Spark, Hive, Kafka, Pub-Sub Jira More about the role: Able to define Data Product Definition, Architecture and Roadmap. Define Target Model backlog in collaboration More ❯

and handling large datasets. Job Responsibilities Develop scalable data pipelines for analysts and data scientists. Work hands-on with Data Engineering technologies like Databricks, Spark, Python, SQL Server, Scala. Collaborate with data scientists and engineers to develop and deploy machine learning models. Mentor team members to improve development standards. … designing data solutions on cloud-based distributed systems. Proficiency in Python, SQL, and modern software development practices. Experience with testing automation, data warehousing, IaC, Spark, Databricks, data quality frameworks, and data privacy principles. Knowledge of real-time data ingestion and streaming architectures. Experience within airline, e-commerce, or retail More ❯

Get AI-powered advice on this job and more exclusive features. This range is provided by Anson McCade. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range Direct More ❯

Principal Data Engineer, Consulting Leeds Based You must be eligible for SC Clearance Role Overview The Principal Data Engineer will be responsible for designing and implementing cloud-based data solutions using a range of AWS services. This role involves working More ❯

Principal Data Engineer, Consulting Leeds Based You must be eligible for SC Clearance Role Overview The Principal Data Engineer will be responsible for designing and implementing cloud-based data solutions using a range of AWS services. This role involves working More ❯

policies and standards. Lead Architecture Committee governance approvals for artefacts. Mandatory Skills :- GCP, Data Products, Data Mesh, ETL, EDW, Composer, Big Query, DataProc, DBT, Spark, Hive, Kafka, Pub-Sub, Jira, GCP BigQuery, GCP Dataflow, BigTable, Python, Kafka, Cloud DataProc, Cloud Data Flow, Cloud Data Fusion, GCS, Cloud Pub/… Sub, Cloud functions, BigTable, Python, Scala, Java, .NET, ANSI SQL, Spark, Airflow, ShellScripts, Control-M, Git, CI & CD, HDFS, Unix File System, RDBMS, Azure Devops, Harness More ❯

policies and standards. Lead Architecture Committee governance approvals for artefacts. Mandatory Skills :- GCP, Data Products, Data Mesh, ETL, EDW, Composer, Big Query, DataProc, DBT, Spark, Hive, Kafka, Pub-Sub, Jira, GCP BigQuery, GCP Dataflow, BigTable, Python, Kafka, Cloud DataProc, Cloud Data Flow, Cloud Data Fusion, GCS, Cloud Pub/… Sub, Cloud functions, BigTable, Python, Scala, Java, .NET, ANSI SQL, Spark, Airflow, ShellScripts, Control-M, Git, CI & CD, HDFS, Unix File System, RDBMS, Azure Devops, Harness More ❯

to join its innovative team. This role requires hands-on experience with machine learning techniques and proficiency in data manipulation libraries such as Pandas, Spark, and SQL. As a Data Scientist at PwC, you will work on cutting-edge projects, using data to drive strategic insights and business decisions. … e.g. Sklearn) and (Deep learning frameworks such as Pytorch and Tensorflow). Understanding of machine learning techniques. Experience with data manipulation libraries (e.g. Pandas, Spark, SQL). Git for version control. Cloud experience (we use Azure/GCP/AWS). Skills we'd also like to hear about More ❯

and scalability, including understanding and explaining features like KRAFT. Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink, and Beam. Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. Implementing security … open source Kafka Focus on disaster recovery aspects Knowledge of Kafka resiliency and new features like KRAFT Experience with real-time technologies such as Spark Required Skills & Experience Extensive experience with Apache Kafka and real-time architecture including event-driven frameworks. Strong knowledge of Kafka Streams, Kafka Connect … Spark Streaming, Schema Registry, Flink, and Beam. Experience with cloud platforms such as GCP Pub/Sub. Excellent problem-solving skills. Knowledge & Experience/Qualifications Knowledge of Kafka data pipelines and messaging solutions to support critical business operations and enable real-time data processing. Monitoring Kafka performance, enhancing decision More ❯

maintaining Kafka clusters to ensure high availability and scalability. Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming Schema Registry, Flink and Beam. Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. Implementing security … guidance and support to development operations teams. Staying updated with the latest Kafka features, updates and industry practices. Required Skills Experience : Extensive Experience with Apache Kafka and real-time architecture including event driven frameworks. Strong Knowledge of Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink and Beam. More ❯

maintaining Kafka clusters to ensure high availability and scalability. Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming Schema Registry, Flink and Beam. Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. Implementing security … guidance and support to development operations teams. Staying updated with the latest Kafka features, updates and industry practices. Required Skills Experience : Extensive Experience with Apache Kafka and real-time architecture including event driven frameworks. Strong Knowledge of Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink and Beam. More ❯