1 to 25 of 31 Apache Spark Jobs in the East of England

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … data Billing & revenue assurance Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection , etc. 7. Performance Optimization & Monitoring Tune Spark/Glue jobs for performance and cost. Monitor Redshift/Athena/S3 efficiency and implement best practices. Perform data quality checks and validation ...

such as Kubernetes, Docker, MLflow, Airflow, Terraform, and CI/CD. Preferred Skills Data Streaming and Orchestration: Familiarity with tools such as Kafka, Flink, Spark, dbt, and Airflow. Multi-Modal LLM Systems: Experience with multi-modal LLM systems (text + image embeddings). AI Observability and Evaluation: Exposure ...

such as Kubernetes, Docker, MLflow, Airflow, Terraform, and CI/CD. Preferred Skills Data Streaming and Orchestration: Familiarity with tools such as Kafka, Flink, Spark, dbt, and Airflow. Multi-Modal LLM Systems: Experience with multi-modal LLM systems (text + image embeddings). AI Observability and Evaluation: Exposure ...

such as Kubernetes, Docker, MLflow, Airflow, Terraform, and CI/CD. Preferred Skills Data Streaming and Orchestration: Familiarity with tools such as Kafka, Flink, Spark, dbt, and Airflow. Multi-Modal LLM Systems: Experience with multi-modal LLM systems (text + image embeddings). AI Observability and Evaluation: Exposure ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

also have Experience developing or using AI applications Experience with NoSQL databases (e.g. MongoDB). Experience with big data processing frameworks (e.g. Apache Spark). Knowledge of Large Language Model (LLM) integration, Retrieval-Augmented Generation (RAG), or Agentic AI frameworks. Experience with high-performance computing (HPC) environments (e.g. ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Apply now! Benefits and Contract Information Financial incentives: depending on circumstances, monthly family/marriage allowance of £278 monthly child allowance ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

shared across the team for technical feedback. In This Job, You Will Bring These Skills Skills/Experience - Required: Streaming data systems - using Kafka, Spark, Map/Reduce or similar to process large data sets Experience with building and operating distributed systems and services at a high scale (~billions ...