12 of 12 Apache Spark Jobs in the South West

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g. ...

share knowledge with team members. Required Qualifications Proven experience in designing and building data pipelines (ETL/ELT) using modern technologies (Python, SQL, Spark, Airflow, etc.). Strong analytical skills with the ability to interpret complex data and deliver business value. Experience integrating data from multiple sources and systems. ...

model lifecycle management Experience with NLP, deep learning, or computer vision Knowledge of data visualisation tools (Power BI, Tableau) Exposure to big‐data technologies (Spark, Hadoop) AWS certifications Experience supporting government or highly regulated environments Clearance Requirements: Clearance to Start BPSS Clearance for Role SC Applicants must have ...

solutions while remaining hands-on. You should have: Proven experience building production-ready solutions on Google Cloud Expertise with batch and streaming frameworks like Apache Spark or Beam Strong understanding of data storage, pipeline patterns, and event-driven architectures Experience with CI/CD, version control, automated testing ...

MLOps, model serving, scaling and monitoring workflows (e.g., BentoML, MLflow, Vertex AI, AWS Sagemaker) Experience building streaming + batch data ingestion and transformation pipelines (Spark/Airflow/Beam) Mobile and front-end web application development experience What We Offer At Newpage, we’re building a company that works ...

Requirements: Technical expertise in designing, building, and maintaining data pipelines and warehouses Experience with ETL/ELT frameworks and Big Data Processing Tools (e.g., Spark, Airflow, Hive) Knowledge of programming languages (e.g., Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in a STEM field ...

team engineering organisations Strong background in distributed systems, large-scale data pipelines and ML model deployment Deep understanding of modern data stacks such as Spark, Databricks or cloud-native architectures Solid understanding of modern frontend engineering, DevOps knowledge and experience delivering machine learning systems Strong communication, organisation and stakeholder ...

Months About the Role: Design, develop, and maintain scalable microservices using Java and Spring Boot. Build and optimize real-time data pipelines leveraging Apache Kafka, Flink, and Spark/Databricks. Develop robust data distribution and streaming solutions for high-throughput systems. Deploy, manage, and monitor services in containerized … Kafka and distributed streaming systems. Solid understanding of SQL and data modeling. Experience with containerization (Docker) and orchestration (Kubernetes). Working knowledge of Flink, Spark, or Databricks for data processing. Familiarity with AWS services (ECS, EKS, S3, Lambda, etc.). Basic scripting in Python for automation or data manipulation. ...

Description:- About the Role: Design, develop, and maintain scalable microservices using Java and Spring Boot. Build and optimize real-time data pipelines leveraging Apache Kafka, Flink, and Spark/Databricks. Develop robust data distribution and streaming solutions for high-throughput systems. Deploy, manage, and monitor services in containerized … Kafka and distributed streaming systems. Solid understanding of SQL and data modeling. Experience with containerization (Docker) and orchestration (Kubernetes). Working knowledge of Flink, Spark, or Databricks for data processing. Familiarity with AWS services (ECS, EKS, S3, Lambda, etc.). Basic scripting in Python for automation or data manipulation. ...

MLOps and model serving platforms such as BentoML, MLflow, or SageMaker . Hands-on work with streaming and batch data pipelines using tools like Spark, Airflow, or Apache Beam . Background in front-end or mobile development , enabling end-to-end solution thinking. Prior experience in healthcare, life ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...