11 of 11 Apache Spark Jobs in Bristol

LLMs), and specialized models in Computer Vision and NLP. Big Data Engineering: Design, build, and optimize high-throughput, distributed data pipelines and features using Apache Spark (Scala) and Hive on massive datasets to support model training and inference. Cross-Cloud Execution: Lead the design and deployment … environment. Required Skills and Expertise (10+ Years) 1. Big Data and Cloud Mastery Programming & Big Data: 10+ years of extensive, hands-on experience with Apache Spark, with strong preference for production development using Scala. Deep expertise with Apache Hive for data querying and management. Cloud Proficiency: Demonstrated ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data ...

high-quality, secure, and scalable data solutions — gaining exposure to multiple cloud platforms (AWS, Azure, OCI, GCP) and technologies such as Palantir Foundry, Databricks, Spark, and Airflow. Other times you will be working directly with the client, using your analytical and data skills to support their decision-making. … environment. Eligible for UK Security Clearance (or 5+ years UK residency. Desirable Skills & Exposure Familiarity with data lakes, warehouses, or streaming architectures (e.g. Databricks, Spark, Kafka). Experience with ETL tools, orchestration frameworks, or automation pipelines. Understanding of version control and CI/CD (e.g. GitHub, Azure DevOps). ...

high-quality, secure, and scalable data solutions gaining exposure to multiple cloud platforms (AWS, Azure, OCI, GCP) and technologies such as Palantir Foundry, Databricks, Spark, and Airflow. Other times you will be working directly with the client, using your analytical and data skills to support their decision-making. … Eligible for UK Security Clearance (or 5+ years UK residency . Desirable Skills & Exposure Familiarity with data lakes, warehouses, or streaming architectures (e.g. Databricks, Spark, Kafka). Experience with ETL tools, orchestration frameworks, or automation pipelines. Understanding of version control and CI/CD (e.g. GitHub, Azure DevOps). ...

Data Engineer (Hybrid) Bristol – Spark, S3, Redshift, Lambda, EMR, Glue Avanti Recruitment is working with a rapidly growing technology business in Bristol to recruit a talented AWS Data Engineer to join their team. You will help build and evolve their AWS cloud-based data infrastructure. You’ll work with … scalable pipelines and delivering reliable data to teams across the organisation. In this role, you will develop and maintain ETL and ELT workflows using Apache Spark, Python and AWS Glue, while architecting solutions that make extensive use of AWS services including S3, Redshift, EMR, Lambda, Kinesis, DynamoDB ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

looking for an experienced Senior Data Engineer to assist with building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data … Skillset: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache Kafka, Spark, or Hadoop Familiarity with containerization and orchestration tools such as Docker and Kubernetes. Experience with monitoring and alerting tools such ...

HOLD DV CLEARANCE OUTSIDE IR35 This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This … compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … experience in data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue ...

Engineer Job Location: Remote (UK) Job Type: Full-time permanent Experience: 10 to 12 years Joining: Immediate joiner preferred Mandatory Skills: Scala, AWS, Spark Streaming, DevOps, Bash Scripting. Job Description: Design, develop, and maintain scalable backend services on AWS. Build event-driven architectures using SQS, Kafka, Kinesis, or RabbitMQ. … . Proficiency in SQL and NoSQL databases (DynamoDB, RDS). Experience building and deploying applications on Kubernetes (EKS). Bash Scripting Real time with Spark structured streaming ...