16 of 16 Permanent Apache Spark Jobs in Gloucestershire

ready data processing solutions using BigQuery, Pub/Sub, Dataflow and DataprocExperience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam.Expert understanding of when to use a range of data storage technologies including relational/non-relational, document ...

skills, with experience of Java or Scala beneficial Experience building large-scale ETL/ELT and data processing pipelines Hands-on experience with Apache Spark and distributed data processing technologies Experience designing and operating cloud-native data platforms, particularly within AWS Familiarity with Docker, Kubernetes and CI/ ...

ResponsibilitiesData Pipeline DevelopmentData Platform ManagementStakeholder CollaborationLeadership & MentorshipTechnology StackThe successful candidate will have experience working with some or all of the following technologies:Data Processing: Apache Spark/PySparkWorkflow Orchestration: Apache AirflowContainerisation: KubernetesCloud Platform: Microsoft AzureData APIs & Semantic Layer: CubeJSProgramming Languages: Python, SQLEssential Skills & ExperienceDegree in a STEM ...

Palantir Foundry, such as Pipeline Builder or AIP, or similar platforms for application development and data integrationExperience with distributed data and compute, such as Spark or Databricks, Hadoop or Hive, Kafka, or AWS EMR, and cloud environments used by the ICExperience with geospatial analysis tools, such as ArcGIS ...

Proficiency in the Python data science ecosystem (e.g. NumPy, Pandas, scikit-learn, PyTorch, TensorFlow). Experience with big data and distributed analytics platforms (e.g. Spark, Hadoop). Hands-on experience with Natural Language Processing, graph analytics and advanced analytical methods. Experience using cloud analytics platforms, with strong ...

robust data models, ETL pipelines, metadata frameworks, and modern data warehousing solutions. Design and optimise large-scale streaming data platforms using technologies such as Apache Flink and other real-time processing frameworks. Ensure all solutions comply with industry best practices, governance frameworks, and regulatory requirements, maintaining the highest standards … Lambda, Redshift , and related data services. Proven experience designing secure architectures within sensitive or highly regulated environments. Hands-on experience with Apache Flink and real-time data processing architectures. Strong understanding of data modelling methodologies, architectural patterns, and industry standards. Expertise across ETL processes, data integration, data warehousing, distributed ...

large language model (LLM) systems, including retrieval augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, so data is high quality, time-stamped, and ready for both training and real-time inference. Design and deploy data collection systems ...

large language model (LLM) systems, including retrieval-augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, ensuring data is high quality, time-stamped and ready for both training and real-time inference. Design and deploy data collection systems ...

development of algorithms leveraging R, Python, or SQL/NoSQL Experience with distributed data and computing tools, including MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL Experience with visualization packages, including Plotly, Seaborn, or ggplot2 TS/SCI clearance with a polygraph Clearance:Applicants selected will be subject ...

. Experience with Natural Language Processing (NLP) and Large Language Models (LLMs). Experience in areas of model development, data processing and streaming (Spark, Kafka), microservices in python (Flask or FastAPI), and interactive visualisations and User Interfaces (Streamlit, Plotly, Gradio etc). These lists are not exhaustive ...

Azure Data Lake Storage.Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server.Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing.Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks.Knowledge of data quality, data governance, metadata management … scale data solutions.Collaborative working environment with opportunities to contribute to continuous improvement initiatives.If you are an experienced Data Engineer with expertise in Azure, Databricks, Spark, SQL, Scala and Python, apply now to develop your career while delivering innovative data solutions. ...

scale.Familiarity with personalization and named-entity evaluation challenges in speech systems.Experience with multilingual or international audio dataset development.Experience with distributed data processing frameworks (e.g., Spark) for large-scale audio dataset generation.Publication record or demonstrated contributions in speech, audio ML, or NLP evaluation.Pay & BenefitsAt Apple, base pay is one part ...

Automated system deployment using tools such as Ansible, Puppet or ChefDesirable Skills/Experience• Familiar with the Big Data tech stacks, including: o Trinoo Spark o Apache Hiveo Apache Airflowo Zeppelin notebooksOur PackageTailored benefits make a real difference. That’s why we offer a comprehensive range ...

implement robust data models, ETL pipelines, metadata frameworks, and modern data warehousing solutions.Design and optimise large-scale streaming data platforms using technologies such as Apache Flink and other real-time processing frameworks.Ensure all solutions comply with industry best practices, governance frameworks, and regulatory requirements, maintaining the highest standards … services, including EC2, S3, RDS, Lambda, Redshift, and related data services.Proven experience designing secure architectures within sensitive or highly regulated environments.Hands-on experience with Apache Flink and real-time data processing architectures.Strong understanding of data modelling methodologies, architectural patterns, and industry standards.Expertise across ETL processes, data integration, data warehousing ...

Java, C++ and experience building distributed systems.Experience building data processing pipelines and large scale machine learning systems with experience in big data technologies like Spark, SQL, Snowflake/Hadoop, etc.Skilled in communication, problem solving, critical thinking Attention to detail, data accuracy and quality of output.Preferred QualificationsPhD or Graduate degree … transformer architecture.Skilled in communication, problem solving, critical thinking.Experience using Deep Learning, Bandits, Probabilistic Graphical Models, or Reinforcement Learning in real applications a plus.Experience with Spark, TensorFlow, Keras, and PyTorch a plusPay & BenefitsAt Apple, base pay is one part of our total compensation package and is determined within a range. ...

experience building highly scalable distributed systemsHands-on experience with building data processing pipelines, large scale machine learning systems, and big data technologies (eg: Spark, SQL, Snowflake/Hadoop, etc)Bachelors in a quantitative field, such as Computer Science, Applied Mathematics, Statistics, or Bachelors degree in quantitative field with … including RAG based Generative AI and transformer architectureExperience using Deep Learning, Bandits, Probabilistic Graphical Models, or Reinforcement Learning in real applications a plusExperience with Spark, TensorFlow, Keras, and PyTorch a plusSkilled in communication, problem solving, strategic thinkingPay & BenefitsAt Apple, base pay is one part of our total compensation package ...