176 to 200 of 297 Apache Spark Jobs in England

statistics, machine learning, and mathematical modelling. Experience designing evaluation frameworks for AI systems, including model performance and monitoring. Hands-on experience with Databricks, MLflow, Spark, or similar cloud-based data platforms. Knowledge of responsible AI, model governance, and explainability. Strong communication skills with the ability to explain complex technical ...

Strong commercial experience in data engineering and modern data platforms. Advanced programming skills in Java or Python. Expertise working with technologies such as Kafka, Spark, Flink or similar distributed data frameworks. Experience delivering solutions within AWS or other major cloud platforms. Strong understanding of data architecture, scalability and performance ...

serving training/inference, focusing on cost efficiency, performance, and resilience. Big data & distributed processing: Develop and tune large-scale batch/stream workloads (Spark or similar); improve job efficiency, stability, and operational readiness in production. Software engineering: Strong Python + SQL; apply CI/CD, automated testing, modular ...

Python. Experience with Kubernetes and Docker. Understanding of CI/CD pipelines (e.g. GitLab). Exposure to data platforms, MLOps or machine learning environments. Spark or Scala. AWS services (e.g. S3). Elasticsearch or graph databases. OIDC/OAuth. Node.js or React. #J-18808-Ljbffr ...

Qlik). Knowledge of Python or R for advanced analytics and automation is highly desirable. Familiarity with cloud/big data platforms (Snowflake, Databricks, Spark) is preferred. Key Requirements Minimum 5+ years of hands-on experience in Data Analytics or Data Science (experience within Customer Success, Operations, or Digital ...

Data; prior experience with Tier-1 Cloud Service Providers (CSPs) is highly preferred. Expertise in the Hadoop ecosystem and deep understanding of components like Spark, Flink, Elasticsearch, Doris, ClickHouse, and Trino., Strong knowledge of big data architecture, including Data Lake (Iceberg, Hudi) and Serverless data concepts. Familiarity with mainstream ...

implementing process improvements and automation Strong Python development skills (including Pandas and related data-processing libraries). Experience with big data technologies such as Spark, PySpark, Hadoop, and Hive Exposure to quantitative modelling or financial modelling is a plus but not required. Skills that will help: Global Risk Management ...

track record of driving technical strategy across teams—not just executing within one. Valuable ExperienceDistributed storage systems (e.g. Ceph)Big data infrastructure: Hadoop, Spark, HBase, Kafka. Observability stack design: Prometheus, Grafana, ELK, Mimir, Loki, Tempo. Secrets management (Vault), certificate management, access control at scale. Hybrid cloud architectures: federating public ...

Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access control, data quality and compliance standards * Optimise performance, scalability and cloud cost management * Lead and mentor a small team … Required experience * Strong experience as a senior or lead data engineer building and scaling production data platforms * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows * Python and SQL proficiency * Infrastructure as code experience using Terraform, Bicep or CloudFormation * Cloud data platform experience, ideally Azure ...

bronze/silver/gold layers, and partner with analytics, ML, and trading teams to deliver trusted data assets. You will work with dbt, Spark, Airflow, Kafka, and streaming technologies to ensure data quality, governance, and efficient storage. #J-18808-Ljbffr ...

Candidates Must Have: 4 years commercial experience with Linux CentOS , bare metal and virtualisation Puppet configuration management or equivalent automation experience Big data (Hadoop, Spark, Kafka) Containers and scheduling experience (Docker, LXC, Kubernetes, Mesos) Strong scripting You will be joining a progressive and exciting company committed to excellence. They ...

powered services into production. Strong Python and Java, both of which are in active use across the team's production services. Experience with Snowflake, Spark, Databricks or similar technologies, CI/CD pipelines, and modern DevOps tooling. Solid understanding of SecOps practices and security‐conscious system design. Demonstrable track ...

machine learning, data mining, information retrieval, and statisticsUnderstanding of machine learning frameworks: Tensorflow, Pytorch, pyG, Keras, MXNet, Scikit-LearnStrong programming knowledge of Python and Spark; proficiency in vector operations using numpy, scipy; experience with distributed computation using multithreading, multi GPUs, Dask, Ray, PolarsFamiliarity with AWS Cloud services such ...

DatabricksFamiliarity in Cloud-based application development (we use AWS & Azure)Familiarity with one or more ML frameworks and technologies: scikit-learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, TritonStrong communication skills, able to express ideas clearly and collaborate across teamsAdditional InformationBring all of you to workWe create ...

data mining, information retrieval, and statistics Understanding of machine learning frameworks: Tensorflow, Pytorch, pyG, Keras, MXNet, Scikit-Learn Strong programming knowledge of Python and Spark; proficiency in vector operations using numpy, scipy; experience with distributed computation using multithreading, multi GPUs, Dask, Ray, Polars Familiarity with AWS Cloud services such ...

leader, and/or technical program manager. Data Engineering Proficiency : Expertise in SQL, Python, Pandas, Matlab, or R. Bonus if familiarity with Influx, Grafana, Spark, Polars, Arrow, Kafka, Beam, and Flink. [Bonus] In-Field Hardware Mastery : Proficiency with hardware engineering tools such as DAQs, DDS, Nix, RTOS, MCAP ...

their techniques * Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make ...

retrieval, and multi‐modal representation learning. Technical Stack: Mastery of Python and deep learning frameworks (TensorFlow, PyTorch) alongside hands‐on experience with distributed computing (Spark, Ray) and cloud infrastructure (AWS/GCP). Graph Neural Networks (GNNs): Strong experience applying GNNs, knowledge graphs, or graph embeddings to map complex ...

retrieval, and multi-modal representation learning. Technical Stack: Mastery of Python and deep learning frameworks (TensorFlow, PyTorch) alongside hands-on experience with distributed computing (Spark, Ray) and cloud infrastructure (AWS/GCP). Graph Neural Networks (GNNs): Strong experience applying GNNs, knowledge graphs, or graph embeddings to map complex ...

Engineer, maintaining and further developing a well-established cloud and CI/CD setup to implement and evolve reliable, well-tested data pipelines in Spark/Scala and to extend the dimensional data models. You will be part of the technical team collaborating closely with platform engineering, data science … ingesting from various sources into Databricks Collaborate closely with software engineers to evolve our software platform and address technical challenges Design, develop, and optimize Spark/Scala jobs for ingestion and transformation, using the existing GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests ...

Roku in Manchester is looking for a Senior Data Engineer to join the Viewer Product Device & Themed Experiences team. You will design data models and build scalable pipelines across Roku devices, apps, and web/ ...

equivalent) and modern DevOps practices. Hands-on experience designing and implementing real-time and batch data processing pipelines using modern frameworks like Spark, Flink, Spark Streaming, Kafka, Debezium, etc. Experience with orchestration tools such as Airflow, dbt, or similar. Exceptional problem-solving skills paired with a proactive, innovative ...

ensure data availability, quality, and efficiency across markets. This role is onsite in London and requires strong SQL and programming skills, plus experience with Spark, Hadoop, and cloud platforms. #J-18808-Ljbffr ...

ensure requirements are understood before design and development. The role focuses on data warehouse and large‐scale data processing using Big Data technologies (Hadoop, Spark), with programming in Scala and Python. You will guide developers, implement best practices, and drive automated testing and CI/CD to production. #J ...

stakeholders. Architecture design informed through the use of principles, patterns, technical radars, practices and standards. AWS Solution expertise and relevant certification - EMR/Spark (Hadoop)/AWS/Data Modelling-Mapping/GitHub/GitLab Technologies Azure AWS Power BI #J-18808-Ljbffr ...