76 to 100 of 101 Apache Spark Jobs in the UK

experience Term structure modelling, stochastic calculus or Monte Carlo methods Interest rate curve bootstrapping Algorithmic execution experience Data lake or warehouse experience (Snowflake, Iceberg, Spark etc.) A product mindset - you think about who uses your systems and why. Clear communicator who can work cross-functionally with other quants, analysts ...

engineering practices (clean code, testing, version control, code review)Proficient with large-scale data: strong SQL skills and familiarity with distributed data processing (e.g., Spark, Hive) for building training datasets, features, and analytical viewsLeadership & CollaborationProven ability to lead through influence: aligning cross-functional stakeholders on problem definitions, success metrics ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. A dedicated ...

doingDesigning and building scalable data pipelines that support machine learning, analytics and data science initiatives. Developing high-performance data solutions using Python, Spark, Databricks and Azure technologies. Creating and maintaining datasets that power customer, marketing, pricing and forecasting use cases. Collaborating closely with Applied Scientists and Machine Learning Engineers … engineering solutions within cloud-based environments. Strong expertise in Python and experience developing production-grade data pipelines and data products. Experience working with Databricks, Spark and distributed processing technologies. Experience designing and operating large-scale data platforms using Azure, AWS or GCP.Deep understanding of data modelling, data architecture ...

building and deploying machine learning models in production, alongside strong hands-on Python skills. You should be comfortable with distributed computing tools such as Spark or PySpark, cloud platforms and modern MLOps practices. Experience managing and developing data scientists is essential, as is the ability to explain technical decisions ...

complex SQL queries over large-scale datasets.Pipeline & Workflow Orchestration: Proven track record building, scheduling, and monitoring production pipelines using modern workflow orchestrators such as Apache Airflow.Systems & Data Architecture Mindset: Experience handling high-throughput event data and a deep understanding of analytical databases, storage formats, and execution plans.Ownership & Pragmatic Autonomy … contracts and schemas.Nice to Have:Hands-on experience with modern data transformation tools like dbt.Familiarity with distributed compute and storage platforms (e.g., Kafka, Spark, Snowflake, or Presto/Trino).Experience in product-led SaaS, fintech, or high-growth tech environments navigating complex event tracking or onboarding funnels (prior FinCrime ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming).Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

person attendance expected at least 3 days per week. At Robinhood, we believe in the power of in-person work to accelerate progress, spark innovation, and strengthen community. Our office experience is intentional, energizing, and designed to fully support high-performing teams. Requires participation in an on-call rotation ...

Python along with proficiency in SQLExperience building data platforms on cloud services like Databricks on Azure or GCP BigQuery and big data technologies like Spark, Flink etc.,Solid grasp of Kubernetes, CI/CD and TerraformWorking knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbtAbility … Python along with proficiency in SQLExperience building data platforms on cloud services like Databricks on Azure or GCP BigQuery and big data technologies like Spark, Flink etc.,Solid grasp of Kubernetes, CI/CD and TerraformWorking knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbtAbility ...

hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/… pairing and design sessions Confidence communicating with technical and non-technical stakeholders, and challenging constructively Desirable: SQL Server or on-prem to cloud migration, Apache Flink, regulated industry experience If you'd rather shape a platform than inherit a finished one, and want a seat at the table while ...

hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/… pairing and design sessions Confidence communicating with technical and non-technical stakeholders, and challenging constructively Desirable: SQL Server or on-prem to cloud migration, Apache Flink, regulated industry experience If you'd rather shape a platform than inherit a finished one, and want a seat at the table while ...

Senior Data Scientist - Python/Machine Learning Permanent Location: UK - Remote Salary: £65,000 - £70,000 (+ benefits) Skills: Python, Machine Learning, Statistical Modelling, AWS, Data Engineering An exciting opportunity has arisen for an experienced ...

work collaboratively and autonomously. Strong problem-solving skills with a pragmatic and analytical outlook. Desirable: Familiar with analytics tools such as Power BI, Apache Superset, or similar for building interactive and impactful visualizations Familiarity with distributed systems (e.g., Spark, Kafka) and how they support scalable analytics solutions Passion ...

Title: Quantexa Data Engineer (Scala/Spark) Location: London Contract Length: Up to 11 months Working Pattern: 2-3 days in the office per week, via Umbrella Company Our client is investing in a major Quantexa-led financial crime and AML transformation programme and is looking for a hands … enhance a new Quantexa capability across EMEA. Key Responsibilities: Design, build and optimise scalable data pipelines and ETL processes. Develop solutions using Scala, Spark, Python and SQL. Support the implementation and growth of the Quantexa platform. Build reusable frameworks and shared engineering capabilities. Work across Azure, Databricks and cloud ...

Title: Quantexa Data Engineer (Scala/Spark) Location: London Contract Length: Up to 11 months Working Pattern: 2-3 days in the office per week, via Umbrella Company Our client is investing in a major Quantexa-led financial crime and AML transformation programme and is looking for a hands … enhance a new Quantexa capability across EMEA. Key Responsibilities: Design, build and optimise scalable data pipelines and ETL processes. Develop solutions using Scala, Spark, Python and SQL. Support the implementation and growth of the Quantexa platform. Build reusable frameworks and shared engineering capabilities. Work across Azure, Databricks and cloud ...

technical decisions and play a key role in shaping the future of the engineering function.ð» Skills required:Strong experience with PythonExcellent SQL knowledgeExperience with Apache technologies (Spark, Airflow or Kafka)Proven experience designing and building scalable data pipelinesStrong understanding of data architecture and distributed systemsExperience working within ...

Title: Quantexa Data Engineer (Scala/Spark) Location: London Contract Length: Up to 11 months Working Pattern: 2-3 days in the office per week, via Umbrella Company Our client is investing in a major Quantexa-led financial crime and AML transformation programme and is looking for a hands … enhance a new Quantexa capability across EMEA. Key Responsibilities: Design, build and optimise scalable data pipelines and ETL processes. Develop solutions using Scala, Spark, Python and SQL. Support the implementation and growth of the Quantexa platform. Build reusable frameworks and shared engineering capabilities. Work across Azure, Databricks and cloud ...

Title: Quantexa Data Engineer (Scala/Spark) Location: London Contract Length: Up to 11 months Working Pattern: 2-3 days in the office per week, via Umbrella Company Our client is investing in a major Quantexa-led financial crime and AML transformation programme and is looking for a hands … enhance a new Quantexa capability across EMEA. Key Responsibilities: Design, build and optimise scalable data pipelines and ETL processes. Develop solutions using Scala, Spark, Python and SQL. Support the implementation and growth of the Quantexa platform. Build reusable frameworks and shared engineering capabilities. Work across Azure, Databricks and cloud ...

retraining, and optimise models for reliability and cost efficiency in AWS.Collaboration & Enablement: Partner with data engineers to integrate ML workflows into wider platforms (Spark, Databricks), and share best practice and mentor other technical professionals. What You'll Love About This RoleThink Big: Build ML and AI solutions that shape … models underperform across data, features and architecture, and make reasoned trade-offs. Real-time & distributed ML: A strong grasp of production inference patterns, plus Spark and distributed data processing. Reproducibility & tooling: Reproducible environments (UV/Docker) and MLflow or equivalent, on AWS with Spark, Databricks and Snowflake. Engineering ...

retraining, and optimise models for reliability and cost efficiency in AWS. Collaboration & Enablement: Partner with data engineers to integrate ML workflows into wider platforms (Spark, Databricks), and share best practice and mentor other technical professionals. What You'll Love About This Role Think Big: Build ML and AI solutions … models underperform across data, features and architecture, and make reasoned trade-offs. Real-time & distributed ML: A strong grasp of production inference patterns, plus Spark and distributed data processing. Reproducibility & tooling: Reproducible environments (UV/Docker) and MLflow or equivalent, on AWS with Spark, Databricks and Snowflake. Engineering ...

bandwidth. The goal: no software rewrites, predictable time to production, early integration, and fast iteration. You'll work with tools such as DBT, Python, Apache Airflow, Spark, Iceberg; AWS services like Lambda, Bedrock, and SageMaker, containers, and whatever adds the most value. You'll design tools that ...

Scala programming to develop robust, scalable, and high-performance data processing applications. Build and optimise large-scale data processing and analytics pipelines using Apache Spark. Develop and maintain end-to-end data pipelines for both batch and real-time data processing. Apply solid understanding of data engineering concepts, including … business stakeholders. Skills & Experience: Proven experience as a Software Engineer with a strong focus on Scala development. Extensive hands-on experience with OpenShift and Apache Spark. In-depth understanding of data engineering principles and practises. Experience in developing and maintaining scalable data pipelines for batch and real-time processing. ...

Our client is a global technology company. A problem isnt truly solved until its solved for all. Thats why colleagues build products that help create opportunities for everyone, whether down the street or across the ...