19 of 19 Apache Spark Jobs in the City of London

/ELT) to support analytics and downstream use cases Developing scalable data solutions using Python , with a focus on maintainability and performance Working with Apache Spark/PySpark to process and transform large datasets Supporting the ingestion, transformation and validation of complex financial data Improving the performance, reliability … That Will Help You Succeed Essential Commercial experience as a Data Engineer at a mid‐level Strong Python development skills Hands‐on experience with Apache Spark/PySpark Solid experience building ETL/ELT pipelines Background within the financial services industry (investment management experience desirable) Comfortable working with ...

Responsibilities will include; Design, implement, and optimize end-to-end solutions using Fabric components: o Data Factory (pipelines, orchestration) o Data Engineering (Lakehouse, notebooks, Apache Spark) o Data Warehouse (SQL endpoints, schemas, MPP performance tuning) o Real-Time Analytics (KQL databases, event ingestion) o Manage and enhance OneLake ...

Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding of data ...

. Hands-on skills in data pipelines, ETL/ELT, distributed processing, APIs and data architecture. Experience with modern tooling (Databricks, Spark, Synapse, Snowflake, BigQuery, Kafka etc.). Excellent client-facing skills with the ability to lead technical and strategic conversations. Consulting experience and confidence working in complex, evolving ...

with attention to detail and accuracy. Adept at queries, report writing, and presenting findings. Experience working with large datasets and distributed computing tools (Hadoop, Spark, etc.) Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests, etc.). Experience with data profiling tools and processes. Knowledge ...

DevOps and broader software engineering standards. Cloud experience (any platform). Previous mentoring experience. Nice to have: Snowflake or Databricks Spark, PySpark, Hadoop or similar big data tooling BI exposure (PowerBI, Tableau, etc.) Interview Process The process is fully structured, transparent, and efficient: Video call - high-level overview ...

+ 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform, who are hiring … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such ...

Data Analytics Sales Executive - Data Sales, Talend, Snowflake, Apache Spark, Kafka, Dataops, AWS, Azure, Data visualisation, SQL, ETL, Pre-Sales, Account Manager, SaaS Sales, Cloud Solutions, Digital Transformation, Business Development, Sales Consultant, CXO Engagement, Strategic Accounts, Deal Negotiation, Remote Working, UK-wide, Permanent Role, Sales Targets, New Business ...

models, transformers, retrieval models). Hands-on experience with LLMs and GenAI technologies. Strong programming and problem-solving skills with proficiency in Python, SQL, Spark, and Hive. Deep understanding of classical and modern ML techniques, A/B testing methodologies, and experiment design. Solid background in ranking, recommendation ...

Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda is a plus. Direct experience in building and applying large-scale (100M+ users) machine learning ...

Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda is a plus. Direct experience in building and applying large-scale (100M+ users) machine learning ...

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

stores (Feast, AWS Feature Store). Containerisation (Docker) and orchestration (Kubernetes, ECS). Infrastructure as Code (Terraform, CloudFormation). Large-scale data processing frameworks (Spark, Dask). Familiarity with data governance and compliance frameworks. Experience in regulated industries such as healthcare or finance. Benefits: Competitive salary ...

500s, Industry leaders and ambitious start-ups across media, e-commerce, healthcare and more. Requirements Must have strong Python and SQL Solid experience with Spark/Airflow and Kafka Strong experience with large cloud-based systems (AWS) Location: London (3 days per week on-site) Start: ASAP Rate ...

solutions, with a focus on innovation, resilience, and regulatory compliance. Key Responsibilities: Lead and develop robust data engineering platforms leveraging technologies such as Hadoop, Spark, and Splunk. Design, implement, and maintain scalable ETL/ELT data pipelines for diverse data types, including raw, structured, semi-structured, and unstructured data … groups and vendor interactions to influence and adopt emerging technologies and standards. Qualifications and Skills: Extensive experience with big data frameworks such as Hadoop, Spark, and Splunk. Strong scripting capabilities in Python, with experience in object-oriented and functional programming paradigms. Proven expertise in handling varied data formats ...

Position Summary The Senior AI Architect is responsible for designing and implementing advanced AI and cloud architectures, with a strong emphasis on integrating generative AI technologies. This role requires a broad understanding of AI and ...

architecture, and network stack optimization. Data Infrastructure: Familiarity with the modern data stack, including time-series databases, object stores, and streaming frameworks like Apache Flink or Spark Streaming . Engineering Rigor: You write clean, testable, and reliable code. You understand that in this environment, a system failure ...