17 of 17 Permanent Apache Spark Jobs in the City of London

Technical Expertise: Solid experience in Python programming, particularly using data manipulation and processing libraries such as Pandas, NumPy, and Apache Spark. Hands-on experience with open-source data frameworks like Apache Spark, Apache Kafka, and Apache Airflow. Strong proficiency in SQL, including advanced query development and performance tuning. Good understanding of distributed computing principles and … automation pipelines. Experience working with relational databases such as PostgreSQL, MySQL, or equivalent platforms. Skilled in using containerization technologies including Docker and Kubernetes. Experience with workflow orchestration tools like Apache Airflow or Dagster. Familiar with streaming data pipelines and real-time analytics solutions. More ❯

Technical Expertise: Solid experience in Python programming, particularly using data manipulation and processing libraries such as Pandas, NumPy, and Apache Spark. Hands-on experience with open-source data frameworks like Apache Spark, Apache Kafka, and Apache Airflow. Strong proficiency in SQL, including advanced query development and performance tuning. Good understanding of distributed computing principles and … automation pipelines. Experience working with relational databases such as PostgreSQL, MySQL, or equivalent platforms. Skilled in using containerization technologies including Docker and Kubernetes. Experience with workflow orchestration tools like Apache Airflow or Dagster. Familiar with streaming data pipelines and real-time analytics solutions. More ❯

Be Doing You'll be a key contributor to the development of a next-generation data platform, with responsibilities including: Designing and implementing scalable data pipelines using Python and Apache Spark Building and orchestrating workflows using AWS services such as Glue , Lambda , S3 , and EMR Serverless Applying best practices in software engineering: CI/CD , version control , automated … testing , and modular design Supporting the development of a lakehouse architecture using Apache Iceberg Collaborating with product and business teams to deliver data-driven solutions Embedding observability and quality checks into data workflows Participating in code reviews, pair programming, and architectural discussions Gaining domain knowledge in financial data and sharing insights with the team What They're Looking For … for experience with type hints, linters, and testing frameworks like pytest) Solid understanding of data engineering fundamentals: ETL/ELT, schema evolution, batch processing Experience or strong interest in Apache Spark for distributed data processing Familiarity with AWS data tools (e.g., S3, Glue, Lambda, EMR) Strong communication skills and a collaborative mindset Comfortable working in Agile environments and More ❯

Excellent problem-solving skills and ability to work independently in a fast-paced environment. Desirable: Experience with NLP, computer vision, or time-series forecasting. Familiarity with distributed computing frameworks (Spark, Ray). Experience with MLOps and model governance practices. Previous contract experience in a similar ML engineering role. Contract Details Duration: 6–12 months (extension possible) Location: London (Hybrid More ❯

Excellent problem-solving skills and ability to work independently in a fast-paced environment. Desirable: Experience with NLP, computer vision, or time-series forecasting. Familiarity with distributed computing frameworks (Spark, Ray). Experience with MLOps and model governance practices. Previous contract experience in a similar ML engineering role. Contract Details Duration: 6–12 months (extension possible) Location: London (Hybrid More ❯

ideal candidate will lead the delivery of modern data solutions across multiple projects, leveraging Azure and Databricks technologies in an agile environment. Core Requirements Cloud & Data Engineering: Azure, Databricks, Apache Spark, Azure Data Factory, Delta Lake Programming & Querying: Python, SQL (complex, high-performance queries) Data Governance & DevOps: Unity Catalog/Purview, Terraform, Azure DevOps Consulting: Requirements gathering, stakeholder More ❯

GCP including BigQuery, Pub/Sub, Cloud Composer and IAM You have strong Python, SQL and PySpark skills You have experience with real-time data streaming using Kafka or Spark You have a good knowledge of Data Lakes, Data Warehousing, Data Modelling You're familiar with DevOps principles, containerisation and CI/CD tools such as Jenkins or GitHub More ❯

testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for data querying and validation. 4. Knowledge of big data technologies such as hadoop, spark, or kafka is a plus. 5. Familiarity with scripting languages like python, java, or shell scripting. 6. Excellent analytical and problem-solving skills with a keen attention to detail. More ❯

testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for data querying and validation. 4. Knowledge of big data technologies such as hadoop, spark, or kafka is a plus. 5. Familiarity with scripting languages like python, java, or shell scripting. 6. Excellent analytical and problem-solving skills with a keen attention to detail. More ❯

and NoSQL databases such as PostgreSQL, DynamoDB, or OpenSearch. -Problem-Solver: Comfortable working in Linux environments and confident debugging logs, scripts, and production issues. -Additional Skills: Exposure to Kafka, Spark, or dbt Core, with an interest in domain-driven data contracts. Meet Citywire We cover - and connect - all sides of the $100 trillion global asset management industry - through our More ❯

and NoSQL databases such as PostgreSQL, DynamoDB, or OpenSearch. -Problem-Solver: Comfortable working in Linux environments and confident debugging logs, scripts, and production issues. -Additional Skills: Exposure to Kafka, Spark, or dbt Core, with an interest in domain-driven data contracts. Meet Citywire We cover - and connect - all sides of the $100 trillion global asset management industry - through our More ❯

in Python for data pipelines, transformation, and orchestration. Deep understanding of the Azure ecosystem (e.g., Data Factory, Blob Storage, Synapse, etc.) Proficiency in Databricks (or strong equivalent experience with Apache Spark ). Proven ability to work within enterprise-level environments with a focus on clean, scalable, and secure data solutions. If you are the right fit - contact me More ❯

in Python for data pipelines, transformation, and orchestration. Deep understanding of the Azure ecosystem (e.g., Data Factory, Blob Storage, Synapse, etc.) Proficiency in Databricks (or strong equivalent experience with Apache Spark ). Proven ability to work within enterprise-level environments with a focus on clean, scalable, and secure data solutions. If you are the right fit - contact me More ❯

to create, test and validate data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What’s in it for you? Base More ❯

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

affordable charging infrastructure that is accessible to all, particularly the 40% of UK households without private driveways that rely on public charging. We believe our employees are the electric spark igniting our success. Were dedicated to our culture, and ensure that every individual feels charged up and empowered to accelerate their career which is why we have been recognised More ❯