1 to 25 of 27 Permanent Apache Spark Jobs in Manchester

Data Engineer

Location
Wigan, England, United Kingdom
driving business growth and commercial success. Reporting into the Data Discipline Lead you will build streaming and batch data pipelines on AWS using Apache Spark Structured Streaming, Lakeflow Spark Declarative Pipelines (formerly DLT), and Kafka (MSK), ensuring our business teams have fast, trusted data at their fingertips. … Looking For: We’re looking for experienced Data Engineers with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding ...

Senior Data Engineer

Location
Manchester, England, United Kingdom
solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted ...

Senior Data Engineer, Technology Consulting- London, Leeds, Manchester or Newcastle

Location
Manchester, England, United Kingdom
agreed scope, acting as the key point of escalation for decisions that impact your area of work - Demonstrable experience of building data pipelines using Spark, Python or SQL - Experience working with AWS and its data and AI/ML services such as SageMaker - Experience of big data and lakehouse ...

Data Operations Engineer

Location
Manchester, England, United Kingdom
established standards. Strong communication and collaboration skills. Desirable Experience Experience across some of the following would be beneficial: AWS Redshift, Athena, EMR or Kinesis. Apache Spark or PySpark for distributed data processing. AWS Step Functions or other workflow orchestration technologies. Terraform, CloudFormation or other Infrastructure as Code tooling. ...

Lead Data Engineer

Location
Manchester, England, United Kingdom
possess hands‐on experience with one or more of the following technological areas (or their equivalent cloud‐native alternatives): Analytics & Platform Frameworks like Apache Spark or similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like ...

Senior Data Engineer

Hiring Organisation
Sellick Partnership
Location
Greater Manchester, United Kingdom
Employment Type
Permanent
Salary
£47000 - £57000/annum
requirements into practical data engineering solutions * Excellent analytical and problem-solving skills Desirable experience/qualifications will include * Azure Data Factory * Microsoft Purview * Python, Spark or PySpark * DevOps, CI/CD pipelines and Infrastructure as Code * DataOps practices * AI, machine learning or advanced analytics platforms * Dynamics ...

Principal Data Engineer

Location
Manchester, England, United Kingdom
analytics and AI/ML at enterprise scale.* Develop and optimise batch and streaming ETL/ELT pipelines using Azure Data Factory, PySpark, SQL, Apache Spark and Airflow, integrated with modern tooling like dbt and Git-based CI/CD.* Implement data cataloguing, lineage tracking and governance frameworks … data engineering, including designing and delivering data lake or lakehouse architectures in an enterprise cloud environment, ideally Microsoft Azure.* Hands-on expertise with Databricks, Spark, PySpark, SQL and core Azure Data Services (Data Lake Storage, Synapse Analytics, Data Factory).* Experience with data governance and cataloguing tools such ...

Senior Data Engineer

Hiring Organisation
Reed Technology
Location
Manchester, United Kingdom
Employment Type
Permanent
Salary
£50000 - £57650/annum
Experience integrating multiple business systems, applications and third-party data sources using cloud-based technologies. Desirable experience with Azure Data Factory, Microsoft Purview, Python, Spark, DataOps, DevOps, AI platforms or large-scale cloud modernisation projects. ...

Senior Data Engineer

Hiring Organisation
Reed Technology
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, On-site
Salary
£57,650
Experience integrating multiple business systems, applications and third-party data sources using cloud-based technologies. Desirable experience with Azure Data Factory, Microsoft Purview, Python, Spark, DataOps, DevOps, AI platforms or large-scale cloud modernisation projects. ...

Senior Data Engineer

Hiring Organisation
Data Careers
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
Salary
£60,000
Experience integrating multiple business systems, applications and third-party data sources using cloud-based technologies. Desirable experience with Azure Data Factory, Microsoft Purview, Python, Spark, DataOps, DevOps, AI platforms or large-scale cloud modernisation projects. ...

Senior Software Engineer

Location
Manchester, England, United Kingdom
Lambda, ECS, RDS, S3, SQS, API Gateway, and CloudWatch. Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms. Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools. Drive observability, monitoring, logging, and operational readiness ...

Senior Solution Architect

Location
Manchester, England, United Kingdom
well as competitive ETL solutions Experience with value-based sales motion, especially through MEDDPPICC and Command of the Message Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza) Familiarity with version control tools such as Git, and experience with Python Degree in Computer ...

Data Science & AI Graduate Scheme (Manchester)

Location
Manchester, England, United Kingdom
modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK — using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

Data Scientist (D&D UK)

Location
Manchester, England, United Kingdom
Power BI Understanding of model evaluation and data science methodologies Strong problem-solving, analytical and communication skills Experience working in Agile environments Databricks and Apache Spark experience Azure or other cloud platform experience Experience with Generative AI and Large Language Models (LLMs) Experience deploying machine learning solutions into ...

Senior Data Specialist

Location
Salford, England, United Kingdom
problem‐solving skills to collaborate across global teams, and translate complex data insights into actionable outputs Familiarity with data platforms such as Databricks, Spark, Snowflake, or similar technologies Ability to communicate technical concepts clearly to non‐technical stakeholders Basic understanding of artificial intelligence concepts, with curiosity and enthusiasm ...

Senior Data Scientist (UK)

Location
Sale, England, United Kingdom
Nice to have: Proven experience leading projects and managing teams, Experience in presales, client workshops, or building Proof-of-Concepts (PoCs), Familiarity with Hadoop, Spark, Research or R&D background. Soft Skills Ability to structure projects teams, make technical decisions, and own outcomes, Ability to manage multiple projects ...

Data Architect

Location
Manchester, England, United Kingdom
Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform ...

Principal Solution Architect, Data & AI

Location
Manchester, England, United Kingdom
Private cloud platforms (HPE Private AI, Nvidia AI Enterprise, Dell AI Factories) Understanding of data engineering practices (ETL, data lakes, big data tools like Spark). Knowledge of designing scalable, distributed systems. Familiarity with model lifecycle management and MLOps practices. Ability to align AI initiatives with business outcomes ...

Data Science & AI Specialist

Location
Manchester, England, United Kingdom
data, cloud platforms or distributed processing environments. Experience operating within regulated or security-sensitive environments. Experience supporting, coaching or mentoring others. Tools & Technologies: Python Spark Scala Databricks AWS SQL Azure Snowflake MLflow GitHub Our Package A structured Technical Progression Pathway to recognise and reward appropriate technical people. Competitive ...

Assistant Director - Data Specialist

Location
Salford, England, United Kingdom
maintain operational continuity and quality output Knowledge of data processing frameworks, including ETL pipelines, data integration, and data modeling (experience with Databricks or Spark preferred) Strong analytical and problem‐solving skills with the ability to communicate clearly to both technical and non‐technical stakeholders Basic understanding of artificial intelligence ...

Lead Data Engineer

Location
Manchester, England, United Kingdom
including target modelling, cutover approach, reconciliation strategy and technical risk management. Design and build the real‐time data architecture using Amazon MSK/Apache Kafka and PySpark Structured Streaming (evaluating Apache Flink where it is the better fit), including event schema governance, exactly‐once semantics and stream … large‐scale batch and streaming processing, and strong command of data modelling and data warehousing concepts. Production experience designing and operating streaming architectures with Apache Kafka or Amazon MSK, including stream processing with Spark Structured Streaming. Hands‐on experience with open table formats, such as Apache Iceberg ...

Senior Data Scientist

Location
Manchester, England, United Kingdom
senior stakeholders and contribute to business development initiatives. Provide thought leadership through industry engagement and innovation. Requirements Strong Python, SQL, Pandas, Scikit-Learn and Spark experience. Experience delivering real-world AI and machine learning systems. Understanding of statistics and machine learning methodologies. Experience with cloud platforms and large-scale ...

Senior Data Engineer

Hiring Organisation
Adria Solutions
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
Salary
£80,000
Senior Data Engineer – SQL/Python/AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data ...

Head of Data Science

Location
Manchester, England, United Kingdom
lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands‐on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practice Credibility to lead long‐tenured individual contributors while providing ...

Lead Data Engineer - Perm up to £100k

Hiring Organisation
Lorien Resourcing Limited
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/… pairing and design sessions Confidence communicating with technical and non-technical stakeholders, and challenging constructively Desirable: SQL Server or on-prem to cloud migration, Apache Flink, regulated industry experience If you'd rather shape a platform than inherit a finished one, and want a seat at the table while ...