Engineer for Backend services you will be responsible for a range of backend systems such as Software applications that include areas such as: Payment Gateway, Settlement, Transaction Fraud Monitoring, Datalake Service and the Core Banking System. You will oversee their deployment, development, enhancements and production operations. You should have extensive experience and skills in Software Engineering, DevOps, project management, incident More ❯
Social network you want to login/join with: Data Governance Business Analyst, London col-narrow-left Client: TEKsystems Location: London, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Reference: 5a8327bcb590 Job Views: 2 Posted: 02.06.2025 Expiry Date: 17.07.2025 col-wide Job Description: About the Role This role is heavily focused on … the implementation and optimisation of Microsoft Purview within the organisation's data ecosystem. We are looking for a highly skilled Data Governance Consultant with deep expertise in Purview to lead this … initiative. The ideal candidate will have a strong understanding of data architecture, regulatory compliance (GDPR, DORA), and Microsoft's data stack (Azure, DataLake, Power BI). Key Responsibilities Lead the end-to-end implementation of Microsoft Purview , including data cataloguing, classification, and lineage tracking. Map and document the dataMore ❯
Senior Data Engineer - Azure - Leeds I'm looking for a Senior Azure Data Engineer for a client near to Leeds. The company are building a greenfield Data Platform in Azure and Snowflake and … so looking for someone who has been heavily involved in building end to end data platforms in Azure using Synapse, Data Factory, DataLake and SQL. Role & Responsibilities Building end to end data pipelines in Azure (Data Factory and SQL) Building workflows in SQL, Spark and DBT Data and dimensional modelling Skills & Qualifications Azure Data factory, Synapse and SSIS Python/Park/PySpark Ideally Snowflake and DBT More ❯
One of the biggest UK rail network companies operating with rail infrastructure and asset management has a need to develop and support enterprise-level information systems. Align the data science initiatives with the overall business goals. This could range from improving customer satisfaction … enhancing operational efficiency, to driving sales growth. Responsibilities DWH development Data modelling Requirements clarification Customer communications Skills Must have Data Modelling Azure DataLake Azure Data Factory Data Pipeline development Postgres, MySQL and SQLite Nice to have Machine learning Python More ❯
/day Inside | Remote with occasional London travel We are seeking a PySpark Data Engineer to support the development of a modern, scalable datalake for a new strategic programme. This is a greenfield initiative to replace fragmented legacy reporting solutions, offering the opportunity to shape a long-term, high-impact platform from the ground … Key Responsibilities: * Design, build, and maintain scalable data pipelines using PySpark 3/4 and Python 3. * Contribute to the creation of a unified datalake following medallion architecture principles. * Leverage Databricks and Delta Lake (Parquet format) for efficient, reliable data processing. * Apply BDD testing practices using Python Behave and ensure code … and Azure DevOps. Required Skills & Experience: * Proven expertise in PySpark 3/4 and Python 3 for large-scale data engineering. * Hands-on experience with Databricks, Delta Lake, and medallion architecture. * Familiarity with Python Behave for Behaviour Driven Development. * Strong understanding of YAML, code quality tools (e.g. Python Coverage), and CI/CD pipelines. * Knowledge of Azure More ❯
/day Inside | Remote with occasional London travel We are seeking a PySpark Data Engineer to support the development of a modern, scalable datalake for a new strategic programme. This is a greenfield initiative to replace fragmented legacy reporting solutions, offering the opportunity to shape a long-term, high-impact platform from the ground … Key Responsibilities: * Design, build, and maintain scalable data pipelines using PySpark 3/4 and Python 3. * Contribute to the creation of a unified datalake following medallion architecture principles. * Leverage Databricks and Delta Lake (Parquet format) for efficient, reliable data processing. * Apply BDD testing practices using Python Behave and ensure code … and Azure DevOps. Required Skills & Experience: * Proven expertise in PySpark 3/4 and Python 3 for large-scale data engineering. * Hands-on experience with Databricks, Delta Lake, and medallion architecture. * Familiarity with Python Behave for Behaviour Driven Development. * Strong understanding of YAML, code quality tools (e.g. Python Coverage), and CI/CD pipelines. * Knowledge of Azure More ❯
City of London, London, United Kingdom Hybrid / WFH Options
Opus Recruitment Solutions
Are you a passionate Data Engineering Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end … to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For … Proven Data Engineering experience (5+ years). Consultancy experience is a must. Leadership and multi-project environments experience. Expertise in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, datalakes, and data warehouses. Hands-on experience with Apache Spark and bonus points More ❯
Data Engineer Location: London (4 days/week on-site) Type: Contract (Outside IR35) Duration: 6 months A leading physical commodities trading firm is seeking a Data Engineer for a 6-month contract (Outside IR35). You’ll work on-site in London 4 days per week, building and optimizing data pipelines in a … fast-paced, data-driven trading environment. Responsibilities: Build and maintain scalable data pipelines (ETL/ELT) Support data users with ingestion, debugging, and navigation Integrate data from diverse sources (structured & unstructured) Monitor and ensure data quality Maintain and improve existing production processes Develop dashboards and visualizations Work closely with data scientists and stakeholders Follow CI/CD and code best practices (Git, testing, reviews) Tech Stack & Experience: Strong Python (Pandas), PySpark, and SQL skills Cloud data tools (Azure Data Factory, Synapse, Databricks, etc.) Data integration experience across formats and platforms Strong communication and data literacy Nice to Have: Commodities/ More ❯
Data Engineer Location: London (4 days/week on-site) Type: Contract (Outside IR35) Duration: 6 months A leading physical commodities trading firm is seeking a Data Engineer for a 6-month contract (Outside IR35). You’ll work on-site in London 4 days per week, building and optimizing data pipelines in a … fast-paced, data-driven trading environment. Responsibilities: Build and maintain scalable data pipelines (ETL/ELT) Support data users with ingestion, debugging, and navigation Integrate data from diverse sources (structured & unstructured) Monitor and ensure data quality Maintain and improve existing production processes Develop dashboards and visualizations Work closely with data scientists and stakeholders Follow CI/CD and code best practices (Git, testing, reviews) Tech Stack & Experience: Strong Python (Pandas), PySpark, and SQL skills Cloud data tools (Azure Data Factory, Synapse, Databricks, etc.) Data integration experience across formats and platforms Strong communication and data literacy Nice to Have: Commodities/ More ❯
Data Engineer Location: London (4 days/week on-site) Type: Contract (Outside IR35) Duration: 6 months A leading physical commodities trading firm is seeking a Data Engineer for a 6-month contract (Outside IR35). You’ll work on-site in London 4 days per week, building and optimizing data pipelines in a … fast-paced, data-driven trading environment. Responsibilities: Build and maintain scalable data pipelines (ETL/ELT) Support data users with ingestion, debugging, and navigation Integrate data from diverse sources (structured & unstructured) Monitor and ensure data quality Maintain and improve existing production processes Develop dashboards and visualizations Work closely with data scientists and stakeholders Follow CI/CD and code best practices (Git, testing, reviews) Tech Stack & Experience: Strong Python (Pandas), PySpark, and SQL skills Cloud data tools (Azure Data Factory, Synapse, Databricks, etc.) Data integration experience across formats and platforms Strong communication and data literacy Nice to Have: Commodities/ More ❯
Senior Data Engineer I am looking for a Senior Data Engineer to join an established retailer as they look to migrate to Microsoft Fabric. They are looking to integrate a number of their data systems into one fully-scalable solution and create a brand-new data warehouse that will enable data … you would work collaboratively with the Engineering manager and other Engineers. You will also work collaboratively with the wider IT function to enable to organisation to leverage its data and orchestrate data-driven decisions. As part of this role, you will be responsible for. Help create a scalable data platform and integrate data … PySpark Data pipeline development experience utilising Azure Data Factory or Fabric Pipelines Experience working within an Azure environment such as Lakehouse Architecture, DataLake, Delta Lake, Azure Synapse Strong SQL knowledge Strong communication skills This is a hybrid role based out of the organisations office in Bristol 3 times per week with More ❯
Data Architect – Remote with UK Travel £100,000 + Benefits Join a fast-growing business as a Data Architect , designing and implementing cutting-edge data solutions that drive success. With flexible working and occasional travel, this is a prime opportunity to shape data architecture in a collaborative, forward-thinking environment. Why Apply … Ownership & Impact – Play a key role in data strategy Innovation – Work with the latest Azure tech stack Growth & Support – Access expert teams & resources Great Perks – Private healthcare, car allowance, pension & more What You’ll Do: Architect and optimize data solutions for business needs Lead migrations from legacy systems to modern platforms Ensure performance, scalability & compliance with … influence stakeholders through insights & training What You Need: Proven experience as a Data Architect. Expert in Microsoft BI Stack, Azure Data Factory, DataLake & Databricks Skilled in SQL, SPARK & Python Strong knowledge of Lakehouse architecture & data governance Experience in Scrum environments with Git-based workflows Apply Now! If the above opportunity More ❯
Data Architect – Remote with UK Travel £100,000 + Benefits Join a fast-growing business as a Data Architect , designing and implementing cutting-edge data solutions that drive success. With flexible working and occasional travel, this is a prime opportunity to shape data architecture in a collaborative, forward-thinking environment. Why Apply … Ownership & Impact – Play a key role in data strategy Innovation – Work with the latest Azure tech stack Growth & Support – Access expert teams & resources Great Perks – Private healthcare, car allowance, pension & more What You’ll Do: Architect and optimize data solutions for business needs Lead migrations from legacy systems to modern platforms Ensure performance, scalability & compliance with … influence stakeholders through insights & training What You Need: Proven experience as a Data Architect. Expert in Microsoft BI Stack, Azure Data Factory, DataLake & Databricks Skilled in SQL, SPARK & Python Strong knowledge of Lakehouse architecture & data governance Experience in Scrum environments with Git-based workflows Apply Now! If the above opportunity More ❯
High Wycombe, England, United Kingdom Hybrid / WFH Options
VIQU IT Recruitment
Role: Data Analyst Location: High Wycombe/Hybrid Salary: £40,000 - £45,000 per annum VIQU have partnered with a leading retail company who are currently recruiting for a data analyst to optimise their business by leveraging data driven insights. The role will focus on analytics on an existing Azure based data platform, and will include data extraction, cleaning, analysis, and visualisation whilst using SQL SSIS/SSRS software. Responsibilities of the Data Analyst: Identify trends by analysing data. Collaborate with cross functional teams to deliver insights. Build Power BI dashboards. Extract, clean, and transform data from existing platforms (e.g., ERP, CRM, or bespoke … analyst, driving actionable insights. Experience using SQL (SSRS/SSIS) for managing databases. Ability to manipulate data. Knowledge of Azure data tools, including DataLake, Data Factory, and Synapse Analytics. Experience with data visualisation tools like Power BI. Strong communication and presentation skills. Role: Data Analyst Location: High More ❯
Data Engineer Warrington - Hybrid £50,000 - £60,000 - Depending on experience Focus 5 Recruitment are working with an exciting software business to help recruit a Data Engineer. The company have just been awarded 2 large contracts with international Mobile Network Operators. Appointed to help them source a Data Engineer, were looking for an experienced … Data Engineerto design and optimize our clients data pipelines and storage solutions. This is an amazing opportunity to work with a growing and ambitious software business who have contracts with some of the worlds leading mobile network companies. They are looking for candidates who can come in at a key point in their growth and develop … their career as they grow. Key responsibilities for the Data Engineer Design and build high-performance, low-latency data pipelines capable of processing large volumes of data at high speed. Develop and enhance real-time and batch data processing architectures. Manage both structured and unstructured data, ensuring efficient ingestion, transformation More ❯
What do you want to search? Keyword Apprenticeship Type Location Data Analyst Apprentice Apply From: 01/05/2025 Learning Provider Delivered by QA LIMITED Employer OUTSERVE LIMITED Vacancy Description Key duties/Business and Financial Results: Develop skills in the use of Microsoft Power BI to create reports that provide solutions to clients' challenges. Attain a … detailed understanding of the cloud-based financial software and applications in our portfolio, to build basic reports for trend analysis and key data presentation for clients. Participate in database management within data warehouses and datalakes for consolidating client systems and data. Handle client support queries related to data issues and participate … in scoping meetings to understand client needs. Support software launches and integrations, learning the process of moving data from source to warehouse via middleware and API. Analyze and interpret data and statistics. Review and improve processes to increase efficiency and automate tasks. Key Details Vacancy Title Data Analyst Apprentice Employer Description Outserve Limited is More ❯
Principal Data Engineer – AWS Location : London (Hybrid) Salary : £105,000 Job Type : Permanent A top-tier digital consultancy is making a key strategic hire at the Associate Director level for a hands-on, stakeholder-savvy Principal Data Engineer. You will take the helm of public and private sector data transformation programmes, drive technical delivery … shape a growing practice at scale. This is a greenfield opportunity for a high-impact leader with deep AWS expertise and a consulting mindset as an experienced Principal Data Engineer. Key Responsibilities • Lead and deliver enterprise-grade data engineering solutions using core AWS services (S3, Glue, Lambda, Redshift, Matillion, etc.). • Define architecture, mentor large technical … Proven track record leading multi-million-pound projects within consulting or enterprise-level engagements. • Strong stakeholder engagement at CxO or Director level. • Deep experience in cloud datalake architectures, ETL/ELT patterns, and metadata/data quality management. • Expertise in Matillion, Redshift, Glue, Lambda, DynamoDB, and data pipeline automation. • Familiarity with dataMore ❯
City of London, London, United Kingdom Hybrid / WFH Options
Anson McCade
Principal Data Engineer – AWS Location : London (Hybrid) Salary : £105,000 Job Type : Permanent A top-tier digital consultancy is making a key strategic hire at the Associate Director level for a hands-on, stakeholder-savvy Principal Data Engineer. You will take the helm of public and private sector data transformation programmes, drive technical delivery … shape a growing practice at scale. This is a greenfield opportunity for a high-impact leader with deep AWS expertise and a consulting mindset as an experienced Principal Data Engineer. Key Responsibilities • Lead and deliver enterprise-grade data engineering solutions using core AWS services (S3, Glue, Lambda, Redshift, Matillion, etc.). • Define architecture, mentor large technical … Proven track record leading multi-million-pound projects within consulting or enterprise-level engagements. • Strong stakeholder engagement at CxO or Director level. • Deep experience in cloud datalake architectures, ETL/ELT patterns, and metadata/data quality management. • Expertise in Matillion, Redshift, Glue, Lambda, DynamoDB, and data pipeline automation. • Familiarity with dataMore ❯
South East London, England, United Kingdom Hybrid / WFH Options
Anson McCade
Principal Data Engineer – AWS Location : London (Hybrid) Salary : £105,000 Job Type : Permanent A top-tier digital consultancy is making a key strategic hire at the Associate Director level for a hands-on, stakeholder-savvy Principal Data Engineer. You will take the helm of public and private sector data transformation programmes, drive technical delivery … shape a growing practice at scale. This is a greenfield opportunity for a high-impact leader with deep AWS expertise and a consulting mindset as an experienced Principal Data Engineer. Key Responsibilities • Lead and deliver enterprise-grade data engineering solutions using core AWS services (S3, Glue, Lambda, Redshift, Matillion, etc.). • Define architecture, mentor large technical … Proven track record leading multi-million-pound projects within consulting or enterprise-level engagements. • Strong stakeholder engagement at CxO or Director level. • Deep experience in cloud datalake architectures, ETL/ELT patterns, and metadata/data quality management. • Expertise in Matillion, Redshift, Glue, Lambda, DynamoDB, and data pipeline automation. • Familiarity with dataMore ❯
Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the … latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy. Job responsibilities You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions You will develop … approaches and can apply data security and privacy strategy to solve business problems You have experience with different types of databases (i.e.: SQL, NoSQL, datalake, data schemas, etc.) Professional Skills You understand the importance of stakeholder management and can easily liaise between clients and other key stakeholders throughout projects, ensuring buy-in More ❯
JOB SUMMARY In this role, you'll play a pivotal part in building and optimizing data pipelines that transform large, multi-modal datasets into high-quality training inputs for cutting-edge AI models for drug discovery. You'll help evolve our data pipeline and storage infrastructure to support faster, more reliable turnarounds for research and development … engineering best practices-including version control, testing, and documentation. This role is remote, with preference for candidates on the East Coast or UK. KEY RESPONSIBILITIES Design and improve data pipelines that process large, multi-modal datasets from a variety of internal and external sources into training datasets for AI models. Evolve our data storage layer to … ML workflows is a plus. Hands-on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of datalake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset - able to evaluate tradeoffs find solutions that empower ML researchers to move quickly. Background More ❯
spread information about new treatments to key people in the life science community. You can read more about Veeva Link on our product pages at . As a data engineer, you focus on our data pipelines and take responsibility for a major part of the Link data processing platform. We value end-to-end … ownership, which puts you into the sweet spot of finding, designing, and implementing improvements to the product's data pipelines and adjusting them to changing demands of the market. You take responsibility for features and innovation using SOLID and clean software principles, take part in the architectural enhancement process and care for the quality of the outcome. Monitoring … engineering domain, you will focus on different aspects. We decide together which domains fit best for you. What You'll Do Work on Veeva Link's next-gen Data Platform Improve our current environment with features, refactoring, and innovation Work with JVM-based languages or Python on Spark-based data pipelines Operate ML models in close More ❯
look to our Business Consultants, to drive change, inspire trust with clients, and take responsibility for some of the world's most ambitious transformation projects. As a Senior Data Technology Consultant within our successful Business Consulting practice, you, alongside a collaborative team of consulting colleagues, will take a central role in providing support to our clients with a … focus on driving their business outcomes using data and technology. You will also support the growth of the wider evolving business consulting function. A key focus of the role will be to deliver practical advice to clients relating to their enterprise data technologies and strategies whilst looking out for opportunities to support them further leveraging our … quality consulting services to clients, presenting technical concepts in simple, accessible language. • Leverage your deep understanding of legacy and cloud-based data technologies, including DataLake, Data Warehouse, and Lakehouse concepts. • Bring broad technical knowledge of Azure data products, with subject matter expertise in areas like Azure Data Factory More ❯
Overview: 3 contract data engineers to supplement existing team during implementation phase of new data platform. Main Duties and Responsibilities: Write clean and testable code using PySpark and SparkSQL scripting languages, to enable our customer data products and business applications. Build and manage data pipelines and notebooks, deploying code in a structured … trackable and safe manner. Effectively create, optimise and maintain automated systems and processes across a given project(s) or technical domain. Data Analyse, profile and plan work, aligned with project priorities. Perform reviews of code, refactoring where necessary. Deploy code in a structured, trackable and safe manner. Document your data developments and operational procedures. Ensure adherence … to data/software delivery standards and effective delivery. Help monitor, troubleshoot and resolve production data issues when they occur. Contribute to the continuous improvement of the team. Contribute to the team's ability to make and deliver on their commitments. Innovate and experiment with technology to deliver real business benefits. Regularly launch products and services More ❯
end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure DataLake or other storage services. Write efficient and standardized Spark SQL and PySpark code for data transformations, ensuring data integrity and accuracy across the pipeline. Automate … systems). Implement data transformation logic using Spark, ensuring data is cleaned, transformed, and enriched according to business requirements. Leverage Databricks features such as Delta Lake to manage and track changes to data, enabling better versioning and performance for incremental data loads. Data Publishing & Integration: Publish clean, transformed data to Azure DataLake or other cloud storage solutions for consumption by analytics and reporting tools. Define and document best practices for managing and maintaining robust, scalable data pipelines. Data Governance & Security: Implement and maintain data governance policies using Unity Catalog, ensuring proper organization, access control, and metadata management More ❯
