151 to 175 of 232 Data Engineer Jobs in London

Senior Data Engineer (Inside IR35) - £700 to 825 per day Location: Remote/Hybrid (London) Rate: £825 per day Contract: Inside IR35 Are you an experienced Data Engineer with a strong background in banking? Our client are seeking Senior Data Engineers to contribute to innovative … data projects within a leading financial institution. Key Responsibilities: Data Engineering: Design and implement data models, ETL processes, and data warehousing solutions. Programming: Utilize Python, Java, Scala, or GoLang to build and optimize data pipelines. Distributed Processing: Work with Hadoop, Spark, and other platforms for … large-scale data processing. Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark. Database Management: Handle SQL databases like Oracle, MySQL, or PostgreSQL. Data Governance: Ensure data quality, security, and compliance with best practices. Ideal Candidate: Significant experience in data more »

Role Details: Role: Senior Data Engineer Role Type: Open to both permanent and contract positions Daily rate: £negotiable Location: London & Edinburgh Work Mode: Mostly WFH, candidates should be open to travel as required. Job Description: We are seeking a highly skilled Senior Data Engineer with expertise … in SAS to join our team and contribute to the transformation of our data pipelines. The successful candidate will play a crucial role in rewriting existing pipelines from SAS, Info, pearl, and shell scripts to Python and DBT-based solutions, ultimately loading data into Snowflake and Salesforce on … Responsibilities: Lead the conversion of existing SAS-based modules to Python-based solutions. Design, develop, and implement Python and DBT-based pipelines for efficient data transformation and loading. Collaborate with cross-functional teams to understand requirements and ensure successful pipeline migration. Utilize expertise in SAS, pearl, and shell scripts more »

About the job My client is seeking a dynamic Lead Data Engineer with robust data architecture and warehousing skills to lead their Data Engineering team. Being a leading financial institution, data fuels their operations and growth. being pivotal in ensuring data is expertly modelled … ingested, transformed, and accessible organisation-wide. Responsibilities: Team Leadership: Mentor and develop our Data Engineering team. Strategy Development: Shape and implement our data engineering strategy. Data Architecture: Define and implement solutions within our Data Vault framework. Cross-Functional Collaboration: Partner with data science, credit risk … and analytic teams to meet their data needs. Customer Data Platform: Manage our Rudderstack-based platform for comprehensive Customer 360 views. Quality Assurance: Ensure CI/CD and test automation with robust observability. Cost Management: Optimise data platform costs within budget. Supplier Management: Maintain strong relationships with more »

of knowledge on which to develop their careers and progress in the direction they choose. The Role: We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with … a fantastic team of technology experts. You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. About you: Our … skills Self motivated, enthusiastic and committed The ideal candidates will share our values and be aligned with our ways of working Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design and build data models, schemas to support business requirements Develop and maintain data ingestion more »

Senior Data Engineer (No Sponsorship Available) I am seeking an experienced Data Engineer who has experience managing green field projects and implementing modern data platforms that connect business systems and data sources to drive insights through data analytics. The ideal candidate will have … experience using Azure Platforms for consolidating data sources and making them accessible to the wider business to support growth and business strategy. Must have strong communication skills and be comfortable liaising with key stakeholders and other business units to understand business needs and gather requirements. Required Experience Track record … of designing and building data infrastructure and ETL pipelines Experience with Azure Platform including Data Factory, Synapse, and Data Lake DB development experience with SQL Software development experience in relevant languages like Python or C# Experienced with T-SQL and RESTful APIs Experience with Data Modelling more »

A global company are looking for an experienced BI Developer/Data Engineer to join their growing Data Team based in their London office – this is a hybrid role, requiring 3 days per week in the office to collaborate with colleagues. It’s a really exciting time … to join the team, as they look to leverage the latest Microsoft Azure data technologies, having recently launched a brand-new Azure Data Platform and introduced the use of Databricks. The role is predominantly back-end Data Engineering focused, where you’ll develop various solutions for data collection, integration, transformation and modelling, to support the ongoing development of their global data platform. This will include developing data pipelines, to automate data collection and integration across various sources, with the use of Azure Data Factory and Databricks. You will also do a little more »

Data Engineer | Integration | SSIS | SSRS | Azure Logic Apps | Azure API management | DevOps | Azure Data Factory | SQL | Oracle Database | Systems Integration | Integration Architecture We are thrilled to be partnering with a university who is in the market for a Data Engineer who has strong experience working … implementation of an ERP system that will manage their HR and Finance, due to go live in a year. Summary of the role: Role: Data & Integration Architect Employment Type: Permanent - 2-year FTC but one of the team member will be retiring so there will be an extension Location … of working. Essential skills: System integrations experience (rather than BI Analytics) Strong understanding of SQL OR oracle databases Microsoft integration tools (SSIS, SSRS, Azure data factory) Good level of communication Desirable skills: Experience with administration and deployment of Azure Logic Apps, Azure Function, Azure API management, ARM templates in more »

Leading ecommerce client are now searching for a Senior Data Engineer to contribute towards the delivery of their data strategy. This engineer will shape our client’s data function, delivering end to end solutions for an array of customer data projects. The role: Designing … data pipelines, managing data warehouses, and implementing complex ETL processes. Work closely with data scientists, analysts, and stakeholders to optimise our client’s infrastructure. Optimise data driven products, personalisation, reporting and overall business success. Manage the whole data lifecycle (End to End). Ensure the … reliability and quality of data. Key Requirements: Strong experience designing data pipelines/warehouses using AWS and Snowflake. Exposure to big data technologies such as Kafka, Spark, or Hadoop. Solid experience with Snowflake, including performance optimisation and cost management. Strong experience with SQL and Data modelling. Excellent more »

Lead Data Engineer £80,000 - £100,000 DOE Mainly remote, some travel may be required Blue Wolf Digital are partnered with a boutique consulting firm who are looking to grow significantly this year due to some greenfield project wins. I'm looking for an experienced Lead Data Engineer to play a pivotal role in building state-of-the-art Databricks and Snowflake Lakehouse solutions from scratch. If you have experience with Azure, Databricks or Snowflake, SQL, ETL, and a passion for consulting, please get in touch. Key Responsibilities: Design, develop, and maintain cutting-edge Databricks … or Snowflake Lakehouse solutions from scratch Collaborate with cross-functional teams to understand data requirements and translate them into efficient ETL processes. Work with Azure services to ensure data storage, security, and scalability. Develop and optimize SQL queries for data extraction and analysis. Stay updated on industry more »

Lead Data Engineer London £90k - £110k Hybrid Company Digitech are partnered with a scale-up SAAS business looking to bolster their Data team. The business are a data driven and AI focussed technology firm partnering with well known brands Globally, using cutting edge technology to improve … customer journeys and provide the firms with exceptional and real time data. Required 5 years of commercial data engineering or data architecture experience Hands on experience of developing solutions within public cloud Experience of greenfield Data Warehouse implementations, including Data Architecture principles Knowledge of data science concepts and ability to promote solutions to production via machine learning engineering Hands-on knowledge of NoSQL and relational Databases, alongside relevant data modelling techniques ETL/ELT tooling Knowledge of DBT, Luigi or similar orchestration tooling of managing and developing a team in an agile environment more »

Data Engineer On-Premise/Excel/SQL We have partnered with a leading client based in Central London who are looking to increase their data engineering team. Role: You will be working on a large volume of various financial reporting data sources. The client here … is using on-premise databases including Excel services & SQL. You will be working to gather data from these various sources for reporting, and creating the data pipelines. Experience: Data engineering experience within financial services (Asset Management/Harrington Starr) Experience working to gather data from on more »

DATA ENGINEER LONDON BASED £75,000-95,000 PER ANNUM This growing e-commerce trading company are searching for a new Data Engineer. You will be responsible for building and maintaining a new AWS serverless platform. You will also be responsible for building and maintaining data pipelines, using Python. THE COMPANY This E-commerce trading company have been growing since their formation. They are expanding their Data and Analytics team as the CEO has realised how important Data and Technology are to the business and is investing further in Machine Learning variables. THE … ROLE You will be joining an expanding Data and Analytics team, which currently has more than 20 employees. You will be responsible for building and maintaining a new AWS serverless data platform. Improve best coding practices within the Business. Stream real-time Data using Kafka. Build and more »

Seize the Opportunity: Innovation Meets Expertise! 🌐 Data Engineer urgently required. Are you fueled by data and innovation? Miller Maxwell's client is your gateway to influence the forefront of trade performance solutions. They are not just hiring— they are cultivating pioneers in a data-driven world. … Miller Maxwell's client seeks to hire a Data Engineer with a knack for SQL Server magic, to join their London squad. Here's a chance to enhance an award-winning SaaS platform, vital to blue-chip giants. You'll be immersed in a milieu where your insights … into trading data aren't just valued—they're pivotal. 📈 Your role? A tech alchemist: 🛠️ Refine their product using SQL Server mastery. 🚀 Pioneer solutions to elevate our platform. 🛤️ Construct new pathways to system excellence. You're not working solo; you're at the heart of a collaborative constellation, interfacing more »

Beyond: Putting Data to Work delivers data-led strategic consulting services globally, bringing cutting-edge data science, engineering, analytics and visualisation expertise by having the highest quality talent, borderless data services and best-in-class creative thinking. We pride ourselves on our ability to swiftly get … We deliberately recruit professionals with diverse experience which makes for a culture of healthy challenge, respect and collaboration. We are seeking a mid-level Data Engineer to join our London based team. As part of the Data Engineering team, you will be working alongside the Technology, Analytics … and Commercial Teams. You will design and create cutting edge data solutions to our clients across various domains. This is a hybrid role where you will be working from our Central London office at least once a week and also on client sites as required. The remainder of your more »

re helping to improve our customers’ lives and contributing to the success of the business every day. Job Description Legal and General’s Group Data and Analytics function is looking to hire a Lead Data Engineer to assume responsibility around the delivery of the infrastructure and data … the internal customers in the Retirement Operations team. This role will see you working closely with the team to improve the current mainframe and data flows, building new rest APIs and removing the click replicate functions which are currently in place while evaluating and making improvements to engineering strategy. … improvement and efficiency Overseeing solution development from design to implementation, ensuring solutions are fit for purpose Identifying and mitigating issues and risks associated with data solutions and projects Engaging stakeholders effectively, providing guidance on data solutions and ensuring productive relationships Engaging stakeholders and undertaking appropriate testing prior to more »

driven crypto technology company that specialise in building trading and MEV-related infrastructure and services, based in London. They are looking to add a data engineer to their tech function. Background in or strong understanding of Crypto and/or the Miner Extractable Value (MEV) is needed - please … only apply if you have good knowledge of the space. The team are focusing on designing, building, and optimizing their data infrastructure. Previous experience in low latency systems, algorithmic trading, or MEV-related systems is a plus but not required. Responsibilities: Oversee the complete process of gathering and analyzing … telemetry data to derive insights regarding company performance Support the development team in resolving issues by using data gathered from various sources Designing, developing, and maintaining scalable and reliable data pipelines and ETL processes Building and managing data monitoring tools to ensure data quality and more »

We are partnered with a reputable global consultancy that are recruiting AWS DATA ENGINEERS to work on a very exciting LONG-TERM CONTRACT within the financial service sector. Role: AWS Data Engineer Rate: Up to £400 per day (inside IR35) Location: London – Hybrid Duration: 6 months (initially … in backlog grooming. Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache Spark Expertise in performing complex data transformation using Spark SQL queries Experience in orchestrating data pipelines using Apache Airflow Proficiency in Git based version control tools Proficiency with Linux … commands and Bash Scripting Working experience in AWS Bigdata services (EMR, Glue, Data Pipelines, Athena, S3, Step Functions etc.) & AWS CLI Experience with CI/CD tools such as Jenkins Experience working with relational and/or MPP databases Knowledge of Data warehousing concepts Experience with CI/ more »

Role Responsibilities: We are seeking an experienced data engineer to join the Surveillance IT team. The successful candidate will be responsible for developing data ingestion pipelines and frameworks that support trade surveillance analysts in their strategic decision-making. Qualifications and Skills: Educational Background : Bachelor's degree in more »

and infrastructure investments. Be part of a dynamic team driving growth and maximizing value in the alternative asset management industry. Key Responsibilities: Support the Data Team Lead in system updates. Develop, modify, and maintain software components. Resolve data/software issues and suggest technical solutions. Collaborate with stakeholders … Conduct peer reviews and offer technical guidance. Identify process improvements and adhere to IT policies. Essential Technical Skills: 2-3 years' experience in Azure data systems. Proficiency in Data Warehouse technologies and SQL. Familiarity with Azure services like Data Lake and Data Factory. Essential Experience: Supporting … traditional Data Warehouses. Building Azure data pipelines and data lakes. Agile development experience. Strong problem-solving skills. Awareness of industry trends and stakeholder interaction. Desired Experience: Familiarity with Business Objects Reporting and Power BI. more »

I'm currently looking for a Data Engineer for an Insurance client of mine based in London. The ideal candidate will have 4+ years of Data Engineering experience and will have experience with DataBricks, SQL and Python. This is a 6-month contract that requires 2 days more »

is fragile and career outlook right now is very uncertain. What about working at a hedge fund where their growth is almost certain? This data engineering team hired multiple people last year – most from big-tech – and plan to do the same this year. A large portion of your … work will technically be non-finance, with the main finance focused work completed by other teams. The real-world, complex data engineering problems are all yours. You’ll be joining a team that builds and has built thousands of ingestion pipelines, with data coming from every single potential … avenue it possibly can. To get the best understanding of an investment, all data points are considered. You’ll bring the data into the business, build the data infrastructure around it and deliver the data to the right person in the business to build their investment more »

in the office Salary: Up to £160k + Bonus A tech-driven investment firm in the city is looking to add a number of Data Engineers to their big build-out. The firm has set up a new team, focused on building a massively distributed cloud-hosted data … Dremio, Snowflake, Iceberg, PySpark, Glue, EMR, dbt and Airflow/Dagster. Cloud: AWS, Lambdas, ECS services This role would focus on various areas of Data Engineering including: End to End ETL pipeline development and deployment onto the Cloud Contributing to a centralised Data platform's architecture and design. … Extract, transform and load data from various sources into the data lakehouse. Metadata Management processes and tools Implementing Data Curation, metadata management and data quality tooling. Requirements: Strong Python/Java Software Engineering skills Excellent AWS knowledge, ideally with exposure to Airflow, Glue, Iceberg and Snowflake more »

I am working with a market leading Advertising client, who are currently hiring a Senior Data Engineer. Data is a key component for this client, and this opportunity will allow you to shape and build there Data platform. You will be involved in the design and construction … of new data capabilities from the ground up, and is a unique opportunity to influence key architectural decisions. The role is split with up to 70% of your time being dedicated to designing and writing software, 30% managing infrastructure. What you'll be working on Designing APIs around data … streaming joins. Building a high throughput service to push events to third parties. Managing key infrastructure such as Kafka, Aurora, DynamoDB, and owning the Data infrastructure. We see around 3 Billion kafka messages per day. Key infrastructure migration and upgrades. Write software using Java/Python/Scala. You more »

Our client are a renowned name in the Property Sector, they are on the lookout for 2 AWS Data Engineers to come in on a contract basis. Key Skills/Requirements: Must have Python & Spark experience Must have strong AWS experience Must have Terraform experience SQL & NoSQL experience Have … built out Data Warehouses & built Data Pipelines Strong Databricks & Snowflake experience Docker, ECS, Kubernetes & Orchestration tools like Airflow or Step Functions are nice to have Contracts are running for 6 months initially, paying up to £450p/day (Outside IR35) and will be mostly remote but ideally looking more »

Keen on testing your data engineering skillset in an autonomous role? You will be leading a developing engineering team at a market-leading trading firm, crafting your own data processes, datawarehouses and pipelines from scratch, whilst acting as the connector between stakeholders and mountains of raw data. Every … an ideal opportunity expert SQL developers who are comfortable with deploying the MSBI stack, particularly SSIS, SSRS and PowerBI. Having exposure to Python for data manipulation and AWS or Azure is also a big plus. You will be using your expertise to shape your team, your projects and the … company’s data dictionary, quickly becoming a fundamental piece of the company’s plans. The day-to-day will incorporate a mixture of hands-on and leadership responsibilities, giving you the control over the direction that you and your team take. You’ll need to come from a strong more »