1 to 25 of 272 Data Pipeline Jobs in London

member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Title: Data Architect – Azure Databricks/Azure Data Factory For London Market Insurance Exp Required – 10 years Plus Location: London About the Job you are considering … optimisation Unity Catalog, if applicable Databricks SQL, if applicable Cluster sizing and cost optimisation Security and access control patterns Azure Data Factory Pipeline orchestration Linked services and datasets Integration runtime Parameterised pipelines Triggers and scheduling Error handling and retry patterns Metadata-driven ingestion frameworks Monitoring and operational ...

Principal Data Architect Location: Remote Duration: 6 months £700 - £800 per day - Umbrella only Our client is looking for Principal Data Architect for Data Science family under DSF as part of Professional services sell. This request for NGET under the department of Customer Network … GDPR and Regulatory/CNI control environments In-depth experience in providing resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface. Demonstrable ability to troubleshoot problems across infrastructure, platform, and application domains. Controlling access to the data ...

platforms so they are reliable, secure, and easy to change. You will work across engineering and operations to automate delivery, improve data pipeline observability, and embed good governance so analytics and AI workloads can run at scale. You will be part of the Data Platforms … restricted environments. Delivering data applications in an automated and continuous fashion. Utilising tools like Airflow for orchestration and automating data pipeline deployments using DevOps tools and techniques (Docker, Containers, Terraform, Kubernetes) Building data pipelines using both cloud-native and open-source data ...

largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Who You Will Be Working With The Data Platforms team is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety … projects and sectors. Data Platforms is the home of the Data Engineers, Platform Engineers, Solutions Architects and Business Analysts who are focused on driving our customers digital and data transformation journey using the modern cloud platforms. We specialise on using the latest frameworks, reference ...

Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting ...

Lead Data Engineer Location: London, City (Hybrid) £ Highly competitive base with bonus and benefits Lead Data Engineer is sought by expanding central data platform team within a prestigious global wealth management organisation. This role offers the opportunity to help shape enterprise-wide data engineering standards, contribute to platform strategy, and build modern, scalable data solutions that support business-critical decision-making. Essential to this role are strong python skills, expertise in MS Fabric, SQL, data warehousing, Data Lake and Azure technologies. The Role ...

identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/… concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/GOV.UK/Public Sector domain experience strongly preferred. Must be active SC Cleared or fully eligible ...

identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/… concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/(url removed)/Public Sector domain experience strongly preferred. Must be active SC Cleared or fully ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Data Migration Lead London (Hybrid) £75,000 - £90,000 + benefits We're partnering with a fast-growing fintech organisation building a modern, cloud-native banking platform used by financial institutions across the UK. Following continued client growth and major transformation programmes, they are looking to hire … senior data leader to own and drive data migration and broader data delivery initiatives across complex banking environments. This is a high-impact, business-critical role sitting between data, product, and programme delivery. The successful candidate will act as the primary owner ...

Azure/Microsoft Fabric Data Engineer - remote - outside IR35 - £500 - 535 (and a perm role - up to £80k) You will be an experienced Azure/Microsoft Fabric Data Engineer and will help design, build and maintain modern data platforms. The role will focus … ensure data is handled securely and appropriately. Help create reusable patterns, templates and accelerators for Fabric-based deployments. Troubleshoot data pipeline failures, performance issues, permissions problems and integration challenges. Contribute to technical documentation, solution designs, deployment notes and customer-facing architecture material. Desirable Skills Microsoft ...

exciting technology-led organisation in London is looking for a Lead Data Engineer to join their growing platform team as they continue to build out their data and analytics capability. Based in: London Hybrid working: 1-2 days a week in the office Permanent role Start … delivery of a modern cloud data platform supporting analytics, AI and client-facing data products Owning data pipeline architecture across ingestion, transformation and data product layers Setting and maintaining data engineering standards, patterns and best practices across squads Mentoring ...

Azure Data Engineer London (Hybrid) You'll design and build scalable, automated data pipelines that bring together information from across the organisation. It's a hands-on engineering role covering everything from data ingestion to transformation, modelling and delivery, all within a modern Azure … environment. You'll report to the Lead Data Engineer and join our Data Engineering Team, working alongside architects, analysts, testers and agile delivery squads. Responsibilities: * Build and maintain scalable, automated data pipelines using Azure tools like Data Factory, Databricks and Logic Apps ...

enjoys working in complex, enterprise-scale environments with high standards for quality and reliability. Key Responsibilities Design and build scalable, reusable data pipeline templates across multiple data domains Standardise ingestion and transformation using configuration-driven frameworks Embed data quality checks by default (schema … validation, completeness, freshness, thresholds, alerting) Improve pipeline resilience, monitoring, observability and recovery mechanisms Integrate AI/ML capabilities where appropriate (e.g. anomaly detection, intelligent monitoring) Support delivery of a wider Data Strategy programme , improving consistency and data quality Ensure pipelines are production-ready, documented ...

Robert Half Technology are assisting UK's market leading legal organisation to recruit a Data Engineer on a 12 month contract basis - Hybrid working - London based. A Data Engineer is required to join a growing data team and play a key role in expanding … Experience with data modelling principles and best practices. Hands-on experience working with enterprise data platforms and data pipeline development. Experience working with Azure Databricks. Strong problem-solving skills and attention to detail. Ability to work collaboratively within a cross-functional environment Experience ...

Contract Principal Data Scientist (Senior Data Scientist) The Opportunity We're looking for a Principal Data Scientist (Senior Data Scientist level) to lead the development of advanced analytics, AI models, and data-driven solutions that deliver real organisational impact. This … role sits at the intersection of data science, AI, and business strategy , combining technical expertise with leadership to shape high-value initiatives and unlock insights from complex data. What You'll Be Doing Build strong relationships with stakeholders to identify opportunities where data science ...

Data Engineer £70,000 - West London (Hybrid - 1 day onsite) We are seeking a Data Engineer to join a well-established, international organisation operating within a sustainability-focused industry. The business is undergoing a significant digital and data transformation, with major investment in modern … solutions in fast-paced environments Hands-on experience with Microsoft Fabric Proven track record working with ETL processes, data integration and pipeline development Advanced SQL/T-SQL skills Experience designing and implementing data models to support reporting and analytical use cases Ability ...

Snowflake Data Architect Location: Wembley, London - FULLY ON-SITE Salary: Up to £95,000 + Benefits We are seeking an experienced Snowflake Data Architect to play a pivotal role in shaping and delivering a modern enterprise data platform. This is an opportunity to join … business undergoing significant data transformation, moving from legacy systems and manual processes to a scalable, automated, cloud-first architecture. You will be responsible for defining architectural standards, driving best practices, and enabling engineering teams to deliver high-quality, trusted data products that support analytics, reporting ...

summary Are you passionate about using data science to improve healthcare? Join NHS England's data science teams and play a key role in tackling complex challenges, delivering high-quality insights, and building tools that support better decisions across the NHS. We value modern ways … working, including transparency, reproducibility, agile delivery, and open-source tools. As a Band 7 Data scientist (Data Wrangler), you'll collaborate with multidisciplinary teams, developing advanced data science methods, building data pipelines, and producing impactful analytical products. If you want to make ...

Microsoft Fabric | Azure | Lakehouse | Data Engineering Not all Data Engineering roles are created equal. Some involve maintaining legacy ETL processes and producing reports nobody reads. This isn't one of those roles… You will help shape and enhance a modern cloud-based data platform … that's central to how a large, complex organisation makes decisions, improves services and drives transformation. You'll be joining a well-established data function where Data Governance, Architecture, Engineering and BI all work together to build something genuinely valuable. The foundations are already in place. ...

long-term, high-impact programme within a secure environment. This role is ideal for someone with strong backend engineering and data pipeline experience, combined with modern Java-based development skills. You will play a key role in designing, building, and optimising scalable data solutions that … experience with SQL and data modelling concepts Strong analytical and problem-solving skills Nice to Have Skills Experience with data pipeline tools (Airflow, Kafka, Spark) Knowledge of cloud platforms (AWS, Azure or GCP) Familiarity with relational and NoSQL databases Additional programming experience (Python, Scala) Essential ...

Data Scientist (Palantir/FDP essential) - £560/day (Inside IR35) - RemoteYou will require Palantir (FDP) experience as well and UK Citizenship to apply I'm currently looking for an experienced Data Scientist with strong Data Engineering capability to join a high-profile public … considered, your CV must clearly show experience with the following: *Palantir (FDP) *Python *PySpark *Exploratory Data Analysis *Data pipeline design/data engineering concepts *Dashboard/reporting dataset development *BI tools (Redshift, Databricks or similar) *Agile delivery environments If these technologies aren ...

Data Solution Architect Remote working £650-675/day overall assignment rate to umbrella 3-month initial contract Working with a market leading client who have a requirement for a Data Solution Architect to design, lead and deliver scalable, enterprise-grade data platforms. This … with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

Senior Business Data Analyst Contract: £450-£550 per day Location: London, 1 day per week We are currently looking for a Senior Business Data Analyst (Data Architecture & Discovery) to join a, data-driven team on an initial 3-month contract focused … senior business stakeholders, this person will play a key role in helping the organisation understand, document and future-proof a complex reporting and data infrastructure environment. The successful candidate will act as the bridge between commercial teams and technical data specialists, helping to untangle legacy data ...

DATA ENGINEER Up to £60,000+ Benefits London (Hybrid) This is a great opportunity to join a high-impact regulatory organisation where you can take ownership of large-scale data platform modernisation and transformation initiatives. THE COMPANY: This organisation operates at the heart … healthcare landscape, ensuring high standards of professional practice and safeguarding public health. They are currently undergoing an ambitious transformation of their data and technology function, with substantial investment being made into modern cloud technologies, data platforms, and analytics capabilities. This transformation is enabling the organisation ...