21 of 21 Data Pipeline Jobs in the City of London

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality frameworks, and reconciliation checks Strong error handling, monitoring, and retry strategies in production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary ...

experienced Data Engineer to design, build, and optimise data solutions within the Microsoft Azure ecosystem. The role focuses on pipeline development, data modelling, governance, and supporting analytics teams with high-quality, reliable data. Key Responsibilities: Develop and maintain scalable data pipelines … data security and governance, including Row-Level Security (RLS) and compliance standards. Maintain documentation and metadata for data assets and pipeline architecture. Collaborate with analysts, data scientists, and stakeholders to deliver trusted data products. Provide technical support and troubleshoot production issues. ...

Data Engineering Technical Lead Global Investment Bank London - Hybrid Permanent - Excellent Package + Benefits We are working with one of the world's leading banking groups, who we have partnered with for 15 years. We are seeking an experienced Data Architect/EDM Developer/Data Engineering Lead to join their International Technology team in London. You will be a key part of the Architecture, Middleware, Data & Enterprise Services (AMD) division, driving data engineering, integration and automation initiatives across our clients EMEA banking and securities entities. This is a hands ...

Senior Data Engineer The Senior Data Engineer will play a crucial role in designing, implementing, and maintaining scalable data pipelines and infrastructure. This position is ideal for those with strong technical expertise and a passion for working in the Insurance/Financial services industry. … Client Details Senior Data Engineer The employer is a medium-sized organisation operating in the F sector. They focus on delivering innovative solutions and maintaining a strong reputation for excellence in analytics and data-driven decision-making. Description Senior Data Engineer Develop and maintain ...

Role title: Snr Data Engineer Security Clearance required: SC Clearance IR35 Status: Inside IR35 Pay Rates: £500/d - £525/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/… large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven expertise with Data bricks, including hands on implementation experience and certifications. Reasonable Adjustments: Respect and equality are core values to us. ...

Data Engineer | Hybrid | London | Databricks | Azure | 85k I'm working with a global powerhouse that's been setting the standard for excellence for over 60 years. With more than 1,000 projects delivered worldwide and a combined value exceeding $150 billion, they've earned a reputation … organisation where world-class projects meet an environment that prioritises your well-being and career development. I'm looking for a Data Engineer who thrives on innovation and loves tackling complex data challenges. If building scalable, cloud-based solutions excites you, this is your chance ...

Data Engineer - 14-Week Contract (Outside IR35) Likely to Extend Start Date: 12th January Rate: £350 per day Location: Remote (UK-based) Interview: Immediate - Offer before Christmas We are seeking an experienced Data Engineer to join a 14-week project focused on building robust data … REST APIs , PIM systems, and potentially SAP. Ensure data quality, governance, and lineage tracking throughout the project. Required Skills ETL/ELT pipeline design and data validation frameworks. Advanced Python (pandas, numpy, boto3) and SQL (complex queries, optimisation). Experience with AWS Glue , Step Functions ...

exciting digital transformation where tech solutions will help enable the business to scale. This will be working across both Software and Data Engineering teams. They build .Net Solutions and use SAS, MicroStrategy and Power BI for Data Analytics/Insights. As they move towards an Agile … engineering and software engineering projects Experience of workflows, orchestrations and data/system integration DevOps and end to end data pipeline Prior experience of Python scripting Previous .Net Development experience nice to have Previously designed solutions to move solutions from on-premise to Cloud ...

Founding Solutions Data Architect London/WFH to £115k Are you a data technologist with start-up experience looking for your next opportunity? You could be progressing your career, in a founding position at a tech start-up that is producing an AI native data pipelining platform. As the Founding Solutions Data Architect you'll play a vital role in helping client data teams to adopt and scale the company's platform. You'll be responsible for a broad range of activities including Solution Design and Architecture, Technical Onboarding ...

About the RoleWe are seeking a Databricks Data Engineer with strong expertise in designing and optimising large-scale data engineering solutions within the Databricks Data Intelligence Platform. This role is ideal for someone passionate about building high-performance data pipelines and ensuring … robust data governance across modern cloud environments. Key Responsibilities Design, build, and maintain scalable data pipelines using Databricks Notebooks, Jobs, and Workflows for both batch and streaming data. Optimise Spark and Delta Lake performance through efficient cluster configuration, adaptive query execution, and caching strategies. Conduct performance ...

leading Financial Services client in London is undergoing a major data transformation and is hiring a Data Engineering Manager on a permanent basis. The role offers a base of £85,000, strong benefits, and flexible hybrid/remote working. Were looking for a hands-on leader … from a forward-thinking, modern data environment with a strong foundation in Data Engineering and experience managing a squad of 47 Data Engineers. Youll bring solid SQL skills, cloud expertise (preferably Azure), and familiarity with tools such as Snowflake, Databricks, ADF, Python, and GenAI. ...

engineering along with apis, optimising the application for performance, arbitrage bots and more. Requirements : 3+ years experience with backend Python : Including async systems, data pipelines, or protocol-level integrations. 1+ year of recent Rust experience : With the ability to read, modify, and contribute to Rust-based components. DevOps ...

Databricks Data Engineer London | Senior Manager | Up to £100K + Bonus Ready to take your data engineering career to the next level? Join a global consultancy on a major transformation project within the insurance domain. This is your chance to work with cutting-edge technologies, influence … make a real impact in a collaborative, forward-thinking environment. Why This Role? Be part of a high-profile project driving innovation in data and analytics. Work with a global leader in digital transformation. Enjoy senior-level responsibilities, clear progression, and exposure to decision-makers. Competitive package ...

Data Scientist Location: Canary Wharf Employment Type: Contract, 2 months, strong chance of extension About the Role Morgan Hunt are working with a leading government organisation to recruit a GIS Data Scientist who can blend spatial analysis, advanced analytics, and problem-solving to turn geospatial data into actionable insights. You'll work with large, complex datasets, build predictive models, and support data-driven decisions across the organisation. If you love maps, patterns, and answering real-world questions with data, this role has your name all over it. Key Responsibilities Acquire ...

Senior Data Engineer (GenAI) - Leading Tech Company | London (Hybrid) Hybrid: 1 day in the office/4 days WFH Permanent role only Represented by Robert Half Limited Please note: No visa sponsorship on offer, No B2B,B2C, Fully remote workers or contractors on offer. The Opportunity … looking for a Senior Data Engineer with hands-on GenAI experience to join a forward-thinking technology business that's building data-driven solutions with real-world impact. This is a highly visible, senior-level role where you'll take ownership of data pipelines ...

role: Own and manage the cloud infrastructure, ensuring scalability, resilience, and security. Implement and maintain high-availability architectures across critical services and data layers. Design and optimise … infrastructure to support AI-powered workloads and data pipelines. Lead cost optimisation initiatives across the platform. Manage the end-to-end deployment pipeline to ensure fast, reliable, and low-friction releases. Maintain, optimise, and automate CI/CD processes. Ensure zero-downtime deployment strategies and effective rollback ...

delivery and oversight of technology supporting AML and Financial Crime functions, including: Owning the system architecture of AML platforms (rule engines, alerting systems, data pipelines). Designing scalable, secure and high-performance systems for transaction monitoring and analytics. Translating regulatory and audit requirements (KYC, CDD, CRR) into actionable … technical logic. Managing integrations with third-party data sources (PEP lists, sanctions vendors, screenings). Acting as a technical intermediary between compliance stakeholders and platform engineering teams. Conducting detailed data analysis in line with business and regulatory needs. Performing feasibility assessments, estimating resources, timelines and system ...

Role title: Snr Data Engineer Security Clearance required: SC Clearance IR35 Status: Inside IR35 Pay Rates: £500/d - £525/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/… Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details ...

technical requirements, this isn't essential. Key Responsibilities Own the end-to-end system architecture for anti-money laundering platforms (rule engines, alerting, data pipelines) across banking and securities under a dual-hat remit. Design scalable, secure and high-performance solutions for transaction monitoring, sanctions screening and batch … planning: break large initiatives into phases, assess feasibility, estimate effort and impact, and propose new technologies where appropriate. Oversee integrations with third-party data providers (e.g. sanctions/PEP list vendors) and ensure data quality, lineage and reconciliation. Prepare and maintain operational documentation (HLD/ ...

work through an umbrella company for the duration of the contract. You must be able to implement and support content-related configurations and data pipelines, ensuring structured metadata and reliable delivery. You will have experience of designing and maintaining schemas, mappings and validation rules for content. You will … also be able to build/operate ETL/ELT jobs, ensuring data quality and lineage. You will collaborate with editors and product teams on release readiness and automate checks and monitoring. Experience of Data Engineering and SQL, Python and ETL tools. You must have knowledge ...

Time Top-Tier Compensation + Performance Bonus We're working with a high-performing TOP hedge fund that thrives at the intersection of data, technology, and rigorous research. They're lean, agile, and scaling-backed by performance, not bureaucracy. They're now hiring a Python Developer to help … performance research and trading platforms in Python Collaborate directly with quants to prototype and deploy new investment models Design and maintain fast, reliable data pipelines Drive improvements across the firm's core technology stack and system architecture What You'll Bring: Advanced Python skills Strong academic background - STEM ...