1 to 25 of 76 Data Pipeline Jobs in the City of London

data-enabled products. Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures. Work closely with Data Architects to implement target-state data models and modern data engineering principles. Partner with … Factory Azure Synapse Analytics Microsoft Fabric Azure Data Lake Advanced SQL development skills. Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation. Track record of building reliable, monitored production-grade data pipelines. Experience integrating data ...

Manager - AI & Data Architect £70,000 to 90,000 GBP Bonus Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Manager - AI & Data Architect Location: London | Hybrid Salary: Up to £90,000 + bonus We're working with a leading change and transformation consultancy … that is looking to hire an experienced AI & Data Architect into its Technology & Data practice. This is an excellent opportunity for a Data Architect from a consultancy background who enjoys working directly with clients, solving complex business problems and designing modern, scalable data ...

About Billigence: Billigence is a boutique data consultancy with global outreach and clientele, transforming the way organisations work with data. We leverage proven, cutting-edge technologies to design, tailor, and implement advanced Business Intelligence solutions across Cloud Data Warehousing, Visualisation, Data Science, Data Engineering, and Data Governance. Headquartered in Sydney, Australia, with offices around the world, we help clients navigate complex business challenges, eliminate inefficiencies, and enable scalable, data-driven decision making. About the Role: We are looking for two experienced Data Engineers to join ...

Lead Data Engineer or architect at Made Tech, you'll play a pivotal role in helping public sector organisations become truly data‐lead, by equipping them with robust data platforms. You'll also join a data team on its mission … data knowledge and skills out of silos and embedded into delivery teams. You'll also help implement efficient data pipelines & storage. Key responsibilities Define, shape and perfect data strategies in central and local government. Help public sector teams understand the value of their data ...

production batch ETL/data pipelines Write and optimise SQL across large-scale datasets Use Python for data transformation and pipeline orchestration Maintain and improve existing pipelines, including backfills and schema changes Troubleshoot data-quality and pipeline issues Implement data … Collaborate with Data Engineers, Analysts and Engineering teams The core work is production data engineering: ETL pipelines, large-scale SQL, pipeline changes, data-quality troubleshooting and code review. What we’re looking for 5+ years' experience building production data pipelines Advanced ...

excellent opportunity for an engineer with strong Python and SQL skills who wants to build broader expertise across software engineering, cloud, data engineering and AI-enabled applications . Rather than focusing on one part of the technology stack, you'll work across backend services, APIs, databases, data … applications and contributing to client projects. Why this role? You’ll gain hands‐on exposure to: AWS cloud & migration Python development SQL, databases & data platforms Data pipelines & ETL/ELT APIs & backend engineering CI/CD & production systems Analytics & modern data platforms AI/ ...

Senior Data Engineer This Senior Data Engineer role focuses on designing, building, and maintaining strong data pipelines and systems to support the analytics department in the property industry. You'll work closely with cross-functional teams to ensure data is accessible, reliable … ready to drive informed business decisions. Client Details Senior Data Engineer Our client is a well-established, nationally recognised organisation undergoing a significant data and technology transformation. Description Senior Data Engineer Develop and maintain scalable data pipelines to support analytics and business ...

Data Engineer London - Hybrid £65,000 - £75,000 Azure Data/SQL/Financial Services I'm currently working with a growing data and technology consultancy who are looking to bring a Data Engineer into their expanding delivery team. The business works … with a range of organisations across telecommunications, financial services, insurance and healthcare, helping clients develop modern data platforms and turn complex data into valuable business insights. Joining a client-facing consulting team, you'll have the opportunity to work across a variety of data ...

seeking a highly technical Business Intelligence Engineer to join our growing BI team. Sitting at the crucial intersection of Data Engineering and Business Intelligence, you will be the technical cornerstone of our BI function. You will architect robust data solutions, automate data ingestion, quality … governance checks, build user-friendly dashboards, and promote a culture of data-driven decision-making across the organization. You will be a self-starter, taking the initiative to understand the whole data lifecycle, Trayport's products and business to transform complex data into actionable ...

About the Role Grade Level (for internal use): 12 Role Overview: The Finance Associate Director, Sales Compensation (Data Architecture & Strategy) serves as the primary business partner for the S&P Global Market Intelligence division, focusing on data architecture, database building, and continuous systems maintenance. This position … leadership; proactively detect and resolve data anomalies (e.g., double‐counted revenue, incorrect territory mapping) before they impact calculations. Lead end‐to‐end pipeline and calculation troubleshooting with Commissions and Systems Analytics to ensure clean, validated data foundations. Lead process design and standardization by converting ambiguous ...

creating exceptional places for students to live and learn, and this opportunity plays a key role in helping us harness the power of data to drive insight, innovation and operational excellence across our business. We are looking for a Data Engineer to join our Data & Business Intelligence team and help develop, maintain and optimise our cloud-based data platform. Working within a modern Azure and Databricks environment, you'll design and maintain scalable data pipelines, transform and model data, and ensure our data architecture provides ...

hands-on SME expertise throughout the Monte Carlo implementation. Advise delivery teams and stakeholders on appropriate configuration and implementation approaches. Work closely with data and technology teams to understand requirements and translate them into effective Monte Carlo solutions. Support the implementation of data observability, monitoring … data quality capabilities. Review existing configuration and implementation approaches and recommend improvements. Troubleshoot issues relating to Monte Carlo configuration and implementation. Support the integration of Monte Carlo within the wider Aladdin/Snowflake data environment. Provide documentation and knowledge transfer to internal teams. ...

patterns and leaves behind maintainable, standards-compliant assets. Maintain delivery rigour: shape sprint cadences, elevate technical risk early, and report on pipeline SLOs, data freshness, and DORA-style engineering metrics to the Head of Data Engineering and CDO. What you’ll need for this role … standards, conduct code and design reviews, and grow engineers’ capabilities Strong grasp of data quality practices: data contracts, pipeline observability, SLA/SLO definition, and automated alerting and remediation Solid understanding of SQL transformation patterns and modern tooling such as dbt, alongside experience managing ingestion ...

Data Architect London - £70-79k + Benefits We are currently working with a highly regarded organisation that is actively recruiting for a Data Architect to join on a permanent basis. The organisation is in the process of evolving from traditional data technologies towards … focus on Fabric, Synapse and AI automation, the Data Architect will be vital in driving this progress. The role holder will be a key player in a number of vital projects in the pipeline. Our client operates a hybrid working policy with two days per week on-site. ...

solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Role Specification - Data Architect (Data, Analytics & AI) Role Overview We are seeking a Data Architect to support the design and delivery of cloud … based data, analytics, and AI/ML data platforms. The role focuses on defining scalable data architectures and guiding engineering teams across GCP, Snowflake, and dbt-based ecosystems in a regulated enterprise environment. Key Responsibilities Define end-to-end data architecture across ...

Description Canon EMEA is looking for a talented Data Engineer to take ownership of the design, development, and maintenance of robust data pipelines and cloud-based infrastructure across Google Cloud Platform (GCP) and Microsoft Azure. In this role, you will play a key part in ensuring … seamless data integration and accessibility, enabling Canon's ITCG Marketing, eCommerce, and CRM teams to make data-driven decisions and deliver impactful customer and commercial initiatives. This is an opportunity to join a globally recognised brand where your expertise will directly support marketing effectiveness, customer engagement ...

Working Environment Data engineering work is grounded in real operational needs, often involving imperfect data, technical constraints and organisational complexity. Engineers work closely with delivery teams and stakeholders to understand problems end to end and deliver robust, production-grade solutions. Collaboration, pragmatism and persistence are valued … alongside strong engineering craft. What You'll Be Doing - Design, build and improve enterprise-scale data pipelines, platforms and services - Lead complex data engineering work end to end, from problem definition through build and operational use - Work directly with technical and non-technical stakeholders to translate ...

financial insights for stakeholders across Trayport. As a core member of the BI team, you will be instrumental in transforming complex, often unstructured data into clear, actionable insights that drive strategic business decisions. You are a self-starter who thrives in ambiguity and isn't afraid of "messy … data. More than just a technical expert, you act as a bridge between our data infrastructure and the wider business, seamlessly adapting your communication style to collaborate with both technical engineering teams and non-technical stakeholders. Key Responsibilities End-to-End Analytics: Design, develop, and maintain interactive, high ...

This is a new and exclusive opportunity for a Senior Data Engineer with strengths in Microsoft Azure data to be a key lead in strengthening, evolving, and modernising our core data platforms. Job details Location: London or Glasgow with 1 day a week … office Permanent role salary £50- £70,000 dependant on skills and location Requirements: Strong experience as a Senior Data Engineer in enterprise data environments. This is a new opportunity to join a thriving team to be a key voice in modernizing core data platforms ...

Data Engineer - London Salary: £60,000-£75,000 Dependent on Experience London: Hybrid working - 3 days per week in the office, 2 days home-based Job Ref: J13171 An exciting opportunity for a Data Engineer to join a growing technology business specialising in consumer data and analytics. Working within a small, talented team of engineers, analysts and data scientists, you will play a hand-on role in developing and improving the data platform, building scalable data pipelines and working with large volumes of consumer data. Strong coding ...

Senior Data Engineer – Databricks & DLT (Delta Live Tables) Certain Advantage are recruiting on behalf of our trading client for a Senior Data Engineerwith strong analytical capabilities and hands‐on experience in building scalable data solutions using a programming language like Python or Scala … well as experience in Databricks and Delta Live Tables - This experience is essential. Suitable candidates will have expertise inDatabricks,streaming architectures, data ingestion frameworks, anddashboarding, along with experience working withSAP data sources. You will be responsible for designing and implementing scalable data pipelines ...

Role: We are looking for an experienced, highly motivated Python developer with expertise in building robust data pipelines, creating and manipulating DataFrames, and turning defined specifications into reliable, production grade scripts. The role centres on developing and orchestrating the data infrastructure behind a systematic research platform … with Python, especially Pandas and NumPy Mastery of SQL and PostgreSQL: schema design, complex queries, indexing and performance optimisation, beyond basic querying Experience with pipeline orchestration and scheduling Proven ability to deliver reliable, tested scripts against a written specification, and to migrate them cleanly from a test environment ...

Azure Data Support Engineer Location: Newcastle or London preferred. Job Type: Full-time | Permanent (On-Call Requirement) Security Clearance Requirement: This role is suitable for candidates who already hold UK Government Security Clearance (SC), or who are eligible and willing to undergo the SC vetting process (eligibility criteria … apply). Key Responsibilities Azure Platform Support & Monitoring Support and maintain Azure-based data solutions, including Azure Data Factory pipelines, datasets, linked services, triggers; Azure Databricks (Spark jobs, notebooks, clusters); Azure Machine Learning models and endpoints; Power BI dashboards and dataset refreshes. Monitor and troubleshoot failures ...

Lead Data Engineer - London London | Hybrid - Monday to Thursday in the office. Friday work from home Salary | £85,000-£95,000 dependent on experience Ref J13147 Build the function. Lead the team. Stay hands on. Join a fast-growing AI SaaS business where Data Engineering sits … heart of the platform. This is an opportunity to join at a pivotal stage of the company's growth. As Lead Data Engineer, you'll shape and scale the Data Engineering function, define engineering standards and build a high-performing team, while remaining hands on with ...

savings, budgeting, financial coaching & rewards. Stream - Employee financial wellbeing support: early access to your earned pay, savings, budgeting, financial coaching & rewards. Responsibilities: Data ingestion: Work to build, test and implement solutions for new data acquisition, building end to end process to support the entire department. Using … python, connect to various third party APIs, returning complex data from multiple endpoints. Implement processes to retain and refresh this data within business systems. Data Modeling: Using the businesses preferred ETL tools, generate curated datasets from raw input tables. Ensure all datasets have clear ...