526 to 550 of 628 ETL Jobs in London

systems for better decision-making. MDM and Data Migration: Experience in delivering MDM and data migration programs using both standard and innovative approaches. ETL Technologies: 8-10 years of experience with recognized enterprise ETL technologies such Azure Data Factory, Talend, PL/SQL etc. Data Profiling and Cleansing: Expertise ...

Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases. Experience designing and supporting ETL/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of Autosys and Apache Airflow (monitoring schedules, reruns, failure triage). … OpenShift or equivalent). Familiarity with AWS S3 for large‐scale data storage and dataset movement patterns. Experience supporting Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice‐to‐Have Experience in regulated or enterprise data environments (banking, compliance). Exposure to data governance, lineage, controls frameworks ...

querying, reconciliation logic, data validation). Strong proficiency in Python for data analysis and automation (pandas, validation frameworks, scripting). Experience supporting or validating ETL/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Exposure to lineage and metadata approaches; ability to validate transformations and trace data … large-scale storage patterns (e.g., AWS S3) for dataset movement and controls. Experience supporting BI/reporting outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Hands ...

experienced Lead Data Engineer to join us on a permanent or contract basis. You’ll lead the design, build and operation of data platforms, ETL pipelines, analytics layers and data-driven digital solutions. Working closely with clients and fellow engineers, you’ll shape the architecture and lead hands‐on delivery … lead role. You have a deep understanding of data architecture and engineering best practices, and extensive experience designing and building cloud data platforms and ETL pipelines that are scalable, efficient, robust and extensible. You’ll need significant recent hands‐on experience using AI tools and techniques to accelerate data architecture ...

doing Leading, mentoring and growing a team of Data Engineers Designing and building scalable data pipelines and cloud based data platforms Developing robust ETL and ELT solutions using Python and SQL Defining engineering standards, data models and best practice Driving CI/CD, automation and modern engineering practices Collaborating with … quality and delivery What you'll bring Strong experience with Python and SQL Expertise building modern data platforms on Azure, AWS or GCP Strong ETL and ELT engineering experience Data modelling and data warehousing expertise CI/CD, Git and engineering best practice Previous experience leading, mentoring or developing engineers ...

cases by providing reliable, governed and timely security data. Key responsibilities · Build scalable batch and streaming pipelines for security telemetry and enterprise data. · Develop ETL/ELT processes, data models, APIs and integration services. · Integrate platforms such as ServiceNow, Dynatrace and security tooling into enterprise data ecosystems. · Build and maintain … experience, including substantial hands-on experience in the core specialisms of this role. · Strong data engineering experience with Python, SQL and modern data platforms. · ETL/ELT, APIs, event streaming and cloud data engineering. · Knowledge of ML engineering and feature/data pipeline requirements. · Graph databases and Security Knowledge Graph ...

Architect - Manager Design and implement end-to-end data solutions from ingestion and processing through to analytics and visualisation. Design scalable data pipelines and ETL processes across modern and legacy platforms. Develop conceptual, logical and physical data models, including dimensional, relational and data vault. Own end-to-end data architecture … architecture methods, tools and technologies. Experience you'll need as a AI & Data Architect - Manager 5+ years experience in data engineering and architecture , including ETL, data pipelines and data storage. Experience leading data engineering projects or workstreams. Strong experience with cloud platforms and cloud data services . Experience with modern ...

technologies, contribute ideas and have a genuine impact as the business continues to scale. Key Responsibilities Design, build and optimise scalable data pipelines and ETL/ELT processes. Develop and maintain data engineering solutions using Python, SQL and Spark. Work with large and complex datasets, ensuring data quality, reliability and … working with Spark for large-scale data processing. Hands-on experience working with AWS data services. Experience designing and developing scalable data pipelines and ETL/ELT processes. Practical experience using AI-assisted development tools such as Claude within your day-to-day work. Strong communication skills and the ability ...

while having the technical ability to understand, review and challenge the code they produce. Key Responsibilities Design, build and optimise scalable data pipelines and ETL/ELT processes. Develop and maintain data engineering solutions using Python, SQL and Spark. Work with large and complex datasets, ensuring data quality, reliability and … working with Spark for large-scale data processing. Hands-on experience working with AWS data services. Experience designing and developing scalable data pipelines and ETL/ELT processes. Practical experience using AI-assisted development tools such as Claude within your day-to-day work. Strong communication skills and the ability ...

and cloud platforms, with a keen eye for reliability and compliance in a global context. Key Responsibilities Design and implement robust data pipelines and ETL processes to support analytics, reporting, and ML model training. Build and optimize scalable data storage and retrieval systems to handle high-volume, multi-market datasets. … degree in Computer Science, Data Engineering, or a related field. 3+ years of experience in data engineering, big data infrastructure, or large-scale ETL development. Proficiency in SQL and programming languages such as Python, Java, or Scala. Hands‐on experience with big data frameworks (e.g., Spark, Hadoop, Flink) and cloud ...

applying robust software engineering practices. Key Responsibilities Design, develop, and optimise scalable, testable data pipelines using Python and Apache Spark. Implement batch workflows and ETL processes adhering to modern engineering standards. Develop Cloud-Based Workflows Orchestrate data workflows using AWS services such as Glue, EMR Serverless, Lambda, and S3. Contribute … code, ideally using type hints, linters, and test frameworks such as pytest. Solid understanding of data engineering fundamentals including batch processing, schema evolution, and ETL pipeline development. Experience with—or strong interest in learning—Apache Spark for large-scale data processing. Familiarity with AWS data ecosystem tools such ...

while having the technical ability to understand, review and challenge the code they produce. Key Responsibilities·Design, build and optimise scalable data pipelines and ETL/ELT processes.·Develop and maintain data engineering solutions using Python, SQL and Spark.·Work with large and complex datasets, ensuring data quality, reliability and … working with Spark for large-scale data processing.·Hands-on experience working with AWS data services.·Experience designing and developing scalable data pipelines and ETL/ELT processes.·Practical experience using AI-assisted development tools such as Claude within your day-to-day work.·Strong communication skills and the ability ...

ETL Data Engineer – FintechTo 75,000 + 10% no-cont pension, bonus and sharesRemote with occasional travel to the London Office. Quant Capital is urgently looking for a Data Engineer to join our well-known Fintech50 client who produces software disrupting the wealth management market. My client has dominated … streaming data pipelines in AWS cloud environmentData AnalysisData modelingThe Data Engineer Must have: Comp Sci or Engineering DegreeExperience in building complex data pipelines/ETL/ELT scriptsExtensive experience with SQLExcellent knowledge of AWS and components such as S3, Kinesis, DMS, SNS, Glue, Athena, RDSExperience with Data Warehouse databases like ...

enterprise IT systems. You will scope data flows, design robust connections and guide customers through real‐world configuration. The role requires strong API/ETL expertise, cross‐functional collaboration with success, product and business development, and the ability to work across time zones. #J-18808-Ljbffr ...

time role delivers risk, valuation and performance attribution implementations for our growing EMEA client base. You will coordinate projects, mentor junior team members, design ETL processes, and translate complex technical requirements into business outcomes while collaborating with product, engineering and sales to ensure successful delivery. #J-18808-Ljbffr ...

associated non-financial risk. Working knowledge of mathematics including statistics, time series analysis and numerical algorithms. Experience with programming in Python and SQL for extract transform load (ETL) operations and data analysis (including performance optimization). Experience in using languages such as R, Java, C++ is beneficial. Experience in developing ...

role: Work with business users and the BI team to deliver BI, DWH and EDP solutions that meet business needs. Build and enhance ETL/ELT processes using SQL Server, SSIS, Azure Data Factory and related cloud services. Support API-based integrations, with Workday integration experience considered good to have. … requirements, technical standards and agreed delivery priorities. Designs are delivered to agreed standards and timescales, with peer review completed where required. Develop and enhance ETL/ELT components across SQL Server, SSIS, ADF and Fabric where appropriate. ETL/ELT components are tested, documented and deployed through agreed release processes. ...

Data and Platform Engineering team are the foundation for the Data Office function. Responsible for designing, building, and maintaining PfP's data platform, we extract data from source, transform it into a usable format, load it into consumer models and marts and build and manage the infrastructure … Lead/Principal level, although this is not a requirement. The successful candidate will have a deep understanding of data concepts including Data Modelling, ETL/ELT, Data Warehousing, Data User Requirements, Quality Assurance, CI/CD and more. You will be expected to get into detailed discussions with Engineers ...

and maintain highly scalable data management systems. Ensure systems meet business requirements and industry practices. Design, implement, automate and maintain large scale enterprise data ETL processes. Build high-performance algorithms, prototypes, predictive models and proof of concepts. Build, test and maintain batch ETL pipelines for assigned datasets, following existing teampatterns ...

work closely with the Technical Lead and Lead Data Scientist to deliver reliable, production-ready data solutions. Key Responsibilities Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue, Redshift, Python and SQL. Ingest, transform and validate data from multiple sources and formats. Implement data matching, deduplication and … desirable) Strong hands-on experience with AWS Glue, Amazon Redshift, Python and SQL . Proven experience building production-grade data pipelines. Strong understanding of ETL/ELT, data modelling and data quality. Experience with data matching, record linkage or complex data transformations. Experience with automated testing, Git and CI/ ...

solutions and build scalable data pipelines, transforming complex datasets, and implementing data-matching and record-linkage solutions. Key Responsibilities Design, develop and maintain scalable ETL/ELT pipelines using Spark and Python Scripting. Ingest, transform and validate data from multiple sources and formats. Implement data matching, deduplication, and record-linkage … with Spark, Python and SQL . Experience working within Agile/Scrum delivery teams. Proven experience building production-grade data pipelines. Strong understanding of ETL/ELT, data modelling and data quality. Experience with data matching, record linkage or complex data transformations. Experience with automated testing, Git and CI/ ...

ability to investigate complex data issues, identify root causes and work across technical teams to implement sustainable solutions. Key Responsibilities Design, build and optimise ETL/ELT data pipelines across multiple data sources Write and optimise complex SQL queries , including joins, window functions and performance tuning Translate business requirements into … Strong hands-on SQL skills, including advanced querying and performance optimisation Commercial Data Engineering experience within complex enterprise environments Strong experience designing and developing ETL/ELT pipelines Data warehouse and/or domain data platform experience Strong understanding of dimensional and/or domain-oriented data modelling Hands ...

driven transformation across critical UK industries. Core Responsibilities Design, build, and optimise data pipelines using Azure Data Factory, Synapse, and Databricks. Develop and maintain ETL/ELT workflows to ensure high data quality and reliability. Collaborate with analysts and AI engineers to deliver robust and reusable data products. Manage data … data architecture, APIs, and modern data pipeline design. Strong experience within Microsoft's Azure ecosystem, combined with a drive to explore Fabric, AI-enhanced ETL, and real-time data streaming. Demonstrated ability to lead technical workstreams and mentor junior colleagues. A deep commitment to Movar's IDEAL values: Integrity, Drive ...

Glue, Kinesis, QuickSight, OpenSearch). Lead analytics modernisation — guide customers migrating from legacy platforms (Teradata, Oracle, Netezza, Hadoop) to cloud‐native analytics, including ETL refactoring and performance validation. Partner with AWS AGS sales teams — collaborate with Account Managers and Account Solutions Architects to identify analytics opportunities, shape deal strategy, and … across technical and non‐technical audiences, including executive level stakeholders or clients Experience architecting analytics or data platform solutions (e.g., data warehousing, data lakes, ETL/ELT pipelines, BI, or real‐time streaming) Experience delivering technical presentations, workshops, or thought leadership content to senior technical and business audiences Preferred Qualifications ...

Defining and enforcing development best practice and coaching team members to ensure consistency. Working with project architects, taking responsibility for non-functional needs of ETL/ELT data processing pipelines such as robustness and performance. Taking responsibility for standards and execution of unit and integration testing done within the team. … development experience in one of Java, Scala, or PythonSoftware development experience with data-processing platforms from vendors including Informatica,Azure Databricks or any relevant ETL toolsExpert in SQL or SQL extensions for analytical use caseExpert understanding of distributed data stores and data processing frameworksAbility to simply and clearly communicate technical ...