12 of 12 Data Lake Jobs in the City of London

Role –Senior Databricks Architect Technology – Databricks (Spark, PySpark/SQL) and data lake architectures Location – London, UK Duration-Permanent Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling … support analytics, reporting, and domain-owned data products. Primary Skills • Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. • Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. • Experience integrating legacy data sources such as DB2 (mainframe ...

Senior Data Engineer Market Data 5 day onsite We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process … actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. Experience building scalable, production data ...

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … maintain scalable data pipelines using Databricks on AWS. Build and optimize data processing frameworks using PySpark, Spark SQL, Python, Delta Lake, and Databricks notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such ...

Azure Data Engineer Rate - up to £350/day Outside IR35 Location - 2 days a week onsite in Central London Duration - 6 Months + Likely Extension We are working with a well-established consultancy supporting a high-impact data transformation programme for an end client based … Analytics . Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2), SQL-based systems and APIs. Collaborate with Data Architects, Analysts, and stakeholders to define data models and ensure best ...

Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data … analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling. Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data ...

utilizing Azure Data Factory (ADF). ETL/ELT Development: Build and maintain seamless pipelines across SQL Server, Azure Data Lake Storage (ADLS), and Snowflake ecosystems. Automation & Frameworks: Build reusable frameworks for data onboarding, schema deployment, and validation. Automate operational processes using … role, you must have strong hands-on experience in the following areas: Core Technologies: Azure Data Factory (ADF), Azure Data Lake (ADLS), Azure SQL, and core ETL concepts. SQL Server Mastery: Advanced SQL Server development and administration skills, including hands-on experience managing SQL Server ...

London (Hybrid - 1-2 days per week onsite) Salary: £90,000 - £100,000 + Excellent Benefits Job Type: Permanent Are you an experienced Data Architect looking to shape the future of enterprise data platforms? We're working with a leading global organisation that is investing heavily … cloud, data and AI transformation. As part of this growth, they're looking for a Senior Data Architect to design and deliver enterprise-scale data platforms that power advanced analytics, machine learning and AI solutions. This is an opportunity to work on large-scale ...

successful multinational technology business is looking for a Data Reliability Engineer to join its growing data team in Central London. This role is hybrid youll be able to work from home 2 days per week. This is a high-impact role focused on improving data quality, reducing incidents, and building scalable observability across a modern enterprise data platform. Youll help ensure data across the organisation is accurate, reliable, and trusted for critical business decision-making. Youll take ownership of data reliability end-to-end, designing and implementing ...

Applied Data Scientist – Fraud Detection Contract: 6 months Engagement: Inside IR35 Location: Fully remote, with occasional travel to the London office. The Role We're looking for an experienced Applied Data Scientist to help improve the performance of our production fraud detection models. Working within … Data Scientist or Applied Data Scientist. Strong Python and Pandas. Advanced SQL. Experience working with Azure Data Lake or similar cloud data platforms. Experience working with very large datasets. Strong feature engineering experience. Experience improving production classification models. Excellent understanding ...

London-based consultancy delivering high-impact services in project controls and digital transformation. We specialise in simplifying complexities, driving efficiency and providing data-driven actionable insights to facilitate better decision-making and project outcomes. With extensive experience and expertise, complemented by the capabilities of our associate partners … through reports, dashboards, graphs, and visualisations. Work with IT and Digital teams to improve data availability and accessibility through Data Lake environments. Engage in and support the development of Power BI dashboards and reporting solutions that support project performance monitoring. Own the quality, consistency ...

Group is a tech-enabled professional services business with two service engines and a product arm that incubates and owns its software and data assets. Services, software, and data compounding together. Wave Talent. UK recruitment and staffing. Permanent and contract technology and digital professionals … Wave Talent's proprietary candidate universe. A quality-controlled alternative to pure AI hiring tools. The ARR engine. Mothership. Internal data lake with LLM copilot capability. Powers Iron Man Suits, candidate matching, financial reporting, and group intelligence. The infrastructure that compounds every year the group exists. Wave ...

Role: GCP Data Engineer Location: Hybrid; 2 days per week onsite in either Bristol, London, or Manchester Salary: £65,000 to £85,000 (Dependent on Experience) + Benefits Security Clearance: Due to the sensitive nature of the client's programme, candidates must hold active SC Clearance … process. Please note that specific client security restrictions mean this role is strictly restricted to Sole British Nationals. We are seeking experienced GCP Data Engineers at both Senior and Principal levels to help lead, design, and deliver high-impact cloud data platforms for our clients. Whether ...