1 to 25 of 32 Contract Data Lake Jobs in the UK

pipelines and data integration workflows using Microsoft Azure services such as Data Factory, Synapse Analytics, Data Lake, Databricks, and related technologies. Implement and support Microsoft Fabric solutions, including dataflows, data warehouses, and real-time analytics features. Develop and maintain … EXPECTATIONS IN THE ROLE Strong hands-on experience with Microsoft Azure data services (Data Factory, Synapse Analytics, Data Lake, Databricks). Direct experience delivering solutions using Microsoft Fabric. Advanced SQL skills and experience with data modelling, transformation, and integration. Familiarity with ...

workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services including Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Event Hub. * Implement data quality validation, monitoring, metadata management, and governance … Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data ...

analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools … Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse, Azure Data Factory, and Databricks workflows Implement best practices … Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … warehousing concepts and dimensional modeling Proficiency in SQL and Python Experience with Azure data services such as Azure Data Lake Storage (ADLS Gen2) , Azure Data Factory , and Azure SQL Familiarity with CI/CD pipelines and version control (Git, Azure DevOps) Experience ...

Senior Data Engineer Rate - up to £450/day Outside IR35 Location - 2 days a week onsite in Central London Duration - 6 Months + Likely Extension We are working with a well-established consultancy supporting a high-impact data transformation programme for an end client based … Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2), SQL-based systems and APIs. Collaborate with Data Architects, Analysts, and stakeholders to define data models and ensure best ...

Skills Proven experience with Databricks (including Spark-based data processing). Strong knowledge of Azure Data Platform (Data Lake, Synapse, etc.). Proficiency in Python and SQL for data engineering tasks. Understanding of data architecture and ETL processes. Ability … work independently in a remote environment. Nice-to-Have Experience with CI/CD pipelines for data solutions. Familiarity with Delta Lake and ML pipelines . Start Date: ASAP Contract Type: Outside IR35 Apply Now: If you're a skilled Data Engineer looking ...

Days Onsite per week) Job Description Looking for L3 Data Engineer | Support | Key Skills & Expertise AWS Core Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/… Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort ...

Permanent Job Summary: AWS Support Engineer/Data Engineer Telecom Domain (JD) Key Skills & Expertise AWS Core Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/… Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort ...

Details Role/Job Title: MLOps Tech Lead Work Location: London, UK Office Requirement (Hybrid): 2 days per week Key Responsibilities (High-Level) Data Pipeline Development: Lead the technical direction of projects and ensure the use of Sainsbury's best practices to the best quality. Data Integration: Lead and provide expertise on Integrate data from various sources, ensuring data consistency, integrity, and quality across the entire data lifecycle. Infrastructure Management: Provide guidance for the junior & Mid Data Engineers on the best practices when building and managing ...

Details Role/Job Title: Data Engineer Work Location: Manchester, UK Mode of Working: Hybrid Office Requirement (Hybrid): Tuesday/Wednesday/Thursday The Role Seeking a Data Engineer profile with deep technical expertise and thought leadership, provides advanced data solutions by using software … process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical skillset on a variety of internal and external data. Your Responsibilities Writes ETL (Extract ...

pipelines using Databricks on Azure. Build and optimize data solutions leveraging Azure Data Services (e.g., Data Lake, Synapse). Work with large datasets to enable advanced analytics and reporting. Collaborate with stakeholders to deliver high-quality data solutions aligned … standards. Essential Skills Proven experience in Data Engineering within complex environments. Strong hands-on expertise with Databricks and Azure (Data Lake, Synapse, etc.). Solid understanding of cloud-based data platforms and ETL processes. Active SC Clearance (must be in place before starting ...

Role Overview We are looking for a Finance-specialised Data Business Analyst to support FP&A transformation and finance data projects. This role combines financial reporting understanding with strong business analysis skills to deliver accurate, insightful data solutions across the Finance/… deliverables. Act as the liaison between finance teams, data engineers, and technology teams. Conduct GAP analysis between current state Data Lake & desired future state Coordinate UAT & ensure compliance with FP&A controls. Manage expectations and provide regular updates on progress and risks. Support creation ...

Details Role/Job Title: Data Architect Work Location: Dublin, Ireland Role Type: Contracting Mode of Working: Hybrid/Office based The Role The Data Architect will lead the assessment and design of customer's data landscape to enable seamless integration with … Skills/Knowledge/Experience Technical Expertise: Strong knowledge of SAP S/4HANA, SAP Analytics Cloud, SAP Datasphere, and large data lake (e.g., Snowflake MDH). Experience with ETL tools, APIs, and data integration frameworks. Proficiency in data modeling and architecture design. ...

Microsoft Fabric Data Engineer - Synapse Migration Project Location: Remote (UK) Rate: £500 - £600 per day (Outside IR35) Contract: 3 months initially, with strong potential to extend iO Associates are partnered with a rapidly scaling data consultancy on a project focused on migrating enterprise-scale data … architecture Strong hands-on experience with Python, SQL, and ETL pipeline development Familiarity with Azure Data Factory, Power BI, Data Lake technologies, and exposure to Microsoft Purview Experience working on data migration or modernisation projects A proactive, problem-solving mindset with the ability ...

Your new company Working for a renowned commodity, metals, trades and exchange group. You'll be a key part of the Enterprise Data team helping to replace legacy ETL tools (Informatica) and deliver modern data engineering capabilities. Your work will include managing data pipelines … visualisation, and collaborating with ETL developers and wider technology teams to deliver solutions aligned with our strategic roadmap. You'll work across backend, data, and infrastructure engineering, contributing to solution design, implementation, deployment, testing, and support. This is a hands-on role for someone with strong data ...

Technical Business Analyst (BA) and Architect to drive enterprise-scale digital transformation initiatives. This role combines business analysis, solution architecture, cloud architecture, and data architecture, acting as a bridge between business stakeholders and technology teams. Key Responsibilities: Business Analysis (BA): - Engage with stakeholders to gather and document business … initiatives Cloud & Technology Leadership: - AWS: EC2, S3, RDS, Lambda, EKS, Elastic Beanstalk, IAM, Neptune - Azure: Data Factory, Databricks, Synapse, Data Lake, Event Hub, AKS, Power BI - Microservices and APIs using Java/Spring Boot - CI/CD using GitLab, Azure DevOps, Kubernetes, Helm, Terraform - Ensure ...

Data Solutions Architect Experience: o 8+ years in data architecture, data engineering, or solution architecture roles. o Proven experience with relational and NoSQL databases, data lakes, and data warehouses. o Strong knowledge of cloud platforms (AWS) and their data services. Technical Skills: o Data modelling (conceptual, logical, physical). o ETL/ELT tools and data integration frameworks. o Proficiency in SQL and at least one programming language (Python, Java, etc.). o Familiarity with BI tools and analytics platforms. Soft Skills ...

Senior Data Engineer Summer-Browning Associates is currently supporting our central government client, who is seeking a Senior Data Engineer for an initial three-month assignment with the option to extend. Location: Hybrid working - London, Bristol, Manchester, Birmingham, Newcastle, Exeter, Cambridge, Southampton The ideal candidate will … hold an active security clearance and strong experience in Data Engineering, with the following skills and qualifications: Proficiency in the Microsoft BI Stack, including Azure SQL, Fabric, Synapse Analytics, and Power BI. Experience in developing data products such as data warehouses, data ...

Azure Data Engineer (Databricks) - Must be SC Cleared (Outside IR35) Contract: 6 months Location: Newcastle Upon Tyne (2-3 days onsite per week) Expenses: Not reimbursable Domain: Public Sector Clearance: SC Clearance is mandatory Role Overview: We are seeking an experienced Azure Data Engineer with strong … experience as a Data Engineer in Azure environments. Strong hands-on expertise with Databricks and Spark. Knowledge of Azure Data Lake, Azure Synapse, and related services. Experience in Public Sector projects. SC Clearance (must be active). Must be SC Cleared Candidates ideally based ...

Role: EU Data Manager Location: Paddington, London (Hybrid/Remote with occasional office visits) Start Date: ASAP Duration: 12-Month Contract Rate: £414.44 - £550 per day (Inside IR35) Are you a data-driven professional with a passion for the FMCG sector? We are looking … shaping the future data strategy for the region. Key Responsibilities Reporting Ownership: Act as the primary lead for the Data Lake and the SPRINT external reporting suite (managed by Red Slim). Data Strategy: Plot and map data suppliers (NielsenIQ, ePOS ...

office Contract duration: 6 months Location: Nottingham Minimum years of experience: 8 to 10 years Design and implement end-to-end data architecture for cloud-based analytics platforms. Conduct current state assessments and define future-state architectures for clients. Lead data modernization and digital transformation initiatives. … Develop data models, data lakes, and data pipelines for structured and unstructured data. Ensure data governance, security, and compliance across all data assets. Collaborate with cross-functional teams including data engineers, analysts, and business stakeholders. Evaluate and recommend ...

mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using TensorFlow, PyTorch … Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

Data Platform Engineer - Terraform, Azure Services, ADF, Synapse £Market Rate (Inside IR35) London/Hybrid 6 months My client is an instantly recognisable firm who urgently require a Data Platform Engineer with core strengths in Building and Operating Data Platforms, and hands on expertise … Factory (ADF), including network configurations and integration runtime options Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts Ability to showcase DevOps and CI/CD capabilities Working knowledge of designing and deploying infrastructure Experience in efficient handling of large ...

Clearance required Role Overview We are seeking an SC Cleared Python Data Engineer with strong hands-on experience in PySpark, Delta Lake, and Azure cloud services. The role focuses on designing and delivering scalable, well-tested data pipelines, with particular importance placed on the ability … both technical and non-technical stakeholders. Implement unit and BDD testing (Behave or similar), including effective mocking and dependency management. Design and optimise Delta Lake tables to support ACID transactions, schema evolution, and incremental processing. Build and manage Docker-based environments for development, testing, and deployment. Develop configuration-driven ...