25 of 25 Contract Data Lake Jobs in England

pipelines and data integration workflows using Microsoft Azure services such as Data Factory, Synapse Analytics, Data Lake, Databricks, and related technologies. Implement and support Microsoft Fabric solutions, including dataflows, data warehouses, and real-time analytics features. Develop and maintain … EXPECTATIONS IN THE ROLE Strong hands-on experience with Microsoft Azure data services (Data Factory, Synapse Analytics, Data Lake, Databricks). Direct experience delivering solutions using Microsoft Fabric. Advanced SQL skills and experience with data modelling, transformation, and integration. Familiarity with ...

workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services including Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Event Hub. * Implement data quality validation, monitoring, metadata management, and governance … Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data ...

analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools … Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse, Azure Data Factory, and Databricks workflows Implement best practices … Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … warehousing concepts and dimensional modeling Proficiency in SQL and Python Experience with Azure data services such as Azure Data Lake Storage (ADLS Gen2) , Azure Data Factory , and Azure SQL Familiarity with CI/CD pipelines and version control (Git, Azure DevOps) Experience ...

Senior Data Engineer Rate - up to £450/day Outside IR35 Location - 2 days a week onsite in Central London Duration - 6 Months + Likely Extension We are working with a well-established consultancy supporting a high-impact data transformation programme for an end client based … Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2), SQL-based systems and APIs. Collaborate with Data Architects, Analysts, and stakeholders to define data models and ensure best ...

Skills Proven experience with Databricks (including Spark-based data processing). Strong knowledge of Azure Data Platform (Data Lake, Synapse, etc.). Proficiency in Python and SQL for data engineering tasks. Understanding of data architecture and ETL processes. Ability … work independently in a remote environment. Nice-to-Have Experience with CI/CD pipelines for data solutions. Familiarity with Delta Lake and ML pipelines . Start Date: ASAP Contract Type: Outside IR35 Apply Now: If you're a skilled Data Engineer looking ...

Days Onsite per week) Job Description Looking for L3 Data Engineer | Support | Key Skills & Expertise AWS Core Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/… Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort ...

Permanent Job Summary: AWS Support Engineer/Data Engineer Telecom Domain (JD) Key Skills & Expertise AWS Core Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/… Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort ...

Details Role/Job Title: MLOps Tech Lead Work Location: London, UK Office Requirement (Hybrid): 2 days per week Key Responsibilities (High-Level) Data Pipeline Development: Lead the technical direction of projects and ensure the use of Sainsbury's best practices to the best quality. Data Integration: Lead and provide expertise on Integrate data from various sources, ensuring data consistency, integrity, and quality across the entire data lifecycle. Infrastructure Management: Provide guidance for the junior & Mid Data Engineers on the best practices when building and managing ...

pipelines using Databricks on Azure. Build and optimize data solutions leveraging Azure Data Services (e.g., Data Lake, Synapse). Work with large datasets to enable advanced analytics and reporting. Collaborate with stakeholders to deliver high-quality data solutions aligned … standards. Essential Skills Proven experience in Data Engineering within complex environments. Strong hands-on expertise with Databricks and Azure (Data Lake, Synapse, etc.). Solid understanding of cloud-based data platforms and ETL processes. Active SC Clearance (must be in place before starting ...

Details Role/Job Title: Data Architect Work Location: Dublin, Ireland Role Type: Contracting Mode of Working: Hybrid/Office based The Role The Data Architect will lead the assessment and design of customer's data landscape to enable seamless integration with … Skills/Knowledge/Experience Technical Expertise: Strong knowledge of SAP S/4HANA, SAP Analytics Cloud, SAP Datasphere, and large data lake (e.g., Snowflake MDH). Experience with ETL tools, APIs, and data integration frameworks. Proficiency in data modeling and architecture design. ...

Your new company Working for a renowned commodity, metals, trades and exchange group. You'll be a key part of the Enterprise Data team helping to replace legacy ETL tools (Informatica) and deliver modern data engineering capabilities. Your work will include managing data pipelines … visualisation, and collaborating with ETL developers and wider technology teams to deliver solutions aligned with our strategic roadmap. You'll work across backend, data, and infrastructure engineering, contributing to solution design, implementation, deployment, testing, and support. This is a hands-on role for someone with strong data ...

Technical Business Analyst (BA) and Architect to drive enterprise-scale digital transformation initiatives. This role combines business analysis, solution architecture, cloud architecture, and data architecture, acting as a bridge between business stakeholders and technology teams. Key Responsibilities: Business Analysis (BA): - Engage with stakeholders to gather and document business … initiatives Cloud & Technology Leadership: - AWS: EC2, S3, RDS, Lambda, EKS, Elastic Beanstalk, IAM, Neptune - Azure: Data Factory, Databricks, Synapse, Data Lake, Event Hub, AKS, Power BI - Microservices and APIs using Java/Spring Boot - CI/CD using GitLab, Azure DevOps, Kubernetes, Helm, Terraform - Ensure ...

Senior Data Engineer Summer-Browning Associates is currently supporting our central government client, who is seeking a Senior Data Engineer for an initial three-month assignment with the option to extend. Location: Hybrid working - London, Bristol, Manchester, Birmingham, Newcastle, Exeter, Cambridge, Southampton The ideal candidate will … hold an active security clearance and strong experience in Data Engineering, with the following skills and qualifications: Proficiency in the Microsoft BI Stack, including Azure SQL, Fabric, Synapse Analytics, and Power BI. Experience in developing data products such as data warehouses, data ...

Azure Data Engineer (Databricks) - Must be SC Cleared (Outside IR35) Contract: 6 months Location: Newcastle Upon Tyne (2-3 days onsite per week) Expenses: Not reimbursable Domain: Public Sector Clearance: SC Clearance is mandatory Role Overview: We are seeking an experienced Azure Data Engineer with strong … experience as a Data Engineer in Azure environments. Strong hands-on expertise with Databricks and Spark. Knowledge of Azure Data Lake, Azure Synapse, and related services. Experience in Public Sector projects. SC Clearance (must be active). Must be SC Cleared Candidates ideally based ...

Role: EU Data Manager Location: Paddington, London (Hybrid/Remote with occasional office visits) Start Date: ASAP Duration: 12-Month Contract Rate: £414.44 - £550 per day (Inside IR35) Are you a data-driven professional with a passion for the FMCG sector? We are looking … shaping the future data strategy for the region. Key Responsibilities Reporting Ownership: Act as the primary lead for the Data Lake and the SPRINT external reporting suite (managed by Red Slim). Data Strategy: Plot and map data suppliers (NielsenIQ, ePOS ...

mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using TensorFlow, PyTorch … Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

Data Platform Engineer - Terraform, Azure Services, ADF, Synapse £Market Rate (Inside IR35) London/Hybrid 6 months My client is an instantly recognisable firm who urgently require a Data Platform Engineer with core strengths in Building and Operating Data Platforms, and hands on expertise … Factory (ADF), including network configurations and integration runtime options Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts Ability to showcase DevOps and CI/CD capabilities Working knowledge of designing and deploying infrastructure Experience in efficient handling of large ...

Role/Job Title: Data Engineer (Alteryx and DataIKU) Work Location: Redhill Mode of Working Hybrid/Office Based-Hybrid (4 days) Any other working conditions - travel/on call/shifts-[London/Hove/Cardiff based on customer ask- TBC] for workshops/PI planning workshops … demand The Role As a Data Engineer specializing in Alteryx and Dataiku, you will design, develop, and maintain ETL pipelines and data workflows using these platforms. The role requires expertise in SQL, Python, cloud technologies (Azure/AWS), and collaboration within Agile teams. You are expected ...

performance tuning: optimise cluster sizing, caching, partitioning, and shuffle management. You will monitor Spark job metrics, analyse logs, and identify bottlenecks in data throughput or latency. You will also be able to implement cost-optimisation strategies for Databricks jobs and clusters using autoscaling and job consolidation. You must … essential. Hands-on experience with Azure, AWS, or multi-cloud Databricks deployments. Knowledge of data storage layers (Azure Data Lake Storage, AWS S3) and performance trade-offs. Experience of version control (Git, GitHub Actions, DevOps pipelines) and CI/CD practices for Databricks. Please apply ...

Data Architect with current SC Clearance, we have a new contract role we would like to discuss with you. Please note this role is inside IR35 and requires one day per week onsite. As a Data Architect you will … responsible for the following: Design & build high performing data solutions utilising Databricks Design & oversee delivery of secure data warehouses and datalakes Design & implement AI/ML Models Ensure data governance and security is managed Work with existing team around MS Cloud platforms including Azure ...

Implementation & Rollout HubSpot rollout across five business areas Replacement of two existing CRMs Integration with Microsoft Dynamics 365 for sales teams Two-way data integration across platforms E-Commerce Replatform Delivery of a new e-commerce platform within critical compliance timelines Oversight of Magento (or alternative platform) replatforming … party systems Key Responsibilities Lead end-to-end delivery of multiple concurrent digital and IT projects Manage cross-platform integrations across CRM, web, data, billing and e-commerce systems Own project governance including planning, RAID logs, change control, reporting and benefits tracking Coordinate stakeholders across IT, Marketing, Sales ...

Manager (EPM) - 1 YEAR CONTRACT - HYBRID WORKING - NO SPONSORSHIP AVAILABLE Job Description We are seeking an experienced Enterprise Project Manager to focus on data capabilities within an innovative organisation. The ideal candidate will be working on enhancing data capabilities, ensuring effective project management in alignment with … strategic objectives. Responsibilities Lead and manage projects related to data capabilities, ensuring alignment with organisational goals. Collaborate with cross-functional teams to implement data solutions using AWS and Data Lakes. Coordinate the integration of message bus systems into existing data infrastructures. Oversee ...

build analysis - Provide architectural input to support Product's evaluation of vendor offerings and internal options. -Defining preferred target architecture -Document current integrations, data flows, and system dependencies - Previous experience with Adtech Integration's within broadcast or non linear advertising solutions including how existing systems connect to client … systems, including ad servers, Freewheel, ATS, DAM/MAM, data platforms, and downstream delivery system -Data lakes -Digital advertising -Addressable and targeted advertising -Advertising market places -Airtime Sales/adsales traffic systems, e.g. Imagine Landmark -Freewheel -Programmatic advertising DSP, e.g. Tradedesk -VOD players -TV Beat -Tech ...