151 to 175 of 470 Data Lake Jobs in the UK

migration of existing data processes to the new Data Analytics Cloud Platform, building knowledge of Data Lake, Databricks, Unity Catalog and related cloud services. Develop, maintain and optimise Databricks notebooks, SQL processes, data pipelines and transformation logic, ensuring solutions are scalable … experience transforming, modelling and optimising large and sophisticated datasets. Experience working with modern cloud data platforms such as Databricks, Data Lake, and related Azure services. Python experience is beneficial. Experience of automating processes, improving efficiency and implementing scalable analytics solutions. Experience owning and supporting enterprise ...

Role You'll own the data lake — the system of record for millions of financial events a day across million of active users. You decide how that data lands, is stored, retained, and governed on S3 + Snowflake/Databricks, so analytics, finance, and regulators … compute spend. What we're looking for 3+ years in data engineering, with real ownership of a large-scale data lake or lakehouse. Cloud object storage (S3/GCS) plus Snowflake, Databricks, or BigQuery in production. Hands-on with an open table format (Iceberg, Delta ...

Platform. From an MSB perspective the role will require the design, development, support and deployment of the AeroConnect Platform to provide connectivity and data analytic solutions for the manufacturing environments. This will focus on the integration of the latest manufacturing technologies both at the production sites and research … analysis. • Implement and manage data lakes, data warehouses, and other data storage solutions, including SQL and Cloud Lake solutions. Data Security & Compliance • Implement and enforce best architectural practices for data security, ensuring compliance with relevant data regulations ...

exciting opportunity has arisen for a Senior Data Engineer to join a growing data team and contribute to the development of a modern cloud-based data platform. You'll be responsible for developing scalable data pipelines, integrating multiple data sources … processes Strong communication and stakeholder engagement skills Desirable Skills Azure Data Factory Databricks Synapse Analytics Microsoft Fabric Azure SQL Data Lake technologies Experience within regulated sectors Cloud-based data platform experience ...

governance, andoperational performance in a high-volume data environment. This role involvesstrategic and technical leadership in building Azure-based data lake solutions,driving maturity in data management, and engaging with cross-functional teamsto leverage data for organizational success. Key Responsibilities Architect … discipline. 15-17 years of experience in the ITindustry with expertise in delivering cost-effective cloud-based dataanalytics solutions. Proven track record in building datalake architectures and optimizing performance in big data environments. Certification in Azure DataEngineering or Databricks preferred. Strong knowledge of Azuretechnologies, including ADF, ADLS, Databricks ...

Data Factory and Azure Databricks . Build data models and analytics solutions using Synapse Analytics and Azure Data Lake Storage . Develop and optimise ETL/ELT processes. Work directly with clients to understand requirements and deliver robust data solutions. Troubleshoot … with Microsoft Azure Data Engineering . Hands-on expertise with Azure Data Factory, Databricks, Synapse Analytics and Data Lake Storage . Development experience in Python, Scala or SQL . Knowledge of data modelling and analytics solutions. Excellent communication skills and experience ...

Data Factory and Azure Databricks . Build data models and analytics solutions using Synapse Analytics and Azure Data Lake Storage . Develop and optimise ETL/ELT processes. Work directly with clients to understand requirements and deliver robust data solutions. Troubleshoot … with Microsoft Azure Data Engineering . Hands-on expertise with Azure Data Factory, Databricks, Synapse Analytics and Data Lake Storage . Development experience in Python, Scala or SQL . Knowledge of data modelling and analytics solutions. Excellent communication skills and experience ...

future of AI, and you are the future of Salesforce. We are seeking a Director, Solution Consulting specialising in Marketing Cloud and Data 360. You will be a recognised expert and thought leader in customer engagement and data architecture. You will act as a strategic advisor … handling. Integration architecture: Marketing Cloud Connect, REST/SOAP APIs, API-triggered and event-driven messaging, and ETL/data lake patterns. Data modelling, MDM, SQL, and DevOps/deployment lifecycle. Email deliverability: sender reputation, IP warm-up, and SPF/DKIM/ ...

want to know about the requirements for this role, read on for all the relevant information. Design scalable, secure, and efficient data architectures Develop data platforms, data lakes/lakehouses, and ETL/ELT solutions Translate business requirements into technical solutions Create data roadmaps aligned to business objectives and regulatory requirements Collaborate with stakeholders across business and technology teams Essential Experience Proven Data Architect experience, ideally within Financial Services or a regulated industry Strong knowledge of data architecture, data modelling, and data management ...

Data Factory and Azure Databricks . Build data models and analytics solutions using Synapse Analytics and Azure Data Lake Storage . Develop and optimise ETL/ELT processes. Work directly with clients to understand requirements and deliver robust data solutions. Troubleshoot … with Microsoft Azure Data Engineering . Hands-on expertise with Azure Data Factory, Databricks, Synapse Analytics and Data Lake Storage . Development experience in Python, Scala or SQL . Knowledge of data modelling and analytics solutions. Excellent communication skills and experience ...

Senior Data Engineer Location : Bolton Salary : Circa £55,000 depending on experience Working Pattern: Hybrid working 2 days per week onsite An exciting opportunity has become available for a Senior Data Engineer to join a leading defence engineering organisation, helping to develop and deliver advanced data solutions across a wide range of business and engineering functions. This role offers the opportunity to transform complex data from engineering systems, simulations, manufacturing operations and business platforms into reliable, accessible and valuable insights. You'll play a key role in designing scalable data ...

Role Description The Technical Analyst – Data Lineage will support a Data Governance, Controls, and Reporting program for a top-tier banking client. Responsible for establishing and validating end-to-end lineage across critical datasets used in operational and regulatory reporting. Translate governance and reporting requirements into … actionable lineage deliverables (source-to-target mapping, lineage diagrams, metadata standards, and audit evidence). Work with Data Platform, Data Engineering, Architecture, Risk, Compliance, and Security teams to define lineage standards, metadata capture, and control points. Maintain lineage artifacts for Critical Data Elements (CDEs ...

Senior Data Engineer Who we are We are an advanced data and business consultancy headquartered in Edinburgh, UK. We are a practitioner-led organisation that collaborates with top consumer brands to drive transformation and foster customer-centricity through our expertise in customer strategy, innovative design … advanced data science and engineering. We work with leading consumer brands to tackle and overcome complex business and customer problems to drive transformation and champion customer-centric agendas. We are proud to include some of the leading UK and global brands among our current clients such as Lloyds ...

Azure Data Engineer | £450 | Outside IR35 | 6 Months Initial | Fully Remote I'm working with a client looking for a Senior Data Engineer to help build and enhance a modern Azure-based data platform. We're keen to speak with engineers who have experience … pipelines within Azure Strong Python development skills for data engineering Azure Data Factory, Azure SQL and Data Lake technologies Data architecture and layered data platform design Metadata-led ingestion and reusable ETL frameworks CI/CD, source control ...

Macquarie, our Data Services Team drives innovation by delivering a diverse suite of data products that empower data-driven decision-making and enhance operational efficiency. With a strong focus on engineering, data science, and analytics, the team delivers innovative solutions that empower … scale data platform powered by modern cloud technologies. In this role, you will manage and enhance a petabyte-scale Data Lake and will create secure, efficient, and scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such ...

change and thrive in it. Loyalty – We stand by each other and our mission. Job Summary About the Role Movar is seeking a Data Engineer with hands‐on experience in building cloud data systems. You will design and implement data pipelines that support analytics …/ELT workflows to ensure data quality and reliability. Work with data lakes and warehouses using Azure Data Lake Storage and SQL Server. Collaborate with analysts and data scientists to deliver robust data products. Implement data quality ...

increasing automation, and mitigating security risk in a dynamic, global Agile environment alongside innovative colleagues. Be part of an exciting role evolution from data scientist and machine learning engineer to AI engineer. Your Impact Data Architecture: Develop and implement data architecture to store, process … regional requirements such as GDPR and country-specific legislation such as workers councils. Technical Guidance: Guide technical peers on database and data lake sources, as well as security requirements such as RBAC/ABAC, row- and column-level security, masking/tokenization, key management, PII classification Insights ...

years. Whatever your priorities – work/life balance, career progression, sustainability, volunteering – you’ll find like-minded people driving change at Howden. Senior Data Engineer Position Senior Data Engineer to own and drive forward high-impact data initiatives, providing technical depth across the data … using Databricks and Azure services (such as Autoloader, Spark Declarative Pipelines, Azure Data Factory (ADF), Azure Functions, and Azure Data Lake Storage (ADLS)). Apply and help embed best practices for data engineering, CI/CD pipelines, and DevOps for data. Collaborate with ...

Whatever your priorities – work/life balance, career progression, sustainability, volunteering – you’ll find like‐minded people driving change at Howden. Position Lead Data Engineer Position Senior Data Engineer to lead and drive forward complex and high‐impact data initiatives. Summary of the Role … using Databricks and Azure services (such as Autoloader, Spark Declarative Pipelines Azure Data Factory (ADF), Azure Functions, and Azure Data Lake Storage (ADLS)).). Establish and enforce best practices for data engineering, CI/CD pipelines, and DevOps for data. Collaborate with ...

Manchester/PERMANENT/SALARY: COMPETITIVE Job Purpose Responsible for developing, maintaining and optimising MPM’s data infrastructure. Provide technical expertise to ensure efficient data handling, ensuring availability, scalability and security of the centralised data asset. Ensure that all ETL pipeline and scheduled jobs … Experience with source control and CI/CD (GitHub, Azure DevOps, DABs) Data architecting, solutions architecting and experience optimising pipelines and Delta Lake Databricks Data Engineer Professional certified #J-18808-Ljbffr ...

pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop … Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps ...

founding in 2013, Quantiphi has tackled some of the world’s most complex business challenges by combining deep industry expertise, disciplined cloud and data engineering practices, and cutting-edge applied AI research. Our work is rooted in delivering accelerated, quantifiable business value, not just technology for technology … cloud innovation.Your next big opportunity starts here!Work Location: London, UKExperience Level: 8+ yearsType: Full timeRole:We are looking for a Senior Data Analyst to support delivery of our Pricing & Exposure data initiatives. This role will focus on hands-on analysis of complex insurance datasets, translating ...

Data Idols, we are working with a large-scale, data-driven organisation investing heavily in its data platform, analytics and AI capabilities. They are now looking for a Data Solutions Architect to provide design and architecture guidance across a broad portfolio … data platforms Experience with Databricks or similar modern data platforms Good understanding of data warehouse, data lake or lakehouse architectures If you are looking for a new challenge and want to help shape data solutions across a large-scale ...

Specialist Reporting to: Principal Data Engineer Role Overview To develop the geospatial data foundation, utilising our data mesh and bringing together geospatial datasets, EO and internal data to create a scalable spatial data solution. The outputs will provide Group wide … sustainable use of land. A key focus will be moving geospatial processing from manual GIS workflows towards automated, maintainable and production-ready data pipelines that can support digital products, analytics and AI/ML applications. Key responsibilities Land Use Intelligence Develop and maintain Origin’s spatial data ...

experience working with SQL Server Strong experience working with Azure Data Factory and Azure DevOps Hands-on experience with data lake platforms, Azure Synapse, Databricks, or equivalent experience/skills Experience with tooling such as CI/CD pipelines and version control The ability … will be responsible for the design, build, and maintenance of Python-based ETL pipelines Accounting and leading the SQL development Data lake development within Azure Synapse, working to the organisation’s architecture standards Meeting with business stakeholders to define requirements and translate them into solution designs Keeping ...