1 to 25 of 1,259 Data Lake Jobs in the UK

significant decisions with confidence, provide strong foundations for the future and guide them towards their goals. Job Description Job Purpose Evelyn Partners is recruiting a skilled and experienced Data Engineer to join a growing team to deliver a large data modernisation programme. As a Data Services Data Engineer, you will be responsible … for designing, developing, and maintaining our MS SQL Server Data Warehouses and associated data feeds into and out of the warehouses, and developing on our new modern cloud data platform, requiring Snowflake, dbt and Azure Data Factory experience. Our data platform's support regulatory requirements, business intelligence & reporting needs and … GitHub. • Extensive experience in designing, building, deploying, and supporting cloud-based data products and pipelines. • Experience with Azure Storage services such as Blob Storage, Data Lake Storage, or Azure Files. • Hands-on experience designing and developing data pipelines using Azure Data Factory. • Experience of data warehouse design and implementation More ❯

provides its customers with clear advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services and infrastructure globally to all … design. The IT Group runs our enterprise-wide services to end users and actively manages the firm's infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data … a regulated environment and knowledge of the risk and compliance requirements associated with this. • Oracle Database • MongoDB • Cloud Data Technologies (Mainly Azure - SQL Database, Data Lake, HD Insight, Data Factory etc.) • A collaborative team player, approachable, self-efficient and influences a positive work environment • Demonstrates curiosity • Resilient in a challenging, fast-paced environment More ❯

Senior Java Software Engineer AWS Python Data Finance London Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager … Trading, Front Office finance Spark Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front … used across the Front Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python and AWS. If you have any experience in Iceberg, Dremio More ❯

About the role North Northamptonshire Council are working to modernise their data management and visualisation processes. We are working to deliver our data analytics strategy. This role will support the design and delivery of the data warehousing, data infrastructure and (where required) the migration of existing data sources (data … role A deep understanding of data engineering principles - ideally in an Azure environment Proven experience as a Data Engineer/Specialist in data lake/warehouse and data migration projects Experience building and maintaining ETL processes Experience producing data models and understanding where to use different types of data … years + working with Azure data engineering tools e.g: Azure Data factory Azure Synapse Azure SQL Azure DataBricks Microsoft Fabric Azure data lake Exposure to other data engineering and storage tools: Snowflake AWS tools - Kinesis/Glue/Redshift Google tools - BigQuery/Looker Experience working with open datasets - ingesting data More ❯

About The Role North Northamptonshire Council are working to modernise their data management and visualisation processes. We are working to deliver our data analytics strategy. This role will support the design and delivery of the data warehousing, data infrastructure and (where required) the migration of existing data sources (data … role A deep understanding of data engineering principles – ideally in an Azure environment Proven experience as a Data Engineer/Specialist in data lake/warehouse and data migration projects Experience building and maintaining ETL processes Experience producing data models and understanding where to use different types of data … years + working with Azure data engineering tools e.g: Azure Data factory Azure Synapse Azure SQL Azure DataBricks Microsoft Fabric Azure data lake Exposure to other data engineering and storage tools: Snowflake AWS tools – Kinesis/Glue/Redshift Google tools – BigQuery/Looker Experience working with open datasets – ingesting data More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 15 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test and … warehousing, and real-time streaming. Knowledge of developing and processing full and incremental loads. Experience of automated loads using Databricks workflows and Jobs Expertise in Azure Databricks, including Delta Lake, Spark optimizations, and MLflow. Strong experience with Azure Data Factory (ADF) for data integration and orchestration. Hands-on experience with Azure DevOps, including pipelines, repos More ❯

in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation … Everstone Capital, Apexon now has a global presence of 15 offices (and 10 delivery centers) across four continents. About the Role: We are currently looking for a Senior Data Engineer to manage teams on client projects. Your Profile: Proven experience as a Data Engineer with a focus on Azure cloud services. Experience of managing small teams … applications using Azure, specifically a Data Warehouse and/or Data Lake. Technologies : Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and More ❯

Join to apply for the Senior Data Analytics Developer role at Coveo Join to apply for the Senior Data Analytics Developer role at Coveo Are you the expert who transforms data into strategic insights to drive our success? As a Senior Data Developer, you will be at the heart of developing our … data platform. Your main role? Designing fundamental data models, creating data warehouses to facilitate business analysis, and improving data monitoring and observability solutions. Using advanced technologies, you will build efficient and scalable data pipelines. What’s exciting about this role? Your efforts will have a direct impact on uncovering essential … with hands-on experience in Python, dbt, Snowflake, and/or Databricks. You have in-depth knowledge of SQL, ETL processes, data warehousing, and Data Lake architecture. You have experience with automated testing frameworks and a commitment to code quality, including unit testing, integration testing, and code coverage metrics. You excel at working collaboratively with More ❯

in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation … Everstone Capital, Apexon now has a global presence of 15 offices (and 10 delivery centers) across four continents. About the Role: We are currently looking for a Senior Data Engineer to manage teams on client projects. Your Profile: Proven experience as a Data Engineer with a focus on Azure cloud services. Experience of managing small teams … applications using Azure, specifically a Data Warehouse and/or Data Lake. Technologies : Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and More ❯

Social network you want to login/join with: Data Architect (Enterprise Data Foundations), Slough col-narrow-left Client: Mars Location: Slough, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Reference: 5770903661a6 Job Views: 4 Posted: 02.06.2025 Expiry Date: 17.07.2025 col-wide Job Description: JOB DESCRIPTION Job Description: Come become … a part of a vibrant CDO team serving Mars Snacking Enterprise. Our team helps build the Data Platforms, Model & Engineer data products, craft Analytical products and Integrations to be a truly data driven growth engine. As we continue to expand and transform digitally, we’re actively seeking an experienced and skilled Data … technologies across traditional and contemporary software, with focus as below:Expert understanding: Azure Data Factory, Databricks, Spark, Azure SQL Database, Azure DevOps/Git, Data Lake, Delta Lake/Lakehouse architecture, Power BI.Working Knowledge: Azure WebApp, Azure Networking concepts.Conceptual Knowhow (nice to have): Azure AI Services, ML concepts, Unity Catalog, Purview. Programming: SQL, Java More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

plan a crucial role in architecting a scalable, reliable and efficient data solutions to support our business objectives. Responsibilities Design, develop, and maintain the data lake house architecture on AWS using best practices and industry standards. Implement data labelling strategy and principals as required by Data Governance and Information Security. Design … and implement robust ETL/ELT processes to extract, transform and load data from various sources into the data lake ensuring data quality and integrity. Ensure data quality, reliability, availability, and scalability across the data lake house. Optimize data ingestion, processing, storage, and access for various … engineering. Proficiency in Python and SQL Proficient in AWS data services such as S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, etc. Strong knowledge of data lake concepts, architectures, and design patterns. Experience in building and managing data pipelines using tools such as Airflow, Spark, Kinesis etc. Experience in working with structured, semi-structured More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

We are seeking a strategic and hands-on Lead Data Engineer to own the development of Aerial Direct’s cloud-native data platform. Working primarily within the Microsoft Azure ecosystem, you will be responsible for building and maintaining scalable data pipelines, optimising architecture for performance, and enabling actionable insights across the business. In addition … and evolving Aerial Direct’s Azure-based data platform, ensuring performance, security, and scalability. Design, management, and optimisation of data warehouse and data lake schemas to support downstream analytics, dashboards, and operational processes. Leading development and optimisation of ETL/ELT pipelines using Azure Data Factory, including data ingress … technical team leadership. Extensive hands-on experience with the Microsoft Azure data stack, including Azure Data Factory, Synapse Analytics, Azure SQL DB, Data Lake, Azure Storage, Key Vault, and Databricks. Demonstrated ability to optimise and extend data warehouse and data lake schemas, collaborating with stakeholders to design scalable More ❯

Senior Data Engineering Consultant We are open to full-time or part-time applicants for this role! Are you returning from maternity leave? We'd love to hear from you! Salary: £70,000 to £90,000 (depending on experience). Location: Remote – must be based within the UK. Occasional visits to see customers and the team. Interview process … Team Member. Our Values: Advancing, Come as you are, Zero burnout, Greta goal, We all level up together, and Transparency. Learn more about us: YouTube , Totally Skewed Podcast , Data Science in Production . Read our Blogs . Who Are We? Advancing Analytics is a market leader in delivering cutting-edge analytical solutions globally. We help businesses implement AI … concepts Apache Spark – Databricks, Microsoft Fabric, or other Spark engines Python SQL – complex high-performance queries Azure Data Factory or other orchestration tools Azure Data Lake storage and Delta Lake Unity Catalog, Purview, or other Data Governance tools IAC tools like Terraform, Azure DevOps Preferred but not essential: Visualization Tools – Power BI More ❯

Are you an experienced Data Engineer with in depth knowledge of both on-premises SQL data warehouses and modern Lakehouse architectures hosted on Azure? Do you enjoy bringing ideas to the table and sharing knowledge with stakeholders? If so, the Data Engineer role here at Harding+ could be perfect for you. About the role … smooth integration between the on-prem and future cloud environments. Cloud Infrastructure Design: Design and implement Azure-based data architecture, leveraging tools like Azure Data Lake, Azure Synapse Analytics, and Fabric’s data capabilities to future-proof the platform Azure Expertise: Utilize Azure tools such as Synapse Analytics, Logic Apps, and Azure Functions … applying data architecture principles to create robust, scalable solutions Azure Cloud Proficiency: Deep understanding of Azure cloud tools such as Azure Synapse Analytics, Azure Data Lake, Azure Data Factory, and Logic Apps Microsoft Fabric: Hands-on experience with Microsoft Fabric or similar modern cloud platforms, with a strong focus on data More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £50,000 - £77,500 - depending on experience Department: Technology and Data We’re First Central Insurance & Technology Group (First Central for short), an innovative, market-leading insurance company. We protect the things customers love so they can get on with what matters to them in life. Data drives us. It fuels our outstanding distribution, finance, technology and legal services. Our underwriting skills are built on data expertise; it creates the insights we need to give the right cover to the right customers at the right price. But, it’s the people inside and outside our business that power us. They make us stand … Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies. You’ll build solutions that pipe transform data into data lake storage areas, physical database models and reporting structures across data lake, data warehouse, business intelligence systems and analytics applications. You’ll build physical data More ❯

Senior Support Data Engineer Contract: Full-time, Permanent Salary: Up to £60,000 + Benefits + Bonus Location: Remote (2 days per month in Nottingham) About the Role My client is on the lookout for a talented Senior Data Engineer (Support) to join their growing team. You’ll be working closely with clients to provide top … notch support after deployment, making sure their data platforms run smoothly and efficiently. Plus, you’ll help improve how the team works and find smarter ways to do things. What You’ll Be Doing Taking full ownership of client support post-deployment — keeping systems performing well and clients happy. Leading technical investigations to solve tricky data … to improve support frameworks and the way the team operates. What You’ll Need Technical Skills: Hands-on experience with Azure Data Platforms — think Data Lake, SQL DB/Warehouse, and Synapse. Good knowledge of BI tools like Power BI, Tableau, QlikView, or Google Data Studio. A solid understanding of data More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We’re technology agnostic, and we’re open minded when it More ❯

class opportunity to help shape our organisation for the next stage of its journey. To drive towards this ambition, we are seeking a motivated individual to join our Data Platform team and support Aztec's new technology strategy using Azure Databricks. You will lead our Data Engineering capability and collaborate with others passionate about solving business … problems. Key responsibilities: Data Platform Design and Architecture Design, develop, and maintain a high-performing, secure, and scalable data platform, leveraging Databricks Corporate Lakehouse and Medallion Architectures. Utilise our metadata-driven data platform framework combined with advanced cluster management techniques to create and optimise scalable, robust, and efficient data solutions. Implement comprehensive … Proven experience working with Azure data platform services, including Storage, ADLS Gen2, Azure Functions, Kubernetes. Background in cloud platforms and data architectures, such as Corporate DataLake, Medallion Architecture, Metadata Driven Platform, Event-driven architecture. Proven experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We're technology agnostic, and we're open minded when it More ❯

class opportunity to help shape our organisation for the next stage of its journey. To drive towards this ambition, we are seeking a motivated individual to join our Data Platform team and support Aztec’s new technology strategy using Azure Databricks. You will lead our Data Engineering capability and collaborate with others passionate about solving business … problems. Key responsibilities: Data Platform Design and Architecture Design, develop, and maintain a high-performing, secure, and scalable data platform, leveraging Databricks Corporate Lakehouse and Medallion Architectures. Utilise our metadata-driven data platform framework combined with advanced cluster management techniques to create and optimise scalable, robust, and efficient data solutions. Implement comprehensive … Proven experience working with Azure data platform services, including Storage, ADLS Gen2, Azure Functions, Kubernetes. Background in cloud platforms and data architectures, such as Corporate DataLake, Medallion Architecture, Metadata Driven Platform, Event-driven architecture. Proven experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python More ❯

that we put them first. Additional Info Hybrid Working: 3 days onsite, 2 from home Location: City of London Contract Type: Permanent Job Summary Requirement is for a Data & Analytics Engineer to work within a small Data & Analytics IT team (team size: 3-4) with strong data & analytical technology skills preferably within the financial … & MI systems built using SQL Analysis Services & SQL Reporting Services. Technology stack will include Informatica Intelligent Cloud Service, Azure Data Analytics Tools (Azure Data Lake Storage, Databricks, Synapse, Factory, PowerBI), Python & SQL databases. A strong understanding of ETL tools and Azure Data Analytics is ESSENTIAL and strong PowerBI development and administration skills … are expected. The SME will have background and experience in implementing ETL data pipelines, PowerBI platforms, and proven experience in analysing complex data sets to identify trends, patterns, and insights that will inform strategic decisions. Key Work Outputs and Accountabilities Design, develop, and maintain scalable ETL data pipelines to support data integration More ❯