176 to 200 of 298 Data Lake Jobs

Cleared Contract £400-£458pd (Inside IR35) SC Clearance is Essential Summary Were looking for a Python Data Engineer skilled in PySpark, Delta Lake, Azure services, containerized development, and Behave-based testing. Youll design and build scalable data pipelines and maintain high-quality, test-driven code … pipelines for data ingestion, processing, and validation. Write unit and BDD tests using Behave, including mocking and patching. Create and optimize Delta Lake tables for reliable, performant data storage. Use Docker to manage consistent development, testing, and deployment environments. Build configurable, parameter-driven code ...

engineering solutions. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with … Azure services: Data Lake Storage (Gen2), Key Vault, Azure Functions Familiarity with CI/CD for Databricks deployments Strong troubleshooting in distributed Data environments Excellent communication and collaboration skills Nice to have: Immediate availability Experience with enterprise-scale Databricks environments Knowledge of Azure Synapse, Power ...

engineering solutions. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with … Azure services: Data Lake Storage (Gen2), Key Vault, Azure Functions Familiarity with CI/CD for Databricks deployments Strong troubleshooting in distributed Data environments Excellent communication and collaboration skills Nice to have: Immediate availability Experience with enterprise-scale Databricks environments Knowledge of Azure Synapse, Power ...

Through innovative solutions and cutting edge technologies we provide comprehensive customised services globally. From BIM and Digital Project Control to Immersive Reality and Data Analytics our solutions push boundaries and set new standards. Join us as we shape the future of construction and create a world where innovation … Power Apps, Power Automate, Power Pages, and Microsoft 365 collaboration tools (Teams, SharePoint). Understanding of Azure services such as Data Lake, Synapse, Databricks, Azure Machine Learning; Fabric for end to end analytics desirable. Awareness of AI enabled tooling (e.g. Copilot/Gemini) is an advantage. Leadership ...

Role: BI and Data Engineer Location: Norwich Salary: £60,000 Flexible working: 1-2 days a week in the office My client, a specialist technology organisation, is looking for a BI and Data Engineer to join their growing team. This is an exciting opportunity to work … skills/Experience Own automation/testing for Power Platform roll-out. Power Platform (Power BI, Power Apps, Power Automate) Azure Data Lake, Synapse, Data Factory Comfortable with working data pipelines Experienced working with SSRS Financial services experience would be advantageous If this ...

Role: BI and Data Engineer Location: Norwich Salary: £60,000 Flexible working: 1-2 days a week in the office My client, a specialist technology organisation, is looking for a BI and Data Engineer to join their growing team. This is an exciting opportunity to work … skills/Experience Own automation/testing for Power Platform roll-out. Power Platform (Power BI, Power Apps, Power Automate) Azure Data Lake, Synapse, Data Factory Comfortable with working data pipelines Experienced working with SSRS Financial services experience would be advantageous If this ...

Title: Azure Databricks Data Engineer Primary skills: Advanced SQL, Azure Databricks, Azure Data Factory, Azure Datalake. Secondary skills: Azure SQL, PySpark, Azure Synapse. Experience: 10+ Years of Experience Hybrid - 3 Days/Week onsite is MUST About the job We are looking for an experienced Databricks … Data Engineer to design, develop, and manage data pipelines using Azure services such as Databricks, Data Factory, and Datalake. The role involves building scalable ETL solutions, collaborating with cross-functional teams, and processing large volumes of data. You will work closely with business ...

Title: Azure Databricks Data Engineer Primary skills: Advanced SQL, Azure Databricks, Azure Data Factory, Azure Datalake. Secondary skills: Azure SQL, PySpark, Azure Synapse. Experience: 10+ Years of Experience Hybrid - 3 Days/Week onsite is MUST About the job We are looking for an experienced Databricks … Data Engineer to design, develop, and manage data pipelines using Azure services such as Databricks, Data Factory, and Datalake. The role involves building scalable ETL solutions, collaborating with cross-functional teams, and processing large volumes of data. You will work closely with business ...

Title: Azure Databricks Data Engineer Primary skills: Advanced SQL, Azure Databricks, Azure Data Factory, Azure Datalake. Secondary skills: Azure SQL, PySpark, Azure Synapse. Experience: 10+ Years of Experience Hybrid - 3 Days/Week onsite is MUST About the job We are looking for an experienced Databricks … Data Engineer to design, develop, and manage data pipelines using Azure services such as Databricks, Data Factory, and Datalake. The role involves building scalable ETL solutions, collaborating with cross-functional teams, and processing large volumes of data. You will work closely with business ...

Python Data Engineer - Multi-Strategy Hedge Fund Location: London | Hybrid: 2 days per week on-site | Type: Full-time About the Role A leading multi-strategy hedge fund is seeking a highly skilled Python Data Engineer to join its technology and data team. This … scalable Python-based ETL pipelines for ingesting and transforming market data from multiple sources. Design and manage cloud-based data lake solutions (AWS, Databricks) for large volumes of structured and unstructured data. Implement rigorous data quality, validation, and cleansing routines to ensure accuracy ...

seeking a highly skilled Business Intelligence & Power BI Developer to design, develop, and maintain high-quality reporting and dashboard solutions that support data-driven decision making across the organisation. Working closely with senior finance stakeholders, you will transform complex data into meaningful insights and ensure robust … skills and experience with data cleansing and modelling Experience working with Azure/Microsoft BI technologies (SQL Server, Data Lake, SSIS/SSAS/SSRS) Familiarity with DevOps and Agile delivery methods (Azure DevOps, CI/CD, Scrum/Kanban) Experience with other BI tools ...

Senior Data Engineer Summer-Browning Associates is currently supporting our central government client, who is seeking a Senior Data Engineer for an initial three-month assignment with the option to extend. Location: Hybrid working - London, Bristol, Manchester, Birmingham, Newcastle, Exeter, Cambridge, Southampton The ideal candidate will … hold an active security clearance and strong experience in Data Engineering, with the following skills and qualifications: Proficiency in the Microsoft BI Stack, including Azure SQL, Fabric, Synapse Analytics, and Power BI. Experience in developing data products such as data warehouses, data ...

Azure Data Engineer (Databricks) - Must be SC Cleared (Outside IR35) Contract: 6 months Location: Newcastle Upon Tyne (2-3 days onsite per week) Expenses: Not reimbursable Domain: Public Sector Clearance: SC Clearance is mandatory Role Overview: We are seeking an experienced Azure Data Engineer with strong … experience as a Data Engineer in Azure environments. Strong hands-on expertise with Databricks and Spark. Knowledge of Azure Data Lake, Azure Synapse, and related services. Experience in Public Sector projects. SC Clearance (must be active). Must be SC Cleared Candidates ideally based ...

skilled Data Engineer with deep Snowflake expertise and a passion for building scalable cloud based solutions? A fast growing technology consultancy is looking for a Senior Data Engineer to lead on projects, drive engineering excellence, and help shape innovative data solutions across cloud platforms … skills in Python and SQL for ELT/ETL and transformation workflows. Experience with cloud native storage and compute tools: S3, Data Lake, ADF, Redshift, Synapse, etc. Familiarity with CI/CD pipelines, test automation, and DevOps practices in data workflows. Stakeholder management Career & Development ...

skilled Data Engineer with deep Snowflake expertise and a passion for building scalable cloud based solutions? A fast growing technology consultancy is looking for a Senior Data Engineer to lead on projects, drive engineering excellence, and help shape innovative data solutions across cloud platforms … skills in Python and SQL for ELT/ETL and transformation workflows. Experience with cloud native storage and compute tools: S3, Data Lake, ADF, Redshift, Synapse, etc. Familiarity with CI/CD pipelines, test automation, and DevOps practices in data workflows. Stakeholder management Career & Development ...

reinsurer operating globally, with a focus on complex commercial and industrial risks. The business combines deep sector expertise with advanced use of data, analytics, and modern technology to deliver informed, efficient, and fair underwriting decisions. The company is in a period of significant growth and is committed … C# ecosystem and modern front-end frameworks (ideally Angular). Experience with data architecture concepts including relational, NoSQL, and data lake technologies. Knowledge of cloud-based AI architectures, including LLMs, Generative AI, and Agentic AI. Experience integrating proprietary systems with COTS finance or ERP platforms ...

mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using TensorFlow, PyTorch … Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

Azure range of products, which enables us to deliver powerful, connected business solutions that drive significant client value. The Role: Pre-Sales Data Solutions Architect – Microsoft Business Solutions (MBS) The Pre-Sales Data Solutions Architect will define and articulate data and AI strategies within … Microsoft AI-powered business solutions. This strategic role combines deep technical expertise in Microsoft’s data stack with the ability to translate business requirements into end-to-end Microsoft architectures, operating at the intersection of Microsoft Data, AI, and Application technologies. Responsibilities: Engaging with clients ...

Azure range of products, which enables us to deliver powerful, connected business solutions that drive significant client value. The Role: Pre-Sales Data Solutions Architect – Microsoft Business Solutions (MBS) The Pre-Sales Data Solutions Architect will define and articulate data and AI strategies within … Microsoft AI-powered business solutions. This strategic role combines deep technical expertise in Microsoft’s data stack with the ability to translate business requirements into end-to-end Microsoft architectures, operating at the intersection of Microsoft Data, AI, and Application technologies. Responsibilities: Engaging with clients ...

team champions continuous learning, clean engineering practices, and building scalable, impactful applications that shape the firm's future. If you're passionate about data, eager to solve complex problems, and want to see your insights drive real organisational change, this is the role for you. About the Role … Strong understanding of data modelling, star schema design, and ETL processes Experience with Microsoft Fabric, Synapse, Azure SQL or Data Lake Familiarity with Azure DevOps/GIT for source control Strong analytical mindset with excellent communication and stakeholder engagement skills Experience with systems such ...

Senior Data Scientist Employment Type: Permanent Location: [Specify location or remote flexibility] Role Overview: We are seeking a highly skilled Senior Data Scientist with expertise in multi-cloud environments and a strong background in consulting . The ideal candidate will lead advanced analytics projects, design scalable … data solutions, and provide strategic insights to clients across diverse industries. This role requires a blend of technical proficiency, business acumen, and excellent communication skills. Key Responsibilities: Design and implement data science models (predictive, prescriptive, and descriptive analytics) to solve complex business problems. Work across multi ...

Senior Data Scientist Employment Type: Permanent Location: [Specify location or remote flexibility] Role Overview: We are seeking a highly skilled Senior Data Scientist with expertise in multi-cloud environments and a strong background in consulting . The ideal candidate will lead advanced analytics projects, design scalable … data solutions, and provide strategic insights to clients across diverse industries. This role requires a blend of technical proficiency, business acumen, and excellent communication skills. Key Responsibilities: Design and implement data science models (predictive, prescriptive, and descriptive analytics) to solve complex business problems. Work across multi ...

role We have an exciting opportunity for a Senior DataOps Engineer to join the Group and take responsibility for inspiring and coaching our Data Engineering teams. The person in post will be working alongside domain-oriented, multi-disciplinary data product teams to design, develop, and test … high-quality data solutions that serve as the backbone of our decision-making and digital services. Driving automation and CI/CD practices across data pipelines and infrastructure, the Senior DataOps Engineer will build robust, scalable, and secure infrastructure that supports our data platform ...

with Microsoft technologies like Azure, .NET, Power Platform, and Microsoft 365. They are currently looking for a Power BI Developer that will turn data into actionable insights. You'll design and maintain BI solutions, collaborate across teams, and deliver high-quality dashboards and reports. Key Responsibilities Develop data … Proven Power BI expertise (Desktop & Service), DAX, Power Query. Strong data modelling and ETL knowledge. Experience with Azure SQL, Data Lake, Synapse, or Microsoft Fabric. Understanding of GDPR and data security standards. Excellent communication and stakeholder engagement. Guidant, Carbon60, Lorien & SRG - The Impellam ...

help to build a better working world. General information Location: Dublin Available for Work Visa Sponsorship: No Business area: Technology Consulting (AI and Data) Contract type: Permanent The team and the opportunity Are you ready to shape how organisations use data to make smarter decisions … expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 7 years' experience working with Databricks. Good hands on experience with Spark, Delta Lake, and Unity Catalog Strong understanding of cloud platforms like Azure, AWS and/or Google Cloud Experience designing data lakes, lakehouses ...