61 to 85 of 105 Data Lake Jobs in London

Role - Azure Data Engineer Rate - £600 per day inside Hybrid working - 1 day a week onsite (Centre of London) Duration - Initial 6 months My client is a large Financial services organisation based out of London and they are going through a huge data transformation that is … expected to run for at least 18 months. My client is seeking experienced Azure Data Engineers for a stakeholder-facing project. This role will have particular emphasis on Azure, Databricks and DevOps. Our client is in the consultancy sector but does not have a preference for previous experience … There is a massive emphasis on Pyspark and Databricks for this particular role. Technical Skills Required: Azure (ADF, Functions, Blob Storage, Data Lake Storage, Azure Data Bricks) Databricks Spark Delta Lake SQL Python PySpark ADLS Day To Day Responsibilities: Extensive experience in designing, developing more »

Data Engineering in Animal Health London/Kent – approx. 80% remote £60-70k basic salary plus 10% bonus If you’re as much of an animal lover as you are a data engineer, you won’t find a better opportunity to use your skills for … good. In this newly created role you’ll be setting up the data infrastructure that will help this thriving veterinary care company to hugely improve outcomes for pets – and their two-legged friends[MW1] . It's a role with the potential to be both personally and professionally … a UK work visa already in place. Able to set up, maintain and support a cloud data platforms/data lake such as Snowflake A solid understanding of analytics, reporting and dashboarding technologies Proficiency in API integration, and data extraction methodologies. Strong coding more »

Digital are happy to be supporting one of our clients in the Research sector who are currently on the look-out for a Data Engineer. Data Engineer Tech Stack & Experience; Python & large volumes of data. SQL. Google DataFlow would be handy. Cloud Experience (AWS, Google, Azure … a mindset to grow into more of a leadership position would be very helpful. Have come from a highly statistical/mathematical background. Data Engineer Responsibilities; Design, develop, and maintain scalable data pipelines. Implement ETL/ELT processes to extract data from a variety … of sources and load it into the data warehouse or data lake. Collaborate with data scientists and analysts to understand their requirements and design data models that support their needs. Write efficient Python code to perform data transformation, cleaning, and more »

Role - Azure Data Engineer Rate - £500 Per Day Outside IR35 Remote work with an openness to travel to London if required Duration - 6 months My client is seeking experienced Azure Data Engineers for a stakeholder-facing project. This role will have particular emphasis on Azure, Databricks … There is a massive emphasis on Pyspark and Databricks for this particular role. Technical Skills Required: Azure (ADF, Functions, Blob Storage, Data Lake Storage, Logic Apps, DevOps) Databricks Spark Delta Lake SQL Python PySpark ADLS Soft Skills Required: Stakeholder Management Experience in a Senior Data … support for Continuous Integration/Continuous Deployment (CI/CD) practices and an understanding of agile working methodologies. Knowledge and experience working with Delta Lake, a storage layer that brings ACID transactions to data lakes. Understanding of the Azure Platform setup including Subscriptions, Resource Group, IAM, Networking more »

Data Architect - Azure/Spark/Databricks Role: Data Architect Location: London (3 days a week) Salary: £100k - £110k A leading technology firm, renowned for connecting people globally, is looking for a skilled Data Architect to join their expanding team. You will be spearheading … Azure-based data solutions and helping them further the development of their award winning data platform. In this pivotal role, you'll be responsible for leading the design and implementation of cutting-edge data solutions. Key Responsibilities: Lead the design and implementation of Azure … data design documents to articulate architecture decisions and implementation plans clearly. Requirements: Strong proficiency in Azure services, including Spark, Data Lake, Synapse, and Databricks. Exceptional stakeholder management skills with the ability to communicate effectively with both technical and non-technical audiences. Experienced and knowledgeable of more »

TO WORK IN INVESTMENT DOMAIN* WHAT WE OFFER and WHAT THE POSITION IS ABOUT? Tools & Technologies: You’ll program in Java and model data using Alloy/Legend – a data management and data governance open-source platform that we will teach you. Other technologies … suggest improvements to products and to propose ways in which we can add value for our stakeholders. Interpersonal Communication: You’ll engage with data producers and consumers across all areas of the business to understand their requirements and to propose solutions tailored to their needs. Training: Your manager … Maven/Gradle/Ant) Experience with continuous delivery and deployment Proficient at working with large and complex code bases Alloy, Data Lake PURE Being able to describe, connect and visualise data Model to model mapping Creating data models NOTE : If you have more »

is made possible through our family of trusted brands and businesses. Across the Centrica group we have established a vast rudimentary data lake comprising of UK & Ireland wide customer and enterprise data for group and our trusted businesses (British Gas, Bord Gais Energy, Centrica Business … Solutions, Centrica Energy and Upstream). This data source presents a huge opportunity to create significant value for our customers, for us to build innovative products, provide new services and improve the capabilities of our Centrica employees to drive our journey as we transition towards NetZero. This is … in. We are looking for a highly skilled and motivated AI Product Owner to drive the entire product lifecycle for AI/ML, data, and analytic products. You will collaborate closely with Data Scientists, ML/Data Engineers, business leaders, privacy, Solution Architects, and more »

Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Data Product Owner London Hybrid – 3 days pw onsite £50,000 - £65,000 We have an exciting opportunity for a Data Product Owner working for a unique customer centric within the financial services industry, which is undergoing the biggest transformation it has ever seen. The Data Product Owner will join a newly formed data team. This role will ensure systems are fit for purpose today and ready for the challenges of the future, providing a bridge between operational teams and the technology department and facilitating better data decision making for … acting on user feedback and effectively communicating on updates and issues. Collaborate with key business stakeholders from a variety of teams, including Analytics, Data Engineering, Pricing, Risk, and Economic Capital to develop backlog and priorities in-line with business requirements. Proactively investigate and understand issues and pain points more »

have created is unique and difficult to replicate, with our ‘best of both’ approach combining deep human expertise with advanced digital, tech and data capabilities. This combination not only allows us to deliver excellent customer experiences, but to do so efficiently and at scale. We are proud of … of a business that cares too Want to continue to grow professionally and be the best version of yourself Job Description The Lead Data Engineer is a key role to ensure Shawbrook unlocks the value contained in its data across the organisation in a well-governed … and table formats (Delta etc) This is an incredibly exciting role that rarely becomes available. You will own a well-designed data lake serving a 50+ user community who run '000s of queries across TBs of data each month with a mandate to take it more »

available PAYE or via approved umbrella Initial CED 31/12/2024 The Power BI Developer reports directly to the Head of Data & Reporting/Project Controls Manger who is responsible for the co-ordination of Project Controls activities associated with the development, design, manufacture, construction and … safety, quality, time and cost milestones for a specific area/scope of work. The Power Bi Developer duties include identifying, gathering, analysing data and making their insights accessible to the SZC team through easy-to-understand graphs, charts, tables and reports. The Power Bi developer will work … and implementation of Digital reports and dashboards within SZC and to external clients using data extracted from the SZC Data Lake and information gathering systems. * Support the production of monthly reports through the provision of intelligent information and analysis. * Preparation of training materials and conducting more »

specific business challenges. Implement and maintain scalable and efficient AI systems, ensuring they meet business requirements and performance benchmarks. Collaborate with business analysts, data scientists, and IT teams to integrate AI solutions into existing systems and workflows, enhancing their capabilities and impact. Stay abreast of advancements in AI … efforts. Conduct data analysis and feature engineering to prepare data for use in AI models, utilizing Azure Data Lake and other data storage solutions. Develop robust testing and validation processes to ensure the accuracy and reliability of AI models and solutions. … Open AI Service, Azure Cognitive Services, and Azure Machine Learning. Familiarity with Azure Databricks is a plus. Solid background in machine learning algorithms, data pre-processing, feature engineering, and model evaluation. Experience with deep learning frameworks like TensorFlow or PyTorch is desirable. Proficiency in handling large datasets, experience more »

work together to minimise risk and shorten delivery times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff. ROLE: Data Engineer LOCATION: Remote (UK) CLEARANCE: SC eligible Job Description: We are seeking a highly skilled Data Engineer to join our team. The … project uses Java to ingest the data into the platform. The Consultant would be helping to build out the Data Lake and Lakehouse on Databricks, working with AWS. Stack: AWS: ECS, KMS, SNS, SQS, Lambda, Python, Databricks, MongoDB, Spring Boot AWS, Java (to support ingestion … Databricks - experience in PowerBi would also be useful. Engineer with past experience with Java, Data, and Infrastructure (DevOps). Java is a key skill Programming: Java, Python, PySpark Storage Mechanisms: MongoDB, Redshift, AWS S3 Cloud Environments/Infra: AWS (required), [AWS Lambda, Terraform] (nice to have) Data more »

Salary : £55,000 GBP per annum and Benefits applicable by the Client Purpose of Role: At Client we are committed to ensuring our data and processes are successfully maintained and enhanced to keep our customers moving. We are therefore recruiting for a Data Product Owner to … purpose today and ready for the challenges of the future, providing a bridge between our operational teams and technology department and facilitating better data decision making for our customers. Reporting to the Data Product Manager, you will: Play a vital role in owning and developing the … on their needs; successfully gathering and acting on user feedback and effectively communicating on updates and issues. Become an expert in Asset Risk data systems and processes and provide business knowledge when interacting with internal stakeholders and 3rd parties. Collaborate with key business stakeholders from a variety of more »

GBP per annum and Benefits applicable by the Client Purpose of Role: At Client we are committed to ensuring our data and processes are successfully maintained and enhanced to keep our customers moving. We are therefore recruiting for a Data Product Owner to join our new … purpose today and ready for the challenges of the future, providing a bridge between our operational teams and technology department and facilitating better data decision making for our customers. Reporting to the Data Product Manager, you will: • Play a vital role in owning and developing the … their need’s; successfully gathering and acting on user feedback and effectively communicating on updates and issues. • Become an expert in Asset Risk data systems and processes and provide business knowledge when interacting with internal stakeholders and 3rd parties. • Collaborate with key business stakeholders from a variety of more »

vehicles, it is crucial Motability Operations (MO) continues to evolve and adapt through this period. At MO we are committed to ensuring our data and processes are successfully maintained and enhanced to keep our customers moving. We are therefore recruiting for a Data Product Owner to … purpose today and ready for the challenges of the future, providing a bridge between our operational teams and technology department and facilitating better data decision making for our customers. Reporting to the Data Product Manager, you will: Play a vital role in owning and developing the … on their needs; successfully gathering and acting on user feedback and effectively communicating on updates and issues. Become an expert in Asset Risk data systems and processes and provide business knowledge when interacting with internal stakeholders and 3rd parties. Collaborate with key business stakeholders from a variety of more »

skilled resources and growing your career. Candidates must have the following Strong Developer experience on C# Azure, Logic Apps Database experience SQL Server Data Factory/Data Warehousing/ETL Skills required: Experience of developing technology solutions for large, complex business change programmes Experience of working … their pros and cons in the context of large organisations. Integration Architectures and patterns, including knowledge of Enterprise Service Bus. Working knowledge of data warehousing, data lakes, data migration and Big Data. Good secure coding standards such as OWASP. Auditing and monitoring approaches and more »

Senior Python Software Engineer (Senior Architecture Programmer Developer Python Java Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Senior Architecture Programmer Developer Python Java Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java and AWS. If you have more »

include design and development experience with various relevant Microsoft platforms, tools, technologies, patterns, and techniques related to MSFT suites, Azure tools, integration, and data is required. Experience with industry/domains like pharmaceutical, finance, HR, sales, marketing, and manufacturing is highly preferred. Experience with healthcare industry regulations, data … experience using Azure integration tools like Data factory, Logic Apps, etc. Along with knowledge of utilizing Azure SQL Server, Data lake, etc. In-depth knowledge & experience using Visual Studio, with one of the programming languages: C#/Java/JavaScript/Python, and PowerShell. Thorough … Queries, Datatype conversions, etc.). Experience working with APIs, Postman/SOAPUI tools, MS D365, ServiceNow, Azure Analytics tools, Azure Synapse, Azure BYOD & Data verse, Azure DevOps, Informatica, BI tools, etc. is a big plus. General business skills: Excellent communication to translate and explain business requirements to technology more »

Lead Data Engineer London - Hybrid Up to £70,000 Benefits include: 30 day holidays plus Bank Holidays. 75% commute costs covered, Private Medical, Private Dental, Pension up to 12% This role is remote heavy with some expectations to go into the London office when needed (est. 2 x … is to improve systems available across the UK and provide a more sustainable and efficient experience for all. In this role the Lead Data Engineer will work with Product Owners to understand the business strategy to align and influence the technical solutions throughout. Providing insight, value and suggestions … across complex data engineering projects. This is a hands on Technical Lead role, management experience is not required. Senior Data Engineers with excellent commercial experience ready to step up into a Technical Lead role will also be considered. The Lead Data will work across more »

Python Software Engineer Team Lead (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS GCP Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Apache Iceburg Arrow DBT gRPC protobuf Snowflake TypeScript Manager Finance Trading Front Office Investment Banking Asset Manager … asset classes such as Fixed Income, equities, FX or commodities Role: Python Software Engineer Team Lead (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS GCP Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with AWS, Glue, Dremio, Snowflake, Python and Java. more »

Role - Data Architect Salary - £80,000 Location - Fully Remote We have an exciting opportunity for an Azure Data Architect to join a very exciting consultancy. We are looking for a passionate candidate with fantastic communication skills that is able to design and implement data frameworks and advise on all aspects of Data Architecture. Skills & Responsibilities * Specialising in Azure * Complete understanding of which applications and systems work well together and which do not. * Complete understanding of best design principles for Data Lakes, Databases and Data Warehouses. * Complete … custom pipelines, custom scripts and custom APIs). * The ability to ensure that designs adhere to client governance rules. * The ability to allow data scientist to extract/ingest data and allow for machine learning packages to be included in pipelines (Data Bricks, Synapse more »