126 to 150 of 228 Data Lake Jobs in the UK

this organisation now, all have said glowing reviews about it, they're the leader in legal representation comparison and have rather interesting, unique data to work with. They are using modern Azure tools and have some very exciting initiatives! They are looking for an additional Snr Data Engineer to join a well established team, this could possibly suit a mid weight Data Engineer who has mentored others. Salary is circa 75K with a 5% bonus There is no sponsorship for this role, sorry. Responsibilities: Guiding a team of developers in the creation, optimization … warehouses and data lakes. Familiarity with Azure Cloud Services. Sound understanding of T-SQL programming. Exposure to Azure Data Lake technologies, including ADF, HDFS, and Synapse. Solid knowledge of Data Governance, Data Catalog, and Master Data Management. Understanding more »

Data Platform Architect €110k-€135k + Health + Pension + more Remote working (Two days per month in Dublin Office) We are looking for an experienced Data Architect to join a collaborative team of experts focused on creating a world-class data platform. As … a subject matter expert, you will serve as a technical thought leader in data management, database systems, and information processes across the company. You will have the freedom to design innovative solutions to unique challenges, using data to improve workplace experiences. This opportunity will allow you … strategies. Have demonstrated active listening and used candid feedback effectively. Are proficient in AWS Data technology Stack for building Data Lake solutions. Have expertise in relational and non-relational database systems. A great opportunity to join a global company and dynamic team, don’t delay more »

Lead Data Engineer (Contract/ASAP Start) Location| Workplace policy: Fully Remote (Must be based in the UK without visa/sponsorship requirements ) Day Rate: £575 to £625 (outside IR35) Contract Length: 6-month term initially An exciting market-leading tech-savvy organization that works directly alongside some … of the biggest brands in the retail sector is seeking a contract Lead Data Engineer with hands-on experience working directly with C-Suite execs. Technical Expertise Needed At least 4 years recent Databricks (must have) At least 10+ years experience in a Data Engineer role … recently operating at a hands on lead level Ideally educated to degree level in a QS top 100 university Proven experience delivering scalable data pipelines PySpark SQLDevOps/DataOps/CICD Expertise in designing, constructing, administering, and maintaining data warehouses and data lakes Data more »

working with companies in the finance sector highly desirable • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment. • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

SME underwriting decisions per month. Responsibilities include: To perform data extraction , manipulation and cleansing techniques within a data lake environment. Support the delivery of strategic initiatives into Credit SME to build scale, reduce manual touchpoints and enhance the controls environment. Providing Portfolio analysis through the … use of analytical methods. Implementation and design of risk management & changes within 3rd party tools i.e. CRM/UD portal etc Utilise customer data to build risk segmentation and mitigation strategies Lead role for supporting SME commercial strategy and input into key working groups across pricing, system development … you have experience in, or comfortable with: Experience in the extraction and cleansing techniques of data held within a data lake environment Develop Credit Risk Strategies supported by Decision Engine Rules, and Scorecards (Nice to have) Working knowledge of other automating solutions (implementation and maintenance more »

will develop superior digital and technology concepts and architecture solutions as well as support technical implementations actively and on site, applying your sound Data/Cloud technical know-how, your understanding of business contexts, along with your analytical and conceptual skills. As you acquire experience, you will be … technical project management, digital delivery, or technology consulting environment • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

working with a leading Blockchain analytics platform, with a stellar leadership team and excellent backing. Seeking to hire an experienced senior level Backend & Data Engineer, with strong Python skills and backend architecture ability. The Role Working alongside a strong technical and engineering team, you will take ownership of … redeveloping the architecture of their entire backend systems, focusing on growth. Working with Python and extrapolating high volumes of data from the blockchain, you will be responsible for the overall vision, strategy and execution of the company’s Software products. You will take ownership of: Technical Leadership: Team … building out complex Backend services, along with high-throughput Data warehousing. Min 7yrs with Python Big Data & Data lake solutions; PostgreSQL, Clickhouse or SnowFlake etc Cloud Infrasutcurre (AWS services) Data processing pipelines using Kafka, Hadoop, Hive, Storm, or Zookeeper Hands-on more »

Data Architect IAM, Financial Service, Models, Big Data, Warehouses, Data Lakes, Agile, CI/CD Reference: SB/DaAr/01 Sector: Competitive Hybrid Working Town/City: London Contract Type: Contract Closing Date: 24/04/2023 Data Architect IAM … London/Hybrid Contract - 6 Months Competitive Daily Rate Our client needs an experience IAM Data Architect to work an innovative Access Management Programme for a Financial Service organisation. You will lead on Data Architect designs to create build builds data models to fulfil … the strategic data needs of the Access Management Programme in accordance with the policies and standards set out by the Chief Data Office. Must have strong recent Identity and Access Management data technologies project experience within a large Big Data environment. Must more »

Data Engineer - London - Hybrid £60,000 - £70,000 + Package You will be responsible for the engineering of the data platform which is essential to the running and evolution of their mass participation event portfolio. This involves working with databases, coding in Python, integrating data from API's, building automated ingest and curation data pipelines, and using Azure cloud services to hydrate their core data system and downstream systems used for key business functions and analytics. You will be reporting into the Head of Data, working closely … thousands of Charities that they work with. Tech Stack: Python and SQL Demonstrable knowledge and skills in Azure Cloud Stack - Azure Data Lake, Azure Data Factory, Azure Logic Apps, Azure Function App and Azure SQL Managed Instance or Microsoft SQL Server. Skilled in T-SQL more »

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … it can improve business performance. The role: Data Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on … be responsible for the day to day maintenance and enhancement of the data quality, security and performance of the Data Lake/Warehouse Assisting your Data Analyst colleagues to generate impactful business insights and analysis Support the Senior Data Engineer in more »

Synapri collaborates with a large transport company, who is seeking a Senior Data & Analytics Developer for a 12 months contract. You will be involved in the design, development and configuration of applications, components and tools according to the technical plans set by the Development Technical Lead. Knowledge and … experience required: In-depth working knowledge of ETL/ELT, data warehousing/business intelligence methodologies including dealing with big data, cloud technology and unstructured data Knowledge of star schema structure & design, detailed understanding of Kimball & Inmon and … hybrid data warehouse design Detailed knowledge using cloud data products like Data Factory, Event Hubs, Data Lake, Synapse, Azure SQL server. Detailed knowledge in developing in Databricks and experience in coding with PySpark. Spark SQL ETL coding standards: ensuring that code more »

Job Title: Data Platform Administrator Location: Edinburgh (hybrid) Company: Primis is currently partnered with a leading technology company dedicated to delivering innovative solutions that empower businesses in the UK. Their team is comprised of talented individuals who thrive in fast paced environments and are passionate about pushing the … boundaries of technology and creating meaningful impact. Position Overview: We are looking for an Data Platform Administrator, someone who can deliver operational excellence and stability in our data platforms that underpin the services we deliver for our customers. As well as the day-to-day you … database administration team. Essential skills : Experienced in design, implementation and day-to-day running of enterprise level data platforms, data lake concepts and data handling Solid monitoring and tuning skills in highly available transactional or API-centric environments Working with large-scale, uptime more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … tools into existing structures. Create data tools to support sport data modelling, prediction and analytics. Work with data and analytics experts to strive for greater functionality in our data systems. Essential Skills Proven experience as a Data Engineer, with expertise more »

team reflective of society improves outcomes. We provide support for any needs or adjustments during the application process. Role Overview: As a Senior Data Engineer, your responsibilities include: Developing modern data systems and platforms to drive decision-making. Collaborating with clients to shape their data strategy. Enabling public sector organizations to adopt a data-driven approach. Owning and managing cloud infrastructure using a DevOps approach. Building data pipelines, handling various data types, and integrating error handling. Implementing data system architectures like Data Warehouse … Data Lakes, and Data Meshes. Creating reusable libraries for data pipelines and documenting end-to-end systems on a cloud environment. Adopting a robust DevOps approach and utilizing tools for DataOps. Ensuring software resilience through testing strategies and best practices. Applying SOLID, DRY, and more »

governance forms (both Technical Design Authority (TDA) & Architecture Review Board (ARB)). Chair the governance forums at times. Scoping and designing any relevant data migration exercise to target platform, identification of relevant tools to aid with migration. Create usable Solution Architecture documentation that can be consumed by internal … and external delivery and engineering teams Work closely with Lead Security Architect, Head of Technology & Data Architecture to ensure that: Technology & Data Architecture layers of the Target architecture are aligned to future strategic direction and taking on Security principles, capabilities, guardrails are reflected in Target Architecture … offer SaaS products Understanding of enterprise estate management in mixed SaaS, bespoke and platform environments. Technical aspects include Azure cloud, Azure Data Lake, Data Warehouse, Integration architectures (event, messaging, API), security models, threat modelling, IDP integration and security designs, MI platforms, portals, MS Power platform more »

Data Architect Salary: £90,000 - £110,000 Location: Remote/Hybrid Data Idols are working with a fast-growing technology company who are disrupting the software engineering industry. They are looking for a Data Architect with strong experience in data engineering and … experience in delivering innovative solutions. The Opportunity In this role as a Data Architect, you will play a pivotal role in ensuring the efficiency of their data architecture while heading the migration from legacy systems to modern cloud solutions. You will design and build scalable database and data lake solutions, implement CICD pipelines and automate the deployment of data products into production environments. Skills and Experience Proficiency in AWS Great communication more »

to join the team. The BI and Datawarehouse Test Analyst will play a crucial role in ensuring the accuracy, reliability, and quality of Data Warehouse (DWH) and Business Intelligence (BI) solutions. Responsibilities: Take ownership of DWH and BI … Testing Design and execute tests to validate various aspects of the system. Validating the flow of data into the Data Lake and Data Warehouse. Performing data reconciliation to ensure consistency and accuracy. Generating and reconciling regulatory reports. Conducting risk-based testing … with a focus on priority scenarios. Contribute to the development and delivery of comprehensive Test Plans and Schedules. Define the necessary test data requirements for each testing phase. Identify and define the test environmental requirements for effective testing. Execute tests according to established procedures and promptly raise defects more »

to join the team. The BI and Datawarehouse Test Analyst will play a crucial role in ensuring the accuracy, reliability, and quality of Data Warehouse (DWH) and Business Intelligence (BI) solutions. Responsibilities: Take ownership of DWH and BI … Testing Design and execute tests to validate various aspects of the system. Validating the flow of data into the Data Lake and Data Warehouse. Performing data reconciliation to ensure consistency and accuracy. Generating and reconciling regulatory reports. Conducting risk-based testing … with a focus on priority scenarios. Contribute to the development and delivery of comprehensive Test Plans and Schedules. Define the necessary test data requirements for each testing phase. Identify and define the test environmental requirements for effective testing. Execute tests according to established procedures and promptly raise defects more »

Data Engineer | ETL | Python | ML | Insuretech | £80,000 | Barnet WeDo has recently partnered with a leading Insurtech company who are currently in the market for a Data Engineer to join the business. They are looking for someone who has worked very much in a Data Engineering capacity however, they need to have showcased prior experience of working with Data Science teams on ML Pipelines and Data Platforms and have a good understanding of modelling/practices. This extends to productionising and having a good grasp on some of the … CI/CD tooling. This position will be sat within the Data Science function within the business. The Data Engineer will have the following experience and skillset. Proficient in Microsoft Azure, Python/PySpark, and SQL. Experienced in using ETL/ELT tools like SSIS and more »

TRM-International are hiring on behalf of a Wholesale financial Institute . We are seeking a Data Infrastructure Engineer to develop and maintain their enterprise Data Platform. The platform aims at serving data processing, data consumption, business intelligence analytical tools, and development … applications. This is an attractive opportunity for you as a Data Infrastructure Engineer to integrate into a respectable position within a leading and established organisation. There will also be long-term career progression along-side an intellectual and friendly team. Objectives and responsibilities of this role Oversee all … the platform’s technical facets to provide best-in-class data and services to users. Create, automate, and maintain data pipelines originating from diverse data sources, intended for consumption by various systems and users. Improve the platform’s architecture with respect to the infrastructure more »

financial backbone of the nation. Inspire People is working with the Bank of England to seek an experienced Solution Architect with expertise in data analytics and cloud-based solutions (Azure) to play a key role in the development and enhancement of the architecture strategy and roadmaps for data … financial system, to collaborate with industry experts and thought leaders and contribute to the shaping of national financial policies and practices through effective data analytics. Key Responsibilities: * Lead pivotal programmes and projects, crafting data solutions that align with business and technology strategies. * Translate business goals into … enterprise-wide technology and data strategies. * Design and produce architectural artefacts for data platforms, migration strategies, and analytics solutions. * Drive the development and enhancement of data and analytics architecture strategy. * Guide the implementation of data governance standards and best practices. * Represent and more »

Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »