51 to 75 of 137 Permanent Data Lake Jobs in London

with one of the best leading insurance companies, in collaboration with a prestigious bank in London. We're actively seeking a talented Senior Data Engineer to enrich our … dynamic team. Responsibilities: Experience designing data models and developing industrialized data pipelines. Strong knowledge of database and data lake systems, with proficiency in Python and SQL (any flavor). Comfortable with shell scripting (e.g., Bash) and provisioning new infrastructure in leading cloud providers more »

Job DescriptionData Architect - Azure/Spark/DatabricksRole: Data ArchitectLocation: London (3 days a week)Salary: £100k - £110k A leading technology firm, renowned for connecting people globally, is looking for a skilled Data Architect to join their expanding team. You will be spearheading Azure-based data solutions and helping them further the development of their award winning data platform. In this pivotal role, you'll be responsible for leading the design and implementation of cutting-edge data solutions. Key Responsibilities:Lead the design and implementation of Azure-based data … comprehensive data design documents to articulate architecture decisions and implementation plans clearly.Requirements:Strong proficiency in Azure services, including Spark, Data Lake, Synapse, and Databricks.Exceptional stakeholder management skills with the ability to communicate effectively with both technical and non-technical audiences.Experienced and knowledgeable of Data more »

exciting time to join this ambitious team!!WHAT YOU'LL BE DOINGAs Senior Machine Learning Engineer, you will sit within the company's Data Science team and report to the Director of Engineering. Their Machine Learning stack runs on on-demand containerised infrastructure – ranging from sub-second latency … Horovod.TECH STACK:Python, Flask, Redis, Postgres, React, Plotly, Docker. Temporal; AWS Athena SQL, Athena & EMR Spark, ECS Fargate; Azure Synapse/Data Lake Analytics, HDInsight.KEY RESPONSIBILITIESLead the productionisation of Monolith’s ML models and data processing pipelines, both mid-low-level system and design and … exemplary hands-on implementations using Spark and other tech stacksShape the ML engineering culture and practices around model & data versioning, scalability, model benchmarking, ML-specific branching & release strategyConcisely break down complex high-level ML requirements into smaller deliverables (epic à task)Proactively identify blockers across the team, help more »

machine-powered detection, enrichment, correlation, prioritization, triage, and investigation, freeing analysts to proactively protect their organizations. Hunters utilizes an open security data lake architecture, ensuring complete and cost-effective coverage of the entire security stack. Enterprises like Booking.com, Snowflake, and ABInBev leverage Hunters SOC Platform to empower more »

Job DescriptionBig Data Architect - Hybrid - £120KMy client is a leading trading platform that is looking to add a talented Big … Data Architect. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives.You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility.The following … skills/experience is essential:-Proven experience as an Architect and excellent knowledge of Big Data-Excellence experience across Azure-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data more »

Job DescriptionBig Data Architect - Hybrid - £110KMy client is a global telecommunications company that are looking to add a talented Big … Data Architect. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives.You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility.The following … skills/experience is essential:-Proven experience as an Architect and excellent knowledge of Big Data-Great understanding of Cloud e.g. Azure and or AWS-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and more »

Job DescriptionLead Big Data Engineer - Hybrid - £120KMy client is a leading trading platform that is looking to add a talented Senior Big … Data Engineer You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives.You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility.The following … skills/experience is essential:-Proven experience as a Lead Big Data Engineer with excellent knowledge of Big Data-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of more »

Job DescriptionSolution Data Architect - Hybrid - £120KMy client is a global telecommunications company that are looking to add a talented Solution … Data Architect. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives.You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility.The following … skills/experience is essential:-Proven experience as an Architect and excellent knowledge of Big Data-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data types-Very more »

Job DescriptionSenior Data Engineer – Higher Education – 6 Months (Rolling) - London - £522.50One of the UK’s leading higher education clients now requires a Senior Data Engineer on an initial six-month contract to improve their learning engagement platform.The Data Engineer will be predominantly responsible for … setting up and configuring a new Azure Data Lake.The ideal Data Engineer will have the following skills/experience:Proven Experience working on Azure Data FactoryPreviously worked with Python and SQLPreviously worked with Power BIInterpersonal and Stakeholder Management SkillsThis is a hybrid position and … would require you onsite 1 – 2 day(s) per week onsite at their office in London.If you’re an experienced Data Engineer looking for an exciting new opportunity with a leading Higher Education client, please apply more »

SQL Engineer | SQL | T-SQL | ETL | Azure | Azure DevOps | Enterprise Data Warehouse SQL Engineer needed on a FTC basis. Gravitas continue their support for a leading non for profit organisation on their search for a SQL Engineer. You will be responsible for maintaining and developing BI applications and … solutions. Supporting the Lead BI SQL Developer with all things Enterprise Data Hub related. Role: SQL Engineer Location: London - Once a month onsite Salary: £55,000 - £62,000 Type: 12 month FTC Benefits: 27 days holidays plus bank holidays, additional leave purchase scheme, group pension plan 6% contribution … as gym, restaurants and cafes. The essential skills needed for this position: Strong, hands on experience with SQL and T-SQL. Experience with Data Warehousing and the ETL process. Strong project management experience, including CICD and DevOps. Strong analytical & communication skills Good stakeholder engagement skills. The knowledge you more »

with Architecture decisions and strategy Requirements: TypeScript/JavaScript, Node.js, and React. Strong database skills (SQL) and RESTful API design experience. Working with data pipelines and data architecture Desired Skills: Experience with Python. Familiarity with AWS and Data Platforms Knowledge of Enterprise SaaS development … data lakes like snowflake This is a role that would be great for a Software Engineer that wants to drive the architecture for a product that has the potential to make a huge impact across this particular sector. Must be eligible to work in the UK and be more »

Job DescriptionA global leader in the design, marketing and distribution of premium lifestyle products is seeking a customer data platform product manager, to join their global customer insights team.You will be responsible for leading the development of their customer data platform. The ideal candidate will understand … customer facing businesses, have ROI-driven digital marketing acumen, be data fluent, and play a pivotal role to bridge the gap between business and technical teams.Responsibilities:Bridge the gap between the business and IT; ensure all requirements are documented, understood, and delivered.Own customer data platform development … US and EMEA, managing best practice communication and guidelines.Serve as the first point of contact for platform-related questions and support resolution of data issues.Understand consumer privacy regulations like GDPR and work with DPO, legal, engineering, and marketing teams to ensure compliance.Work to define data maps more »

The role We are looking for a Senior Data Scientist to help us improve our FlexiPay Acquisitions. Who are we? Our mission is to build the place where small businesses get the funding they need to win. We do this by uniting cutting-edge data and … and other lifecycle management. Work with our Collections and Technology teams to help set up appropriately automated collections processes. Provide requirements to our data team to ensure all necessary data for analysis on FlexiPay is suitably stored within our data lake. Proactively problem solving more »

will include…You will be able to produce a variety of reports focusing on a range of areas, including; Security, Performance, compliance audits, data collected from various IT Systems and tools.Strong ability to maintain and develop technical documentation, including system configurations, network diagrams, operating procedures to support clients … IT Environments.You will be analysing large data-sets to identify trends, patterns and anomalies, and have the ability to extract worthwhile insight to be included in the reports.Be an SME around identifying opportunities where automation report created can be used, using scripting languages and reporting tools to enhance … efficiency and scalability.What you’ve got…Experienced background in Report Writing, Data Analysis, or technical documentation within the IT/Tech industry.Experience on Microsoft office applications! Particularly Excel + Word, for documenting and report writing purposes.Being confident on Azure is a must! Including, Azure SQL, Data more »

business-critical IT architecture and digital solutions. We create technology platforms and deliver exceptionally customized solutions of the future for our clients, from data strategies to next-generation customer journey solutions and transformational business models. We solve the most daunting challenges that companies face today and envision for … will develop superior digital and technology concepts and architecture solutions as well as support technical implementations actively and on site, applying your sound Data/Cloud technical know-how, your understanding of business contexts, along with your analytical and conceptual skills. What You'll Bring (Experience & Qualifications … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

business-critical IT architecture and digital solutions. We create technology platforms and deliver exceptionally customized solutions of the future for our clients, from data strategies to next-generation customer journey solutions and transformational business models. We solve the most daunting challenges that companies face today and envision for … working with companies in the finance sector highly desirable • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment. • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python and AWS. If you have more »

Senior Data Architect Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to Be Yourself At Work. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that … agile and entrepreneurial culture. Your Capco Day/Key Responsibilities You will accompany and lead complex agile consulting projects supporting our clients with data architecture design and delivery. You will work with technology leaders across Financial Services to provide best practice guidance for data management and … procedures, architectures and technologies in one or more of the following topics: Contextual Datamodelling, Entity Relationship Modelling, Logical & Physical Datamodelling and Data Lake design. Experience designing and implementing cloud data migration and storage patterns on one or more of AWS, GCP and Microsoft Azure Experience more »

Senior Strategy Manager - Data & AI At Centrica, our purpose is to energise a greener, fairer future. Achieving this … is made possible through our family of trusted brands and businesses. Across the Centrica group we have established a vast rudimentary data lake comprising of UK & Ireland wide customer and enterprise data for group and our trusted businesses (British Gas, Bord Gais Energy, Centrica Business … Solutions, Centrica Energy and Upstream). This data source presents a huge opportunity to create significant value for our customers, for us to build innovative products, provide new services and improve the capabilities of our Centrica employees to drive our journey as we transition towards NetZero. This is more »

D365, D365 F&O, D365 Consultant, AX 2012, Dynamics AX, Microsoft power BI, SSRS, ETL, Reporting developer, SQL, Azure synapse, Data Lake, Lake House, Delta Lake, Data verse, DAX, BI reports, Paginated Report Builder, South London, Hybrid, £60-£75K Our end user client … with Microsoft Dynamics 365 Finance Operations and Project Operations 4+ years' experience working with Microsoft Power BI 4+ years' experience working with enterprise data warehouse/BI projects. … 4+ years' experience working with Data Analysis Expressions (DAX) Extensive experience with SQL Experience with Data Verse Data Lake, Lake House, and Delta Lake Experience Responsibilities: Dynamics Specific Development, Maintenance, and Support: Developing, maintaining, and providing support for custom analytics and more »

Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits Brown & Brown, Inc. is one of the largest insurance brokers globally having in excess of $4bn revenue in 2023 by providing risk management solutions to help protect what our customers value most. We have experienced significant growth over … size of the business by improving customer experience, growing organically and acquiring, accelerating knowledge and capabilities of teammates and using technology with purpose. Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them … Technical Skills – Deep hands on experience with the following data technologies: Stack including, Azure Data factory, Azure Data Lake Storage, Azure SQL, Azure Cosmos DB, Azure Power Platform, primarily power BI Data Lake & Data Mesh implementation The rewards more »

for designing, developing, and maintaining data pipelines and systems in Azure Cloud environments, with a strong emphasis on Azure Data Lake, Azure Data Factory, and Microservices architecture. Key Responsibilities: Develop and maintain data pipelines in Azure Cloud environments, leveraging Azure Data Lake, Azure Data Factory, and Microservices architecture. Design and implement scalable and efficient data processing solutions using technologies such as ServiceBus, REST API, App Service, SQL Data Warehouse, and Data Mart. Utilize Vector Databases, Cosmos DB, Redis, and Elasticsearch … engineering. Bachelor’s Degree or higher in Computer Science or a related field. Proficiency in Azure Cloud environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Experience with Vector Databases, Cosmos DB, Redis, Elasticsearch. Strong programming skills in Python, Spark, Databricks, Pyspark, SQL more »