1 to 25 of 236 Permanent Data Lake Jobs

Senior) Engineering Manager – Data & ML Global AXA XL recognizes digital, data and information assets are critical for the business, both in terms of managing risk and enabling new business opportunities. Data Science and Applied AI assets should not only be high quality, but also … drive a sustained competitive advantage and delivering a superior experience to our internal, external customers and improving efficiency. Our Innovation, Data and Analytics function is focused on driving innovation by optimizing how we leverage digital, data and AI to drive strategy and differentiate ourselves from the … from data engineering to ML operations (training, serving, monitoring), and microservice/web application engineering. Our tech stack: Azure Data Lake, Azure Databricks (Notebooks, APIs, Workflows), MLflow, dbt, Harness CI/CD, OpenShift, Docker, GitHub, JIRA DISCOVER your opportunity What will your essential responsibilities include more »

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … it can improve business performance. The role: Data Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on … be responsible for the day to day maintenance and enhancement of the data quality, security and performance of the Data Lake/Warehouse Assisting your Data Analyst colleagues to generate impactful business insights and analysis Support the Senior Data Engineer in more »

Cloud Architect/Data Architect/Cloud Data Architect ⚙️ Google Cloud/GCP, Data Engineering Knowledge/Background, Kubernetes, Clusters, Spark, Hadoop, ETL Tools etc 🏡Remote Based, Work From Home/WFH 💷£90k-£120k basic salary + bonus Benefits: 25 days holiday + bank … Gym Membership (Nuffield + Hussle) Life Insurance, Income Protection, Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect/Cloud Data Architect will work … devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to more »

Role: Senior Data Engineer Location: UK (Remote-first with monthly travel to London required) Recently listed as a "RegTech Top Performer" in Market Fintech's RegTech Supplier Performance Report, CUBE is pioneering the development of machine automated compliance. We are a global RegTech business defining and implementing the … change for the financial services industry. We deliver our services through a SaaS platform, powered by an innovative combination of AI and proprietary data ontology, to simplify the complex and everchanging world of compliance for our clients. At CUBE, we are creating the future and are a company … detection, classification, entity extraction, summarisation, and recommendations. Available to customers and partners as APIs and via a UI. Role mission: Establish a strong data architecture in GCP for the backend of AI/ML/DS Services supporting a SaaS product. Build cost-efficient, low latency data more »

will love at Sidetrade As a key member of our team, you will join a technical squad composed of QA, Software Engineers, Devops, Data Scientists, and Data Engineers to take up challenges related to new functionalities, scalability, security, ergonomy, etc. You will participate in the quality … language Familiar with the issues involved in web development Understand and challenge a rich technical stack (Micro-services, High-availability, Multitenancy, Data Lake) Knowledge of Gherkin is a plus Your first 90 days... Join our Immersive Bootcamp Collaborate with the team and participate to the roadmap to more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. … About you We know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing … service components. implement data solutions using a range of Azure cloud data services such as Azure storage (data lake), Cosmos DB and Azure SQL (inc. Synapse) work with members of the Data Science team, supporting and building data processing more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise … in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data more »

About the job Role: Data Platform Product Manager Location: Central London (2 days per week in office) Harmonic is delighted to partner with a high-growth research and insights scale-up, undergoing a phase of rapid expansion. They have grown 20% YoY for the past 5 years, and … following recent investment are continuing to scale. Being a data-led organisation carrying out some extremely interesting work, the data team has a significant role to play in aiding the growth and success of the organisation and offers the opportunity to work on cutting-edge technologies … such as AI and ML. The Data Platform team currently sits at a headcount of 4 but will double to 8 before the end of the year, offering amazing progression opportunities! Job Responsibilities: Directly lead and manage a team of 2 Data Analysts Embrace a visionary more »

car fleet with c.700k vehicles, it is crucial we continue to evolve and adapt through this period. We are committed to ensuring our data and processes are successfully maintained and enhanced to keep our customers moving. We are therefore recruiting for a Data Product Owner to … purpose today and ready for the challenges of the future, providing a bridge between our operational teams and technology department and facilitating better data decision making for our customers. Reporting to the Data Product Manager, you will: Play a vital role in owning and developing the … their need’s; successfully gathering and acting on user feedback and effectively communicating on updates and issues. Become an expert in Asset Risk data systems and processes and provide business knowledge when interacting with internal stakeholders and 3rd parties. Collaborate with key business stakeholders from a variety of more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day more »

Big Data Architect - Hybrid - £110K My client is a global telecommunications company that are looking to add a talented Big … Data Architect. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives. You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility. … The following skills/experience is essential: -Proven experience as an Architect and excellent knowledge of Big Data -Great understanding of Cloud e.g. Azure and or AWS -Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc -Excellent experience of ETL, data more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog. Exposure of Performance Tuning more »

Data Analyst Location: Hybrid - London Package: Negotiable + Benefits The Data Analyst for Data Operations will be responsible for gathering, collating and analysing data issues and translating them into data and reporting needs, using technical expertise to ensure data accuracy and high-quality, whilst ensuring delivery deadlines meet business requirements. Working closely with the data operations team, business stakeholders and the wider data team in resolving operational and data issues. The day to day: You will be an experienced analyst who … communication and collaboration skills Experience with data warehousing Experience in the Microsoft stack - Azure Data factory, Azure Data Lake Storage, Azure SQL Server, Azure Power Platform, Power BI. The rewards: negotiable basic salary and all the normal benefits you’d expect (Holiday, company more »

Data Product Manager – Supply Chain - £585 pd Outside IR35 Our pharmaceutical client now requires a talented Data Product professional to join their Global Supply Chain strategy team. This is a great opportunity to join a globally recognised brand and be a key interface between business customers … and technical teams. The Data Product Manager will be supporting the design, build and iteration of the data products across the Global Supply Chain in line with a pre-existing strategy. They will create data product roadmaps and lead the delivery and management of … the product from inception to maturity. Key Skills: Experience in Data Product Management (data governance/data architecture) Knowledge of Supply Chain Management Background in data engineering concepts (ie. ETL, data modelling, data lakes, master data more »

specific business challenges. Implement and maintain scalable and efficient AI systems, ensuring they meet business requirements and performance benchmarks. Collaborate with business analysts, data scientists, and IT teams to integrate AI solutions into existing systems and workflows, enhancing their capabilities and impact. Stay abreast of advancements in AI … efforts. Conduct data analysis and feature engineering to prepare data for use in AI models, utilizing Azure Data Lake and other data storage solutions. Develop robust testing and validation processes to ensure the accuracy and reliability of AI models and solutions. … Open AI Service, Azure Cognitive Services, and Azure Machine Learning. Familiarity with Azure Databricks is a plus. Solid background in machine learning algorithms, data pre-processing, feature engineering, and model evaluation. Experience with deep learning frameworks like TensorFlow or PyTorch is desirable. Proficiency in handling large datasets, experience more »

work together to minimise risk and shorten delivery times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff. ROLE: Data Engineer LOCATION: Remote (UK) CLEARANCE: SC eligible Job Description: We are seeking a highly skilled Data Engineer to join our team. The … project uses Java to ingest the data into the platform. The Consultant would be helping to build out the Data Lake and Lakehouse on Databricks, working with AWS. Stack: AWS: ECS, KMS, SNS, SQS, Lambda, Python, Databricks, MongoDB, Spring Boot AWS, Java (to support ingestion … Databricks - experience in PowerBi would also be useful. Engineer with past experience with Java, Data, and Infrastructure (DevOps). Java is a key skill Programming: Java, Python, PySpark Storage Mechanisms: MongoDB, Redshift, AWS S3 Cloud Environments/Infra: AWS (required), [AWS Lambda, Terraform] (nice to have) Data more »

I am searching for a talented Data Architect to join my client, a leading Fintech company in London. In this exciting role, you'll be responsible for designing, developing, and implementing the data architecture that forms the backbone of their data-driven strategy. You … ll work closely with stakeholders across the business to understand their needs and translate them into robust, scalable data solutions. Job Title: Data Architect Location: London (Hybrid) Salary: £95,000 - £100,000 Industry: Fintech Start: ASAP What you'll be doing: Design and develop enterprise-level … data architectures and solutions. Select and implement appropriate data management tools and technologies. Ensure data quality, consistency, and security across the organization. Collaborate with data engineers and analysts to deliver successful data projects. Stay up-to-date on the latest more »

Lead Big Data Engineer - Hybrid - £120K My client is a leading trading platform that is looking to add a talented Senior Big … Data Engineer You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives. You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility. … The following skills/experience is essential: -Proven experience as a Lead Big Data Engineer with excellent knowledge of Big Data -Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc -Excellent experience of ETL, data warehousing and handling a more »

and highly reliable Threat Detection Engineering platform. Key Responsibilities: Leverage cutting-edge technologies such as Splunk Enterprise Security, Exabeam Advanced Analytics (UEBA), Security Data Lakes (e.g., Data Bricks or Snowflake), and Cribl for log ingestion. Drive the adoption and implementation of Detection as Code principles. Leveraging … through code. Manage the full lifecycle of security tools and platforms, including evaluation, selection, implementation, and optimization. Develop and manage the security engineering data storage and compute budget, ensuring effective allocation of resources through management of the data lifecycle. Qualifications This role requires a great deal … Additional Information Discover the Unexpected Experian is the world’s leading global information services company. We’re passionate about unlocking the power of data in order to transform lives and create opportunities for consumers, businesses and society. For more than 125 years, we’ve helped economies and communities more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data … Azure stacks, including data pipelines, security and networking Expertise with data warehousing, data lakes and data lake houses Experience with master data management software and technologies Experience with ETL technologies including SQL Server, SSIS, Azure Data Factory more »

our team in London. As a BI Consultant, you will work closely with our clients to understand their business needs and develop innovative data-driven solutions to drive business growth and improve decision-making processes. Responsibilities: 1. Collaborate with clients to identify their business requirements, objectives, and key … performance indicators (KPIs) for effective data analysis and reporting. 2. Design and develop scalable and efficient business intelligence solutions, including data models, dashboards, reports, and analytics using industry-leading tools and technologies. 3. Extract, transform, and load (ETL) data from various sources into data warehouses or data lakes, ensuring data integrity, consistency, and quality. 4. Conduct data profiling, data cleansing, and data validation to ensure accuracy and reliability of business intelligence solutions. 5. Perform data analysis and interpretation to identify more »

A fantastic opportunity has a risen with our Financial Services client, for a Head of Data Engineering to join the team. The Head of Data Engineering will be responsible leading a team of data engineers to architect, build, and maintain a new data … of Data Engineering will be involved from the very beginning of the project and will have experience of Azure Data Lake and Databricks. This is a predominantly remote role, however some office visit will occasionally be required, therefore you must be able to travel to … successful digital transformation initiatives involving data engineering and analytics. Significant hands-on Microsoft Azure stack experience is a must: Data Lake, Data Factory, Synapse, Databricks, Power BI Strong in delivering solutions at scale, on time and within budget. Strong background in data more »

Data Architect - Azure/Spark/Databricks Role: Data Architect Location: London (3 days a week) Salary: £100k - £110k A leading technology firm, renowned for connecting people globally, is looking for a skilled Data Architect to join their expanding team. You will be spearheading … Azure-based data solutions and helping them further the development of their award winning data platform. In this pivotal role, you'll be responsible for leading the design and implementation of cutting-edge data solutions. Key Responsibilities: Lead the design and implementation of Azure … data design documents to articulate architecture decisions and implementation plans clearly. Requirements: Strong proficiency in Azure services, including Spark, Data Lake, Synapse, and Databricks. Exceptional stakeholder management skills with the ability to communicate effectively with both technical and non-technical audiences. Experienced and knowledgeable of more »

subject matter experts, solving interesting problems, and building the products of tomorrow, this opportunity is for you. On our mission to revolutionize the Data Strategy & Management space, we are seeking a dynamic and seasoned Solution Architect with a stellar track record in Data Products. As our … client navigates the complexities of a legacy IT landscape, we need someone who can not only conceptualize, design and build robust data platforms but also provide strategic guidance and stakeholder management expertise. Leveraging your software engineering expertise, you'll play a pivotal role in ensuring the integrity, efficiency … innovative solutions in challenging environments. Bachelor’s degree in computer science, computer engineering, or a relevant field. Responsibilities: Design and implement scalable database and datalake solutions tailored to client requirements. Evaluate client operations, applications, and programming to identify database structural necessities and ensure alignment with organizational goals. Create robust data more »

Solution Data Architect - Hybrid - £120K My client is a global telecommunications company that are looking to add a talented Solution … Data Architect. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives. You'll be responsible for managing a remote engineering team, driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility. … The following skills/experience is essential: -Proven experience as an Architect and excellent knowledge of Big Data -Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc -Excellent experience of ETL, data warehousing and handling a variety of data more »