51 to 75 of 319 Data Lake Jobs

granular financial datasets. You’ll play a central role in shaping large-scale tick-data architecture - building pipelines, improving data-lake structures, and refining storage formats to support demanding research and trading workflows... Key Responsibilities Architect, build, and refine high-throughput pipelines capable of processing … Develop resilient, well-structured Python/OOP solutions underpinning tick-data ingestion, normalisation, storage, and replay Design and evolve data-lake patterns using Parquet and related columnar formats; introduce Iceberg-style table modelling where beneficial Implement cloud-backed processing and storage patterns - leveraging services from ...

granular financial datasets. You’ll play a central role in shaping large-scale tick-data architecture - building pipelines, improving data-lake structures, and refining storage formats to support demanding research and trading workflows... Key Responsibilities Architect, build, and refine high-throughput pipelines capable of processing … Develop resilient, well-structured Python/OOP solutions underpinning tick-data ingestion, normalisation, storage, and replay Design and evolve data-lake patterns using Parquet and related columnar formats; introduce Iceberg-style table modelling where beneficial Implement cloud-backed processing and storage patterns - leveraging services from ...

Company Nigel Wright are delighted to be partnering with the Institute of Directors in their search for a Senior Data Engineer.The Institute of Directors (IoD) is a thriving membership community for Directors in the UK and beyond, where you can connect with other leaders, develop your skills … Requirements You will have strong experience in data engineering, with a particular focus on data warehousing and data lake technologies. Additionally you will have a track record of establishing data governance best practises, including data security, data ...

Description: Responsibilities: AWS Lake Formation Implementation: Configure and manage AWS Lake Formation to establish secure, scalable data lakes, ensuring robust data governance and access control. IAM Configuration: Implement and manage AWS IAM solutions, including role-based access control (RBAC) and least privilege access … Governance: Implement identity governance practices to ensure regular review, validation, and auditing of user access permissions. Data Security and Compliance: Utilize AWS Lake Formation to enforce data governance policies and ensure compliance with export control regulations. Documentation: Create and maintain detailed documentation of configurations, processes ...

Skills Proven experience with Databricks (including Spark-based data processing). Strong knowledge of Azure Data Platform (Data Lake, Synapse, etc.). Proficiency in Python and SQL for data engineering tasks. Understanding of data architecture and ETL processes. Ability … work independently in a remote environment. Nice-to-Have Experience with CI/CD pipelines for data solutions. Familiarity with Delta Lake and ML pipelines . Start Date: ASAP Contract Type: Outside IR35 Apply Now: If you're a skilled Data Engineer looking ...

Skills Proven experience with Databricks (including Spark-based data processing). Strong knowledge of Azure Data Platform (Data Lake, Synapse, etc.). Proficiency in Python and SQL for data engineering tasks. Understanding of data architecture and ETL processes. Ability … work independently in a remote environment. Nice-to-Have Experience with CI/CD pipelines for data solutions. Familiarity with Delta Lake and ML pipelines . Start Date: ASAP Contract Type: Outside IR35 Apply Now: If you're a skilled Data Engineer looking ...

Days Onsite per week) Job Description Looking for L3 Data Engineer | Support | Key Skills & Expertise AWS Core Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/… Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking … experience with Power BI. Strong understanding of SQL, Python, or PySpark. Knowledge of the Azure data platform including Azure Data Lake Storage, Azure SQL Data Warehouse, or Azure Databricks. Some of the package/role details include: Salary up to £70,000 Hybrid ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking … experience with Power BI. Strong understanding of SQL, Python, or PySpark. Knowledge of the Azure data platform including Azure Data Lake Storage, Azure SQL Data Warehouse, or Azure Databricks. Some of the package/role details include: Salary up to £70,000 Hybrid ...

Role: Consulting Data Engineer Location: Bristol , UK Hybrid 2 Day in a week at Onsite RoleType: Permanent/Full time Job Salary: Depends on experience Role Summary: As a Consulting Data Engineer, youll help design, build and maintain data pipelines and platforms that enable … unlock the value in their data. Are you comfortable working in an ambiguous environment, where you build strong working relationships, and use data to understand client problems? Are you inquisitive and comfortable with questioning and challenge constructively? Sometimes youll work with experienced engineers and data scientists ...

Senior Data Engineer (Hybrid - 2 days per week onsite in london) Start Date: 5th January 2026 End Date: 23rd April 2026 Security Clearance: SC (required) We are looking for a Senior Data Engineer to join a multi-disciplinary team delivering a wide range of data solutions. You will play a key role in developing scalable, high-quality data products, enabling data-driven decision-making, and supporting business outcomes through innovative and governed data solutions. Key Responsibilities Develop and implement data flows connecting operational systems, analytics ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

Data & Integration's Lead (Azure) I am recruiting for a rapidly growing, multi-site healthcare organisation in the middle of a major digital transformation. As their Data & Business Intelligence function continues to expand, they require a hands-on Data & Integration's Lead/Manager … enterprise data and integration platforms Design, build, and manage Azure data pipelines using Data Factory, Data Lake, and SQL Own API and system-to-system integrations using Logic Apps, Function Apps, and related tools Oversee data governance, quality ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

pipelines using Databricks on Azure. Build and optimize data solutions leveraging Azure Data Services (e.g., Data Lake, Synapse). Work with large datasets to enable advanced analytics and reporting. Collaborate with stakeholders to deliver high-quality data solutions aligned … standards. Essential Skills Proven experience in Data Engineering within complex environments. Strong hands-on expertise with Databricks and Azure (Data Lake, Synapse, etc.). Solid understanding of cloud-based data platforms and ETL processes. Active SC Clearance (must be in place before starting ...

Lead Data Engineer (AWS Snowflake Python) London/WFH to £110k Are you a technologist Data Engineer with strong technical leadership skills? You could be progressing your career in an impactful, hands-on Lead Data Engineer role at a growing technology company as they … expand their UK presence. As a Lead Data Engineer you'll take responsibility for the team's engineering practices such as code reviews and CI/CD and for overall code quality and test coverage. You'll plan and manage the team's Sprints, collaborating closely with product ...

Lead Data Engineer (AWS Snowflake Python) London/WFH to £110k Are you a technologist Data Engineer with strong technical leadership skills? You could be progressing your career in an impactful, hands-on Lead Data Engineer role at a growing technology company as they … expand their UK presence. As a Lead Data Engineer you'll take responsibility for the team's engineering practices such as code reviews and CI/CD and for overall code quality and test coverage. You'll plan and manage the team's Sprints, collaborating closely with product ...

Lead Data Engineer (AWS Snowflake Python) London/WFH to £110k Are you a technologist Data Engineer with strong technical leadership skills? You could be progressing your career in an impactful, hands-on Lead Data Engineer role at a growing technology company as they … expand their UK presence. As a Lead Data Engineer you'll take responsibility for the team's engineering practices such as code reviews and CI/CD and for overall code quality and test coverage. You'll plan and manage the team's Sprints, collaborating closely with product ...

Data Architect - Fully Remote role Data Architect sought by a well-established Data & Analytics consultancy. Our client is seeking a cloud professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms … data pipeline design, with a technology-agnostic approach. A solid understanding of data warehouse and data lake principles. Expert SQL skills and demonstrable data modelling capabilities. About the Company: Our client is a well-established organisation in their field, known ...

About the Opportunity This is a unique opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role … scientists to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4+ years of industry experience in Data Engineering roles Advanced-level Python for data applications ...

About the Opportunity This is a unique opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role … scientists to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4+ years of industry experience in Data Engineering roles Advanced-level Python for data applications ...

Role Overview We are seeking an experienced Data Architect to lead the design and implementation of enterprise-grade data solutions on the Azure platform. The successful candidate will define and deliver scalable, secure, and cost-optimised architectures that support modern data strategies. Key Responsibilities … landscapes and propose modernisation or migration strategies. Azure Data Platform Expertise Design and implement solutions using: Azure Data Lake Storage Gen2 Azure Synapse Analytics Azure Databricks Azure Data Factory Azure SQL/Managed Instance Azure Cosmos DB Azure Event Hub/ ...

pipelines using Databricks on Azure. Build and optimize data solutions leveraging Azure Data Services (e.g., Data Lake, Synapse). Work with large datasets to enable advanced analytics and reporting. Collaborate with stakeholders to deliver high-quality data solutions aligned … standards. Essential Skills Proven experience in Data Engineering within complex environments. Strong hands-on expertise with Databricks and Azure (Data Lake, Synapse, etc.). Solid understanding of cloud-based data platforms and ETL processes. Active SC Clearance (must be in place before starting ...