51 to 75 of 185 Delta Lake Jobs in England

domain processes, including policy, claims, underwriting, pricing, risk, and regulatory reporting. Hands-on expertise with: Azure Data Factory (ADF) for pipeline orchestration Databricks (PySpark, Delta Lake, MLflow) for data engineering and ML integration Azure ecosystem (Key Vault, Data Lake, Synapse, Functions, Logic Apps) Strong understanding ...

workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development programs ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

data platforms. Expertise with one or more of the following technologies: AWS (Redshift, Glue, S3, Bedrock) Microsoft Azure (Synapse, Data Factory, Azure SQL) Databricks (Delta Lake, ETL pipelines, ML workloads) Snowflake Strong data modelling experience across enterprise-scale environments. Experience building modern AI-enabled data architectures. Knowledge ...

Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...

Lead the assessment and modernisation of legacy SQL Server, ETL and reporting workloads into a Databricks-based architecture. Design scalable data models, Delta Lake structures and transformation pipelines supporting real estate investment data. Analyse data lineage, dependencies, business logic and data quality to determine what should be migrated ...

such as Snowflake's Snowpark, Dynamic Tables, Streams & Tasks, and Snowflake Cortex, as well as Databricks components such as Lakeflow (Connect, Pipelines, and Jobs), Delta Lake, and Unity Catalog. Strong data engineering fundamentals: ETL/ELT pipeline design and implementation, data orchestration and workflow automation, batch and streaming ...

systematic approach to diagnosing and resolving issuesHighly desirable skillsExperience with streaming frameworks such as Flink, Kafka Streams and Kafka ConnectKnowledge of modern data lake technologies including Delta Lake, Iceberg and Glue Data CatalogExposure to DataOps practices and collaboration with Data Engineering teamsFamiliarity with GPU-accelerated analytics using ...

delivery of complex Data & AI programmes, including planning, mobilisation, resourcing, financial management, delivery governance, risk and dependency management.Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data & AI Intelligence ...

extensive benefits package. What you'll be doingDesign, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.Shape enterprise data architecture, balancing scalability, performance, security … mature a growing data function.What you need to bring“Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc)Advanced SQL and Python coding skills.A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions.Solid ...

extensive benefits package. What you'll be doingDesign, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.Shape enterprise data architecture, balancing scalability, performance, security … mature a growing data function.What you need to bring“Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc)Advanced SQL and Python coding skills.A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions.Solid ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

patterns, batch and streaming ingestion, event driven design Databricks and/or Microsoft Fabric at architecture level Azure Data Factory, Synapse, Azure SQL, Data Lake Storage Gen2 Advanced SQL, Python or PySpark, Spark optimisation, Delta Lake, Unity Catalog Azure DevOps or GitHub Actions, Terraform or Bicep Entra ...

Tech Environment My client is actively modernising their platform, moving away from legacy IaaS solutions and building a Databricks‐first Lakehouse . Spark, Delta Lake, CI/CD, and modern DevOps practices are central to how they design and scale data products. What You’ll Need Strong hands ...

Alongside this, you'll be collaborative and customer-focused, with passion for continuous improvement and delivering high-quality data solutions. Experience with Python, Spark, Delta Lake, DevOps tools, agile methodologies or cloud data governance would be advantageous. In return, we are offering a salary of c.£60k. discretionary ...

This is hands‐on delivery in a regulated iGaming environment: production pipelines, not notebooks. Key responsibilities Migrate ML data pipelines from Databricks (Spark/Delta Lake) to the SageMaker-based "golden template" architecture, working to the pattern set by the Lead ML Ops Engineer Build and amend feature … engineering pipelines, feature store integrations, and data access layers (S3, Glue, Lake Formation) supporting migrated models Implement parity and statistical testing to prove migrated pipelines/models match Databricks outputs Handle data migration/integration between Databricks and AWS: storage, permissions, IAM alignment Work within CI/ ...

This is hands‐on delivery in a regulated iGaming environment: production pipelines, not notebooks. Key responsibilities Migrate ML data pipelines from Databricks (Spark/Delta Lake) to the SageMaker-based "golden template" architecture, working to the pattern set by the Lead ML Ops Engineer Build and amend feature … engineering pipelines, feature store integrations, and data access layers (S3, Glue, Lake Formation) supporting migrated models Implement parity and statistical testing to prove migrated pipelines/models match Databricks outputs Handle data migration/integration between Databricks and AWS: storage, permissions, IAM alignment Work within CI/ ...

This is hands‐on delivery in a regulated iGaming environment: production pipelines, not notebooks. Key responsibilities Migrate ML data pipelines from Databricks (Spark/Delta Lake) to the SageMaker-based "golden template" architecture, working to the pattern set by the Lead ML Ops Engineer Build and amend feature … engineering pipelines, feature store integrations, and data access layers (S3, Glue, Lake Formation) supporting migrated models Implement parity and statistical testing to prove migrated pipelines/models match Databricks outputs Handle data migration/integration between Databricks and AWS: storage, permissions, IAM alignment Work within CI/ ...

This is an exciting role for a Senior Data Engineer with strong Databricks and Delta to join a world-leading Energy Trading company. Hybrid: London, Waterloo + Remote - 3 Days in the Office + 2 Days Remote Role Description: We are looking for a highly skilled Senior Data Engineer … scalable data pipelines and enabling real-time and batch analytics across the organization. Key Responsibilities: Design and implement scalable data pipelines using Databricks and Delta Live Tables (DLT) Develop and manage streaming data solutions for real-time analytics and decision-making Ingest data from various sourcesincluding SAP systems using ...

enhance our Databricks platform, ensuring it remains secure, scalable and reliable. Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake. Contribute to platform architecture, engineering standards and best practices across the data estate. Collaborate with Product, Analytics and Data Science teams to deliver … across the engineering team. Our ideal candidate Proven experience building and supporting scalable, production-grade data platforms. Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL. Good understanding of cloud-based data engineering, ideally within Azure environments. Experience working with Infrastructure as Code tools such ...

disciplines within enterprise environments4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse PlatformStrong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practicesExperience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics and artificial … Databricks Lakehouse Platform.Design and implement scalable data and analytics solutions leveraging Databricks technologiesBuild, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta LakeDevelop governed datasets and data products that support reporting, analytics, and artificial intelligence initiativesPartner with business stakeholders and technical teams to translate requirements into ...

customer in Central London. Key Responsibilities Data Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2). Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers … OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls ...

work with cutting-edge cloud technologies while supporting a business that is investing heavily in data-driven decision making. Desired Skills & Advantages Python, Spark, Delta Lake, DevOps tools, agile methodologies or cloud data governance would be advantageous. Salary and Benefits In return, we are offering a salary ...

reliability. Collaborate with analysts and AI engineers to deliver robust and reusable data products. Manage data lakes and warehouses using formats such as Delta Lake and Parquet. Implement best practices for data governance, performance, and security. Continuously evaluate and adopt new technologies to evolve Movar's data platform. ...