76 to 100 of 243 Delta Lake Jobs in the UK

large-scale data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality … . Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent ...

data-protection focus throughout What you'll need: 5+ years' progressive data engineering experience Strong Databricks and Spark expertise - platform internals, performance, DAB, Delta Lake, declarative pipelines Advanced Python, with solid software/DevOps practice (version control, unit testing, pipelines-as-code, CI/CD) Excellent data modelling ...

initiatives by creating trusted, reusable data foundations. Essential Experience developing data engineering solutions on cloud platforms Strong SQL and Python skills. Experience with Databricks, Delta Lake and Spark (PySpark or Scala). Knowledge of modern data warehousing or lakehouse architectures. Hands-on experience developing production-grade data pipelines. ...

processes Experience supporting Generative AI solutions, including RAG architectures, vector search, LLM integration, AI orchestration, or agentic workflows Databricks experience, including Unity Catalog, MLflow, Delta Lake, Lakehouse architecture, and AI tooling Experience working within secure, regulated, or government environments Exposure to large-scale enterprise data and AI platforms ...

delivery, and enhance solution quality while applying sound engineering judgement and responsible AI practices. What You'll DoDesign and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services. Build real-time, event-driven solutions using technologies such as Kafka and Azure ...

OAuth 2.0, Azure Key Vault, and role-based access control (RBAC). Awareness of modern lakehouse concepts and integration with layered data platforms, including Delta Lake and raw/curated data layers. Exposure to Spark/Databricks and PySpark or Spark SQL for collaborating with data engineering teams ...

OAuth 2.0, Azure Key Vault, and role-based access control (RBAC). Awareness of modern lakehouse concepts and integration with layered data platforms, including Delta Lake and raw/curated data layers. Exposure to Spark/Databricks and PySpark or Spark SQL for collaborating with data engineering teams ...

implementing enterprise-scale Azure data platforms. Strong hands-on experience with Azure Data Factory. Strong hands-on experience with Azure Databricks, PySpark, and Delta Lake. Proven experience architecting complex ETL/ELT data integration solutions. Strong experience working with XML, XSD schema validation, XPath, and XSLT transformations. Experience integrating ...

interpersonal skills - you communicate clearly with stakeholders as well as other engineers, fostering a collaborative, supportive working environmentExperience in the financial markets, especially in delta one, store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks … e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about DRW's processing activities ...

skills - you communicate clearly with stakeholders as well as other engineers, fostering a collaborative, supportive working environment Experience in the financial markets, especially in delta one, store of value, and/or FICC options trading Experience with Linux-based, concurrent, high-throughput, low-latency software systems Experience with pipeline … orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW's processing activities ...

/column-level security, lineage and audit Build reusable Terraform modules for Databricks and automate platform changes through CI/CD pipelines Tune Spark, Delta and Photon workloads to improve performance and reduce DBU and cloud spend Manage identity, SSO/SCIM, service principals, secret scopes and network isolation … meet security and compliance needs Operate Databricks Workflows, Delta Live Tables and orchestration integrations used by Data Scientists and Data Support MLflow, model serving and feature store usage for predictive modelling and GenAI work Monitor jobs, clusters and warehouses; lead incident response and post-incident reviews QA own work ...

requirements into technical solutions and working across technical and non‐technical teams. “Nice To Have” Skills and Experience Experience with Databricks, Spark/PySpark, Delta Lake or comparable technologies. Experience with Celonis, process mining or process intelligence. Experience with enterprise platforms such as Salesforce, SAP or ServiceNow. Exposure ...

requirements into technical solutions and working across technical and non-technical teams. Nice To Have Skills and Experience Experience with Databricks, Spark/PySpark, Delta Lake or comparable technologies. Experience with Celonis, process mining or process intelligence. Experience with enterprise platforms such as Salesforce, SAP or ServiceNow. Exposure ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

Databricks Lakehouse Platform to deliver clean, production-ready data for enterprise analytics. Job Requirements Strong proficiency in PySpark, Databricks SQL, T-SQL, and handling Delta Lake formats Hands-on experience building production pipelines inside Databricks workspaces and Fabric capacities Practical knowledge of structural data formats including Parquet, JSON … pipelines and implementing automated data quality checks Job Responsibilities Construct scalable data ingestion pipelines using Fabric Data Factory, Dataflow Gen2, and Databricks Workflows Implement Delta Live Tables (DLT) to automate batch and real-time streaming data processing Build and maintain the Medallion Architecture silver & gold layer across shared storage ...

/CD.The ability to understand customer requirements and translate them into practical technical solutions. Excellent analytical, problem-solving and communication skills. Desirable experienceData-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation. Image or video visualisation, enhancement ...

divergent source systems into canonical models across multiple products. Strong grounding in Lakehouse architecture trade-offs: table formats, query engines, ingestion patterns (Iceberg vs Delta; Dremio vs Trino vs Snowflake; streaming vs batch). Hands-on experience with: Dremio, Snowflake, or Databricks AWS data services (S3, Glue, Athena, Lake … Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation at scale Data and table formats (Parquet, Avro, Apache Iceberg or Delta Lake) Experience designing SaaS data models, tenant isolation, shared schemas, access patterns, at scale. Ability to communicate architecture decisions to engineers, product leadership ...

This is an exciting role for a Senior Data Engineer with strong Databricks and Delta to join a world-leading Energy Trading company. Role Description: We are looking for a highly skilledSenior Data Engineerwith strong analytical capabilities and hands-on experience in building scalable data solutions. The ideal candidate … implementing scalable data pipelines and enabling real-time and batch analytics across the organization. Key Responsibilities: Design and implement scalable data pipelines usingDatabricks and Delta Live Tables (DLT) Develop and managestreaming data solutionsfor real-time analytics and decision-making Ingest data from various sourcesincludingSAP systems using Qlik ReplicateandADFto ingest ...

implementation of scalable data architectures on the Databricks Lakehouse platform. Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake. Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning. Collaborate with data architects, analysts … London office for 3 days per week for collaborative working and stakeholder engagement. Essential Requirements & Skills Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization. Programming Languages: Advanced proficiency in Python/PySpark and SQL. Cloud & Infrastructure: Strong background in cloud data ...

sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectationsPreferred qualifications, capabilities, and skillsExperience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworksFamiliarity with AWS-native data services such as Glue, Lake Formation, Redshift, or AthenaExposure ...

secure handling of inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworks Familiarity with AWS-native data services such as Glue, Lake Formation, Redshift, or Athena ...

data governance frameworks, metadata management, and regulatory compliancIt would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability toolingExperience leading architecture workshops with both technical and non-technical stakeholdersA background providing technical mentorship ...