51 to 75 of 173 Permanent Delta Lake Jobs

data platforms. Expertise with one or more of the following technologies: AWS (Redshift, Glue, S3, Bedrock) Microsoft Azure (Synapse, Data Factory, Azure SQL) Databricks (Delta Lake, ETL pipelines, ML workloads) Snowflake Strong data modelling experience across enterprise-scale environments. Experience building modern AI-enabled data architectures. Knowledge ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development programs ...

decisions from concept through production deliveryCore Skills and ExperienceExperience in designing and implementing enterprise-scale data platforms. Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Spark, Structured Streaming, and Lakehouse architectureStrong hands-on experience with Snowflake, including data modelling, performance optimisation, Snowpark, data sharing ...

Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...

Lead the assessment and modernisation of legacy SQL Server, ETL and reporting workloads into a Databricks-based architecture. Design scalable data models, Delta Lake structures and transformation pipelines supporting real estate investment data. Analyse data lineage, dependencies, business logic and data quality to determine what should be migrated ...

data engineering, data platform delivery, software engineering, or related enterprise technology roles. Strong hands‐on experience with Databricks, Spark/PySpark, Python, SQL, Delta Lake, Azure, ADLS, and cloud‐based data engineering patterns. Experience designing scalable data pipelines, data products, and lakehouse solutions using modern cloud data platform ...

diagnosing and resolving issues Highly Desirable Skills Experience with streaming frameworks such as Flink, Kafka Streams and Kafka Connect Knowledge of modern data lake technologies including Delta Lake, Iceberg and Glue Data Catalog Exposure to DataOps practices and collaboration with Data Engineering teams Familiarity with GPU‐accelerated ...

systematic approach to diagnosing and resolving issuesHighly desirable skillsExperience with streaming frameworks such as Flink, Kafka Streams and Kafka ConnectKnowledge of modern data lake technologies including Delta Lake, Iceberg and Glue Data CatalogExposure to DataOps practices and collaboration with Data Engineering teamsFamiliarity with GPU-accelerated analytics using ...

Databricks Data Intelligence Platform, including migration from legacy data platforms, lakehouse modernisation, performance optimisation, governance, and production‐grade delivery. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data ...

delivery of complex Data & AI programmes, including planning, mobilisation, resourcing, financial management, delivery governance, risk and dependency management. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

Tech Environment My client is actively modernising their platform, moving away from legacy IaaS solutions and building a Databricks‐first Lakehouse . Spark, Delta Lake, CI/CD, and modern DevOps practices are central to how they design and scale data products. What You’ll Need Strong hands ...

reliability. Collaborate with analysts and AI engineers to deliver robust and reusable data products. Manage data lakes and warehouses using formats such as Delta Lake and Parquet. Implement best practices for data governance, performance, and security. Continuously evaluate and adopt new technologies to evolve Movar's data platform. ...

Strong understanding of data pipelines, ETL/ELT frameworks, orchestration tooling, and data engineering best practices. Familiarity with technologies such as Spark, Python, SQL, Delta Lake, Unity Catalog, dbt, Azure Data Factory, Airflow, Kafka, or equivalent tooling. Relevant certifications such as Snowflake SnowPro, Databricks Certified Data Engineer ...

Alongside this, you'll be collaborative and customer-focused, with passion for continuous improvement and delivering high-quality data solutions. Experience with Python, Spark, Delta Lake, DevOps tools, agile methodologies or cloud data governance would be advantageous. In return, we are offering a salary of c.£60k. discretionary ...

Data Architect, Lead Data Architect, Principal Engineer or comparable senior technical leader. Advanced, hands-on Databricks experience in production environments, including lakehouse architecture, Delta Lake and relevant governance, workflow, SQL and optimisation capabilities. Proven ownership of modern data platforms or data products from architecture through production delivery ...

This is an exciting role for a Senior Data Engineer with strong Databricks and Delta to join a world-leading Energy Trading company. Hybrid: London, Waterloo + Remote - 3 Days in the Office + 2 Days Remote Role Description: We are looking for a highly skilled Senior Data Engineer … scalable data pipelines and enabling real-time and batch analytics across the organization. Key Responsibilities: Design and implement scalable data pipelines using Databricks and Delta Live Tables (DLT) Develop and manage streaming data solutions for real-time analytics and decision-making Ingest data from various sourcesincluding SAP systems using ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

customer in Central London. Key Responsibilities Data Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2). Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers … OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls ...

enhance our Databricks platform, ensuring it remains secure, scalable and reliable. Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake. Contribute to platform architecture, engineering standards and best practices across the data estate. Collaborate with Product, Analytics and Data Science teams to deliver … across the engineering team. Our ideal candidate Proven experience building and supporting scalable, production-grade data platforms. Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL. Good understanding of cloud-based data engineering, ideally within Azure environments. Experience working with Infrastructure as Code tools such ...

large-scale data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality … . Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. #J-18808-Ljbffr ...

large-scale data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality … . Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent ...

data-protection focus throughout What you'll need: 5+ years' progressive data engineering experience Strong Databricks and Spark expertise - platform internals, performance, DAB, Delta Lake, declarative pipelines Advanced Python, with solid software/DevOps practice (version control, unit testing, pipelines-as-code, CI/CD) Excellent data modelling ...

delivery, and enhance solution quality while applying sound engineering judgement and responsible AI practices. What You'll DoDesign and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services. Build real-time, event-driven solutions using technologies such as Kafka and Azure ...