12 of 12 Remote Delta Lake Jobs

Key Responsibilities Lead the design, development, and maintenance of scalable, high-performance data pipelines on Databricks. Architect and implement data ingestion, transformation, and integration workflows using PySpark, SQL, and Delta Lake. Guide the team in migrating legacy ETL processes to modern cloud-based data pipelines. Ensure data accuracy, schema consistency, row counts, and KPIs during migration and transformation. Collaborate … and analytics. ________________________________________ Required Skills & Qualifications 10-12 years of experience in data engineering, with at least 3+ years in a technical lead role. Strong expertise in Databricks , PySpark , and Delta Lake . DBT Advanced proficiency in SQL, ETL/ELT pipelines, and data modelling. Experience with Azure Data Services (ADLS, ADF, Synapse) or other major cloud platforms (AWS … of data warehousing , transformation logic , SLAs, and dependencies. Hands-on experience with real-time streaming near-realtime batch is a plus., optimisation of data bricks and DBT workload and Delta Lake Familiarity with CI/CD pipelines, DevOps practices, and Git-based workflows. Knowledge of data security, encryption, and compliance frameworks (GDPR, SOC2, ISO ).good to have Excellent More ❯

Data Engineer/Data Engineering/Data Consultant/Lakehouse/Delta Lake/Data Warehousing/ETL/Azure/Azure Databricks/Python/SQL/Based in the West Midlands/Solihull/Birmingham area, Permanent role, £ + car/allowance (£5,000) + 15% bonus. One of our leading clients is looking to recruit … role Salary £ + car/allowance + bonus Experience: Experience in a Data Engineer/Data Engineering role Large and complex datasets Azure, Azure Databricks Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory Desirable: Exposure ML/Machine Learning/AI/Artificial Intelligence More ❯

mindset with a passion for modern data and platform technologies. Nice to Have: Experience implementing data governance and observability stacks (lineage, data contracts, quality monitoring). Knowledge of data lake formats (Delta Lake, Parquet, Iceberg, Hudi). Familiarity with containerisation and streaming technologies (Docker, Kubernetes, Kafka, Flink). Exposure to lakehouse or medallion architectures within Databricks. More ❯

methodologies that enhance efficiency, scalability, and cost optimisation Essential Skills for the Senior Data Engineer: Proficient with Databricks and Apache Spark, including performance tuning and advanced concepts such as Delta Lake and streaming Strong programming skills in Python with experience in software engineering principles, version control, unit testing and CI/CD pipelines Advanced knowledge of SQL and More ❯

Azure with a core focus on Databricks Provide technical leadership on Databricks Design and implement data pipelines in Azure Databricks, optimizing for performance, cost, and scalability Advise of data lake migration, data modelling and pipeline optimisation Refactor and modernize existing data models to integrate with Power BI Coordinate phased migrations to ensure zero disruption to live operations Replace legacy … a strong focus on building scalable data pipelines Expertise in Azure Databricks (7+years) including building and managing ETL pipelines using PySpark or Scala (essential) Solid understanding of Apache Spark, Delta Lake, and distributed data processing concepts Hands-on experience with Azure Data Lake Storage Gen2, Azure Data Factory, and Azure Synapse Analytics Proficiency in SQL and Python More ❯

support advanced analytics, AI, and business intelligence use cases. Proven experience in designing architectures for structured, semi-structured, and unstructured data , leveraging technologies like Databricks, Snowflake, Apache Kafka , and Delta Lake to enable seamless data processing and analytics. Hands-on experience in data integration , including designing and optimising data pipelines (batch and streaming) and integrating cloud-based platforms More ❯

years as a Data Engineer Hands-on with Databricks, Spark, Python, SQL Cloud experience (Azure, AWS, or GCP) Strong understanding of data quality, governance, and security Nice to Have: Delta Lake, DBT, Snowflake, Terraform, CI/CD, or DevOps exposure You'll: Build and optimise ETL pipelines Enable analytics and reporting teams Drive automation and best practices Why More ❯

performance and cost across the Databricks platform. Collaborate with teams to support data pipelines, ML workflows, and analytics use cases. Integrate Databricks with cloud storage solutions (S3, ADLS) and Delta Lake. Troubleshoot issues related to jobs, performance, or user access. Maintain platform documentation and conduct user training. Stay current with Databricks updates, new features, and best practices. Support hybrid … administering Cloudera Hadoop clusters. 2-3+ years of Databricks experience in production environments. 2+ years of Databricks administration experience on Azure (preferred). Strong knowledge of Spark and Delta Lake architecture. Experience with IAM, Active Directory, and SSO integration. Familiarity with DevOps and CI/CD for data platforms. Deep understanding of Hadoop ecosystem: Hive, Impala, Spark … Python for automation and system administration. Solid foundation in Linux/Unix system administration. Preferred Qualifications Experience with cloud technologies (Azure or GCP preferred). Exposure to modern data lake and hybrid cloud architectures. Cloudera and/or Databricks certifications. Familiarity with infrastructure-as-code and automation tools (e.g., Terraform). Experience supporting Oracle database administration (e.g., backups, user More ❯

Science, Mathematics, Engineering or other STEM A strong team player with empathy, humility and dedication to joint success and shared development. Desirable Experience and Qualifications: Experience with Databricks or Delta Lake architecture. Experience building architecture and Data Warehousing within the Microsoft Stack Experience in development Source control (e.g. Bit Bucket, Github) Experience in Low Code Analytical Tools (e.g. More ❯

facing consulting environment, with the ability to manage stakeholder expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 5+ years' experience working with Databricks, including Spark and Delta Lake Strong skills in Python and/or Scala for data engineering tasks Comfortable working with cloud platforms like Azure, AWS, and/or Google Cloud A problem More ❯

with the ability to manage stakeholder expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 7 years' experience working with Databricks. Good hands on experience with Spark, Delta Lake, and Unity Catalog Strong understanding of cloud platforms like Azure, AWS and/or Google Cloud Experience designing data lakes, lakehouses, and modern data platforms Proven experience More ❯

multiple time zones. Ability to foster a collaborative learning environment that encourages continuous improvement and knowledge sharing. Nice to Have: Familiarity with technologies such as Elasticsearch, Solr, PostgreSQL, Databricks, Delta Share, and Delta Lake. Experience working with complex patent and litigation data models. Exposure to external data sources such as DocDB, Espacenet, and USPTO. Proficiency with Pandas and More ❯