1 to 25 of 2,145 Databricks Jobs

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 15 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent … family or parental status, or any other characteristic protected by law. Accommodation is available upon request for candidates taking part in the selection process. Senior Data Engineer (Azure/Databricks Job Family Grouping: Chief Underwriting Officer Job Family: Data & Analytics Location: London How does this role contribute to our collective success? The Data & Analytics department transforms raw data into actionable … MDW) technologies, big data, and Lakehouse architectures to ensure our data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test More ❯

solutions on the Azure Cloud ecosystem. They design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. This role also involves designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. The data engineer is proficient in Azure … Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB. They are also proficient in Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, Azure DataBricks with Spark, and other open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala. Preferred Education Bachelor's Degree Required Technical And Professional Expertise Commercial … as a Data Engineer or similar role, with a strong emphasis on Azure technologies. Proficiency in Azure data services (Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, Azure Databricks). Experience with data modeling, data warehousing, and big data processing (Hadoop, Spark, Kafka). Strong understanding of SQL and NoSQL databases, data modeling, and ETL/ELT processes. Proficiency More ❯

new, innovative and sustainable products and solutions, both for today and for a new era of building. To support our progress, we are currently recruiting for a Data Engineer (Databricks & BI) to come and join our team at Ibstock Head Office, LE67 6HS. (Applicants must have Right to work in the UK - We are unable to provide sponsorship for this … role) Job Purpose: The Data Engineer/BI Developer will play a critical role in developing and refining our modern data lakehouse platform, with a primary focus on Databricks for data engineering, transformation, and analytics. The role will involve designing, developing, and maintaining scalable data solutions to support business intelligence and reporting needs. This platform integrates Databricks, Power BI, and … JDE systems to provide near real-time insights for decision-making. Key Accountabilities: Ensure the existing design, development, and expansion of a near real-time data platform integrating AWS, Databricks and on-prem JDE systems. Develop ETL processes to integrate and transform data from multiple sources into a centralised data platform. Develop and optimise complex queries and data pipelines. Optimise More ❯

solutions on the Azure Cloud ecosystem. They design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. This role also involves designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. The data engineer is proficient in Azure … Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB. They are also proficient in Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, Azure DataBricks with Spark, and other open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala. Preferred Education Bachelor's Degree Required Technical And Professional Expertise Commercial … as a Data Engineer or similar role, with a strong emphasis on Azure technologies. Proficiency in Azure data services (Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, Azure Databricks). Experience with data modeling, data warehousing, and big data processing (Hadoop, Spark, Kafka). Strong understanding of SQL and NoSQL databases, data modeling, and ETL/ELT processes. Proficiency More ❯

solutions on the Azure Cloud ecosystem. They design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. This role also involves designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. The data engineer is proficient in Azure … Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB. They are also proficient in Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, Azure DataBricks with Spark, and other open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala. Preferred Education Bachelor's Degree Required Technical And Professional Expertise Commercial … as a Data Engineer or similar role, with a strong emphasis on Azure technologies. Proficiency in Azure data services (Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, Azure Databricks). Experience with data modeling, data warehousing, and big data processing (Hadoop, Spark, Kafka). Strong understanding of SQL and NoSQL databases, data modeling, and ETL/ELT processes. Proficiency More ❯

solutions on the Azure Cloud ecosystem. They design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. This role also involves designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. The data engineer is proficient in Azure … Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB. They are also proficient in Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, Azure DataBricks with Spark, and other open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala. Preferred Education Bachelor's Degree Required Technical And Professional Expertise Commercial … as a Data Engineer or similar role, with a strong emphasis on Azure technologies. Proficiency in Azure data services (Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, Azure Databricks). Experience with data modeling, data warehousing, and big data processing (Hadoop, Spark, Kafka). Strong understanding of SQL and NoSQL databases, data modeling, and ETL/ELT processes. Proficiency More ❯

for deploying and scaling data systems. Highly desired experience with Azure, particularly Lakehouse and Eventhouse architectures. Experience with relevant infrastructure and tools including NATS, Power BI, Apache Spark/Databricks, and PySpark. Hands-on experience with data warehousing methodologies and optimization libraries (e.g., OR-Tools). Experience with log analysis, forensic debugging, and system performance tuning. Exposure to cloud-based More ❯

and optimization of data pipelines and resources. Knowledge, skills and experience Essential: Cloud Platforms: Experience with Azure, AWS, or Google Cloud for data engineering. Cloud Data Tools: Expertise in Databricks, Snowflake, BigQuery, or Synapse Analytics. Programming & Scripting: Strong knowledge of Python, SQL, Spark, or similar. Data Modelling & Warehousing: Experience with cloud-based data architecture. CI/CD & DevOps: Knowledge of More ❯

data pipelines and cloud resources. Knowledge, skills and experience Essential: Cloud Platforms: Demonstrable experience with Azure, AWS, or Google Cloud for data engineering. Cloud-Based Data Tools: Expertise in Databricks, Snowflake, BigQuery, or Synapse Analytics for scalable data solutions. Programming & Scripting: Strong knowledge of Python, SQL, Spark, or equivalent cloud technologies. Data Modelling & Warehousing: Experience with modern cloud-based data More ❯

of data pipelines and cloud resources. Knowledge, skills and experience Cloud Platforms: Demonstrable experience with Azure, AWS, or Google Cloud for data engineering. Cloud-Based Data Tools: Expertise in Databricks, Snowflake, BigQuery, or Synapse Analytics for scalable data solutions. Programming & Scripting: Strong knowledge of Python, SQL, Spark, or equivalent cloud technologies. Data Modelling & Warehousing: Experience with modern cloud-based data More ❯

and turning data into powerful insights that drive transformation? We're looking for Data Engineers who have a broad range of data engineering skills, with a focus on Microsoft Databricks . Experience with Microsoft Fabric or Snowflake is also highly desirable. ABOUT THE ROLE As a Data Engineer, you will play a key role in delivering high-quality data solutions … data services. ESSENTIAL TECHNICAL SKILLS REQUIRED We are looking for a technically skilled data professional with a strong foundation in modern data platforms and engineering practices. Key competencies include: Databricks Platform Expertise : Proven experience designing and delivering data solutions using Databricks on Azure or AWS. Databricks Components : Proficient in Delta Lake, Unity Catalog, MLflow, and other core Databricks tools. Programming … with Kafka, Snowflake, Azure Data Factory, Azure Synapse, or Microsoft Fabric (highly desirable). Data Architecture Frameworks : Knowledge of Inmon, Kimball, and Data Vault methodologies. Nice to have certifications: Databricks Certified Data Engineer Associate Databricks Certified Data Engineer Professional ABOUT YOU You're not just a data expert-you're a problem solver, collaborator, and forward-thinker. You will be More ❯

Join to apply for the Senior Data Engineer (Databricks) role at DATAPAO Get AI-powered advice on this job and more exclusive features. This range is provided by DATAPAO. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range Direct message the job poster from DATAPAO At DATAPAO, data … fuels collaboration, and growth knows no bounds. We are a leading Data Engineering and Data Science consulting company, recognized for our innovation and rapid growth. We have been named Databricks EMEA Emerging Business Partner of the Year and have achieved a second consecutive appearance on the Financial Times FT1000 list . We are currently looking for a Senior Data Engineer … of our most complex projects - individually or by leading small delivery teams. Our projects are fast-paced, typically 2 to 4 months long, and primarily use Apache Spark/Databricks on AWS/Azure. You will manage customer relationships either alone or with a Project Manager, and support our pre-sales, mentoring, and hiring efforts. What does it take to More ❯

Join to apply for the Principal Data Engineer (Databricks) role at DATAPAO Join to apply for the Principal Data Engineer (Databricks) role at DATAPAO Get AI-powered advice on this job and more exclusive features. This range is provided by DATAPAO. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base … fuels collaboration, and growth knows no bounds. We are a leading Data Engineering and Data Science consulting company, recognized for our innovation and rapid growth. We have been named Databricks EMEA Emerging Business Partner of the Year and have achieved a second consecutive appearance on the Financial Times FT1000 list . We are currently looking for a Senior Data Engineer … biggest multinational companies where years-long behemoth projects are the norm, our projects are fast-paced, typically 2 to 4 months long. Most are delivered using Apache Spark/Databricks on AWS/Azure and require you to directly manage the customer relationship alone or in collaboration with a Project Manager. Additionally, at this seniority level, we expect you to More ❯

biggest multinational companies where years-long behemoth projects are the norm, our projects are fast-paced, typically 2 to 4 months long. Most are delivered using Apache Spark/Databricks on AWS/Azure and require you to directly manage the customer relationship alone or in collaboration with a Project Manager. Additionally, at this seniority level, we expect you to … Technical Expertise You (ideally) have 5+ years of experience in Data Engineering , with a focus on cloud platforms (AWS, Azure, GCP); You have a proven track record working with Databricks (PySpark, SQL, Delta Lake, Unity Catalog); You have extensive experience in ETL/ELT development and data pipeline orchestration (e.g., Databricks Workflows, DLT, Airflow, ADF, Glue, and Step Functions.); You … and optimize data like a pro; You know your way around CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation, or Bicep); You’ve hands-on experience integrating Databricks with BI tools (Power BI, Tableau, Looker). Consulting & Client-Facing Skills Ideally, you bring a proven history in consulting , from scoping to gathering requirements, designing solutions, and communicating effectively More ❯

new, innovative and sustainable products and solutions, both for today and for a new era of building. To support our progress, we are currently recruiting for a Data Engineer (Databricks & BI) to come and join our team at Ibstock Head Office, LE67 6HS. (Applicants must have Right to work in the UK - We are unable to provide sponsorship for this … role) Job Purpose: The Data Engineer/BI Developer will play a critical role in developing and refining our modern data lakehouse platform, with a primary focus on Databricks for data engineering, transformation, and analytics. The role will involve designing, developing, and maintaining scalable data solutions to support business intelligence and reporting needs. This platform integrates Databricks, Power BI, and … JDE systems to provide near real-time insights for decision-making. Key Accountabilities: Ensure the existing design, development, and expansion of a near real-time data platform integrating AWS, Databricks and on-prem JDE systems. Develop ETL processes to integrate and transform data from multiple sources into a centralised data platform. Develop and optimise complex queries and data pipelines. Optimise More ❯

Jira ensuring timely and high-quality deliverables. Strong written and verbal communication skills for effective team collaboration One or more of the following certifications are desired: AWS Certified Developer, Databricks, Agile/Scrum, Python Programmer Preferred Qualifications: Familiarity with Apache Spark or comparable distributed data processing frameworks, preferably for large-scale data transformations and analytics. Working knowledge of data governance … platforms (e.g., Collibra) and cloud-based analytics solutions (e.g., Databricks). Hands-on experience with modern BI tools (e.g., Qlik, Power BI, Tableau) for data visualization and insight generation. Awareness of data access control best practices (e.g., Immuta) and familiarity with role or attribute-based access control frameworks. Other: Must hold an Active DOD Secret, Top Secret or TS/ More ❯

leadership skills, with the ability to convey complex technical concepts to both technical and non-technical stakeholders. One or more of the following certifications are desired: AWS Certified Developer, Databricks, Agile/Scrum, Python Programmer Preferred Qualifications: DOD 8570 IAT Level II Certification may be required (GSEC, GICSP, CND, CySA+, Security+ CE, SSCP or CCNA-Security). Familiarity with Apache … Spark or comparable distributed data processing frameworks, preferably for large-scale data transformations and analytics. Working knowledge of data governance platforms (e.g., Collibra) and cloud-based analytics solutions (e.g., Databricks). Hands-on experience with modern BI tools (e.g., Qlik, Power BI, Tableau) for data visualization and insight generation. Awareness of data access control best practices (e.g., Immuta) and familiarity More ❯

/ELT principles , data architecture, and data warehouse concepts Familiarity with APIs, RESTful services, and JSON/XML data handling Preferred/Bonus Skills Experience with Azure Data Factory , Databricks , or AWS Glue Familiarity with CI/CD , version control (Git), and DevOps practices Knowledge of cloud platforms (Azure, AWS, or GCP) Basic understanding of data governance , security, and GDPR More ❯

to assess their data needs and recommend Azure-based solutions to streamline data processing and storage. Data Ecosystem Implementation : Design and implement solutions leveraging Azure Synapse Analytics, Data Factory, Databricks, Data Lake Storage, and related tools. Pipeline Optimization : Develop and maintain robust ETL/ELT pipelines to ensure efficient data ingestion and transformation. Technical Implementation Data Modeling and Warehousing : Design More ❯

enterprise-scale data platforms for data ingestion, data processing, data transformation, business intelligence, AI, and advanced analytics. Proven hands-on capability with relevant technology: Azure Platform, Azure Data Services, Databricks, Power BI, SQL DW, Snowflake, Big Query, and Advanced Analytics. Proven ability to understand low-level data engineering solutions and languages (Spark, MPP, Python, Delta, Parquet). Experience with Azure More ❯

years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: Proficiency in Databricks platform Advanced data pipeline design and development Data quality and governance Machine learning model development and maintenance Data integration processes Data security and privacy regulations Data visualization tools development Data warehouse More ❯

years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: Proficiency in Databricks platform Advanced data pipeline design and development Data quality and governance Machine learning model development and maintenance Data integration processes Data security and privacy regulations Data visualization tools development Data warehouse More ❯

years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: Proficiency in Databricks platform Advanced data pipeline design and development Data quality and governance Machine learning model development and maintenance Data integration processes Data security and privacy regulations Data visualization tools development Data warehouse More ❯

years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: Proficiency in Databricks platform Advanced data pipeline design and development Data quality and governance Machine learning model development and maintenance Data integration processes Data security and privacy regulations Data visualization tools development Data warehouse More ❯

years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: Proficiency in Databricks platform Advanced data pipeline design and development Data quality and governance Machine learning model development and maintenance Data integration processes Data security and privacy regulations Data visualization tools development Data warehouse More ❯