126 to 150 of 188 Databricks Jobs in the North of England

that business users can effectively harness data insights to drive strategic decision making. Our data strategy is centred around an enterprise Lakehouse platform on Databricks, avoiding fragmented, ungoverned silos on legacy technologies that hamper creativity and scalability. We are building a governed, interoperable data estate that enables self-service … contribute to business transformation projects, working closely with colleagues across Data and with business stakeholders, and using modern technologies such as Power BI and Databricks to deliver trusted, accessible insight. The ideal candidate is an analytics engineer looking for a new challenge who is enthusiastic about using technology to improve ...

/ETL workflows within a Fabric Lakehouse environment Work with OneLake, Lakehouse architecture, and modern data engineering best practices Utilise PySpark, SQL and Databricks for complex data transformation and engineering workloads Create robust data ingestion, orchestration and monitoring processes Support the development of enterprise-wide analytics solutions and Power … production-grade ELT/ETL pipelines Strong skills in PySpark and SQL Experience working with OneLake, Delta formats or modern Lakehouse architectures Exposure to Databricks and distributed data processing environments Understanding of data modelling, governance and optimisation principles Experience delivering data solutions that support analytics, reporting and business intelligence Passion ...

doing Deliver large-scale data migrations from on-premises or legacy platforms to cloud-native AWS architectures, including Lakehouse implementations using Databricks or Snowflake. Design, build, and optimise low-latency data pipelines that process real-time betting events, odds updates, and market data at scale. Implement robust data quality, validation … Kinesis, Apache Kafka/MSK, Apache Flink, Spark Streaming Languages Python, SQL, Spark Pipeline tools dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Data quality Great Expectations, dbt tests, custom reconciliation frameworks IaC/DevOps Terraform, AWS CDK (desirable) What we’ll offer you We trust ...

traditional ecosystems—with extensive experience building data solutions using Microsoft Azure and/or AWS native services, with development to production experience ideally in Databricks, but also Snowflake or Fabric, managing delivery with an Agile mindset. Data Product mentality: You understand how to build Data Products ensuring that every pipeline … decision making through advanced analytics.AI: You have a deep expertise or experience of AI, specifically how to build AI solutions in modern data platforms (Databricks/Snowflake). In addition to Microsoft Azure, leveraging core technologies in the AI Foundry ecosystem, understanding different business challenges require varied solutions from ...

An exciting opportunity for a Data & ML Engineer to join a thriving technology and innovation hub where talented engineers, data specialists and industry experts come together to build solutions that make a real difference. Working ...

journey. Let’s create tomorrow, together, today. About the role and team: Role overview CreateFuture is delivering the migration of an ML estate from Databricks to an AWS SageMaker-based MLOps platform, working alongside AWS. The Senior Data Engineer (ML) sits in the Databricks workstream team (Delivery Manager, Lead … platforms. This is hands‐on delivery in a regulated iGaming environment: production pipelines, not notebooks. Key responsibilities Migrate ML data pipelines from Databricks (Spark/Delta Lake) to the SageMaker-based "golden template" architecture, working to the pattern set by the Lead ML Ops Engineer Build and amend feature engineering ...

journey. Let’s create tomorrow, together, today. About the role and team: Role overview CreateFuture is delivering the migration of an ML estate from Databricks to an AWS SageMaker-based MLOps platform, working alongside AWS. The Senior Data Engineer (ML) sits in the Databricks workstream team (Delivery Manager, Lead … platforms. This is hands‐on delivery in a regulated iGaming environment: production pipelines, not notebooks. Key responsibilities Migrate ML data pipelines from Databricks (Spark/Delta Lake) to the SageMaker-based "golden template" architecture, working to the pattern set by the Lead ML Ops Engineer Build and amend feature engineering ...

Data Engineering profile. They’re open to people from Data Engineering, Platform, DevOps or Systems Engineering backgrounds, but strong data foundations and hands-on Databricks experience are key.The roleYou’ll work within a cross-functional team, helping continue the move towards a Databricks-native environment and building reliable, scalable data … platform capabilities.You’ll be working across:Databricks, Python, SQL and Spark High-volume data ingestion and transformation Production ETL/ELT pipelines APIs, SFTP/FTPS and automated data retrieval CI/CD, automated testing and deployment Platform reliability, monitoring and troubleshooting Azure infrastructure, security and governance Data Science ...

data science adoption Key Skills and Experience Own end-to-end EMEAPAC data platform across design, delivery and steady-state operation Strong understanding of Databricks and Fabric architecture, including Lakehouse patterns Ability to oversee ingestion, transformation and reporting layers across enterprise data flows Working knowledge of Databricks (Delta Lake, pipelines ...

from diverse data sources and formats Hands‐on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross‐functional stakeholders, with strong communication skills for both technical and non‐technical audiences Awareness … support highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical teams ...

from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross-functional stakeholders, with strong communication skills for both technical and non-technical audiences Awareness … support highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical teams ...

from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross-functional stakeholders, with strong communication skills for both technical and non-technical audiences Experience applying … build, and support highly scalable data engineering solutions that power Moodys next-generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moodys technical teams ...

Data Engineer to build, maintain, and optimise robust data pipelines. This role develops efficient ETL/ELT workflows using Microsoft Fabric tools and the Databricks Lakehouse Platform to deliver clean, production-ready data for enterprise analytics. You will implement Delta Live Tables, write PySpark and SQL in Fabric Notebooks … Databricks clusters, and ensure data access control, quality checks, and governance through Unity Catalog. #J-18808-Ljbffr ...

optimising data pipelines, ensuring high-quality, business-ready data is available across the organisation. The role offers exposure to the latest Microsoft Fabric and Databricks technologies, with responsibility spanning data ingestion, transformation, Lakehouse development and platform optimisation. Working closely with stakeholders across the business, you will help deliver scalable data … solutions that support enterprise reporting, analytics and future data initiatives. Key Responsibilities Build and maintain scalable data pipelines using Microsoft Fabric and Databricks, supporting the ongoing development of the organisation's modern data platform. Develop and maintain Medallion Architecture data layers, ensuring data is available for reporting and analytics. Create ...

Senior Data Engineer to help shape the future of its data platform. You'll take ownership of building scalable data solutions in Databricks primarily, establishing a single source of truth, and enabling trusted, self-service access to data across the business click apply for full job details ...

scalable pipelines and platforms that power reporting and analytics across the whole organisation. You'll be working hands-on with Microsoft Fabric and Databricks, building out a new Lakehouse architecture and introducing data products that people across the business can actually trust and use. Essential Skills & Experience: Hands-on experience … with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi-structured data (Parquet, JSON, CSV) Solid understanding of CI/ ...

needs and translate them into clear, deliverable requirements Design, build and maintain semantic models, datasets and reporting solutions using Power BI, paginated reports and Databricks Improve report and dataset performance by reviewing data models, DAX calculations, queries and refresh approaches Apply data validation, reconciliation and testing to ensure analytics outputs … technical audiences Experience with AI-assisted development tools and copilots to improve productivity and delivery outcomes is advantageous Familiarity with Tabular Editor and Databricks is desirable Familiarity with specialty (re)insurance or Lloyd’s market data such as bordereaux, delegated authority, underwriting and claims is advantageous Core Competencies Demonstrates expertise ...

/Bronze, Silver & Gold patterns Delta tables SQL Azure Git/source control CI/CD Testing, monitoring and data quality Experience with Azure Databricks is particularly relevant and transfers well into what the team is building within Fabric. You don't necessarily need commercial Fabric experience … worked in a modern Databricks/Spark/Python engineering environment. Equally, we're open to strong Python Data Engineers who have built production-grade data solutions through code and are looking to move deeper into PySpark, distributed processing and Fabric. What makes this different? The business isn't simply ...

mandatory) Employment Type: Permanent | Fixed-Term Contract | Contract (Inside IR35) Start Date: Immediate Key Responsibilities Lead the architecture, design, and implementation of Azure and Databricks-based data platforms. Provide technical leadership, mentoring, and guidance to engineering teams. Collaborate with business and technical stakeholders to define platform roadmaps and best practices. … using Terraform and Azure services. Support strategic decision-making by aligning platform capabilities with business objectives. Essential Skills & Experience Strong hands-on experience with Databricks development and platform architecture. Good understanding of Azure infrastructure, networking, and Terraform. Proven experience in solution architecture, technical design, and stakeholder management. Experience leading technical ...

best practices and quality checks.Performance Optimization: Optimize queries and improve performance of data retrieval operations.Technology Stewardship: Stay updated with the latest advancements in AWS, Databricks, and the data engineering field in general. Provide recommendations on the adoption of new tools and technologies.Requirements:AWS Services: Proficiency in AWS services like Lambda … Kinesis, and Glue.Databricks & Delta Lake: Hands-on experience with Databricks and Delta Lakehouse architectures.Programming: Proficient in Python, Scala, and SQL for data engineering tasks.ETL Processes: Demonstrated expertise in designing, building, and maintaining ETL processes.Data Modelling: Experience in logical and physical data modelling with large datasets.Problem Solving: Strong analytical and problem ...

time zones The authority to act as the ultimate point of technical escalation Exposure to a modern Microsoft data environment, including Azure Data Lake, Databricks, Azure SQL and Python The chance to improve data pipelines, medallion architecture, governance, quality and engineering practices Significant visibility across technology leadership, architecture, product teams … critical data services The ability to engage and influence a diverse range of technical and non-technical stakeholders Strong experience across Azure Data Lake, Databricks, Azure SQL and Python A good understanding of data pipelines, medallion architecture and modern data platform design Knowledge of data management, lineage, governance and quality ...

clearance for this role.Desirable skills- Public sector or regulated industry experience (government, healthcare orsimilar)- Experience with modern data warehouse and lakehouse ecosystems (Redshift, Snowflake, Databricks, BigQuery, Synapse)- Familiarity with SQL and Python or Java- Experience with MLOps and Infrastructure as Code (Terraform, CloudFormation etc.) for data, analytics and AI services … infrastructuredelivery- Cloud platform architecture certification, with a data or AI specialism beneficial- Certification in modern data warehouse/AI ecosystems (Redshift, Snowflake, Databricks etc.)- Experience using AI or Agentic techniques in application of the role through GitHub CoPilot, Claude Code, or equivalentBenefitsAlong with a great company culture, Credera provides ...

Location: Manchester (Hybrid, 2 days per week in the office) The Opportunity I'm currently working with a leading insurance organisation based in Manchester that is looking to appoint a Data Governance Manager to help ...

data platform to enable analytics, reporting, and AI initiatives across Pricing, Underwriting and Claims. You will design and build scalable data pipelines on Azure Databricks and Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong ...

standards.Own the data platform roadmap from an architectural perspective, defining how the platform evolves over time (e.g., SAP BW transition/retirement path, Databricks maturation, Datasphere integration layer decisions).Contribute to enterprise architecture governance forums, representing data platform considerations in broader technology decisions.Provide architectural input to vendor and tooling decisions … bound and governed.Maintain platform adoption, performance, cost and standards-compliance metrics, using them to guide roadmap priorities and architecture governance decisions.PERSONAL PROFILEDeep expertise in Databricks (Unity Catalog, Delta Lake, Spark, Databricks SQL) or equivalent Lakehouse platforms, including administration, cost management, and performance optimisation.Strong understanding of cloud-native architecture principles, cost ...