1 to 25 of 78 ETL Jobs in the West Midlands

warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.* Design and build high-performance data pipelines: to extract, transform, and load data into Cloud Data Lake Storage and other Cloud services.* Develop and maintain secure data warehouses and data lakehouses: Implement data models, data quality … more of the following technologies**Azure Databricks*** Design and build high-performance data pipelines: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.* Experience of Databricks ML and Azure ML to develop predictive models and drive business insights.* Proven ...

working with Microsoft Fabric and/or Azure Databricks. Expertise in Spark-based data engineering and cloud data platforms. Strong knowledge of ELT/ETL processes, orchestration, data modelling and Lakehouse architectures. Experience using Azure DevOps, Git repositories and CI/CD pipelines. Understanding of data governance, security, GDPR and ...

Python for automation.Hands-on experience with Microsoft Fabric and Azure Data Factory, including pipeline orchestration, linked services, environment management, and parameterisation.Experience with SSIS for ETL processes and legacy data integration tasks.Expertise in designing and implementing CI/CD pipelines using tools such as Azure DevOps.Experience with Infrastructure as Code (e.g. ...

pipelines and automated deployments Experience with Python for data engineering or analytics An understanding of data warehousing principles and methodologies is preferable Experience developing ETL/ELT processes Knowledge of Microsoft Fabric and modern data platform technologies is preferable Knowledge of Microsoft Purview and data governance frameworks Microsoft certifications including ...

data requirements across complex environments. Gather, profile, and analyse large-scale structured datasets to ensure data accuracy, quality, and completeness. Design and implement robust ETL/ELT frameworks for ingesting, transforming, validating, normalising, and cleansing data. Develop and maintain scalable data pipelines to support analytics and operational decision-making. Implement ...

user engagement, considering communication requirements and business readiness.Experience in data mining and manipulation for large datasets, experience of data profiling and experience of writing ETL code to present data for data quality rules and analysis.Experience in working within or product managing data development environment, acting as a stakeholder contact and ...

scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python (or equivalent language) skills for data manipulation and automation. Hands ...

quality scalable data solutions.Technical Expertise• Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow).• Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent.• Strong SQL and Python (or equivalent language) skills for data manipulation and automation.• Hands ...

engagement, considering communication requirements and business readiness. Experience in data mining and manipulation for large datasets, experience of data profiling and experience of writing ETL code to present data for data quality rules and analysis. Experience in working within or product managing data development environment, acting as a stakeholder contact ...

accelerate future onboarding of systems and entities. Data Pipeline Development & Support Develop, enhance, and maintain data pipelines across the data platform. Build and optimise ETL/ELT processes for new and existing data sources. Troubleshoot and resolve pipeline failures and data quality issues. Implement monitoring and alerting to ensure reliable … Provide technical expertise on integration and architecture decisions. Skills, Knowledge & Expertise Data Engineering: Proven experience in a Data Engineer role. Experience building and maintaining ETL/ELT pipelines. Strong understanding of data integration practices and data lifecycle management. Experience supporting production data platforms. Cloud Platforms: Experience with Microsoft Azure data ...

scale enterprise environments. Strong expertise in data architecture, modelling, governance, and data management. Experience designing: Data Lakes Data Warehouses Lakehouse Architectures Data Virtualisation Solutions ETL/ELT Frameworks Strong understanding of: Relational Databases NoSQL Databases Graph Databases Cloud Data Platforms Experience delivering cloud migration and transformation programmes. Knowledge of analytics ...

management skills.Desirable Skills· Experience supporting applications hosted within Microsoft Azure.· Azure DevOps and CI/CD pipeline exposure.· Power Platform.· Logic Apps.· SSIS and ETL processes.· Oracle databases.· Experience supporting integrations with third-party systems.· Experience working within public sector environments.· Microservices architecture.· ITIL Foundation certification.· Microsoft Azure certifications.· Security ...

Skills & Experience: Experience as a Data Engineer, Data Developer or similar . Strong Python and SQL skills. Commercial Databricks experience. Experience with data pipelines, ETL/ELT or data processing. Understanding of cloud data engineering. Active SC or DV Clearance Desirable: AWS, Google Cloud, Defence/Central Government, large-scale ...

operations, security and business stakeholders. Support governance, reporting, audit readiness and continuous improvement. Implement automation and operational best practices where applicable. Technical Skills: SQL, ETL, Power BI, Tableau, Data Modelling Preferred Skills: Enterprise-scale operations experience, stakeholder management, automation mindset, strong analytical and problem-solving capabilities. ...

components for large-scale data processing solutions and deploying into a production environment Proficient in SQL and SQL extensions for analytical queriesSolid understanding of ETL/ELT data processing pipelines and design patternsAware of key features and pitfalls of distributed data processing frameworks, data stores and data serialisation formatsAble ...

technologies including Azure Data Lake Storage Gen2, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL and Event Hubs Designing and developing robust ETL/ELT frameworks, data pipelines, Lakehouse architectures and real-time data solutions, ensuring engineering excellence and reusability across the platform Establishing and governing enterprise data … Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL and Event Hubs Extensive experience designing and implementing enterprise-scale data architectures, including data modelling, ETL/ELT frameworks, Lakehouse solutions, batch and real-time data processing, and cloud-native data platforms Strong understanding of data governance, security, architecture best practices ...

Home/Malvern (onsite 3 days a week) Rate: £550 - £600 per day (Inside IR35) Skills: SC Clearance, Python, SQL, Databricks, AWS, Data Engineering, ETL/ELT, Data Pipelines We are looking to recruit an SC Cleared Data Engineer for a leading technology consultancy to join a long-term Central ...

queries using execution plans, cardinality estimates, IO and wait stats Design optimized queries for CPU, IO and memory usage, reducing locking. Checking job schedules, ETL Pipelines and data freshness Build and Maintain ingestion pipelines Enforce constraints, referential integrity and validation rules Support compliance requirements Document data pipelines, schemas, stored procedures ...

and release management. Cloud architecture (Azure/AWS/GCP), Kubernetes/Docker, serverless, IAM and network security. Data engineering depth (Spark/Databricks; ETL/ELT); cloud‐native data + AI architectures. Enterprise integration and SRE principles (SLIs/SLOs, runbooks, rollback). Consulting leadership: stakeholder, budget and risk ...

verbal communication skills Great attention to detail Excellent organisational skills. Strong analytical and problem-solving abilities Experience using Microsoft packages, including Excel Knowledge of ETL tools and data pipelines, SQL, Java and GitHub is required Applicants should be educated to degree level or possess solid work experience in related field. ...

modern cloud-based platform. * Utilising Power BI to create insightful dashboards and reports that empower stakeholders with actionable information for decision-making. * Developing ETL processes to efficiently manage healthcare data across different systems. * Leading proof-of-concept initiatives for new data platforms, demonstrating their benefits and building out robust solutions ...

Microsoft SQL Server database solutions. Expert knowledge of T-SQL, including stored procedures, functions, views and performance optimisation. Strong experience with relational database design, ETL processes (SSIS or equivalent) and high-volume transactional databases. Proven ability to troubleshoot and optimise SQL performance, including query tuning, execution plans, locking and deadlocks. ...

Microsoft SQL Server database solutions. Expert knowledge of T-SQL, including stored procedures, functions, views and performance optimisation. Strong experience with relational database design, ETL processes (SSIS or equivalent) and high-volume transactional databases. Proven ability to troubleshoot and optimise SQL performance, including query tuning, execution plans, locking and deadlocks. ...

procedures and performance tuning. Working knowledge of GCP services such as Cloud Storage, Cloud Functions, Pub/Sub or Cloud Composer. Experience of building ETL or ELT data pipelines. Exposure to orchestration tooling for scheduled and dependency-driven workloads. Familiarity with SQL Server in support of legacy systems. Experience with ...

procedures and performance tuning. Working knowledge of GCP services such as Cloud Storage, Cloud Functions, Pub/Sub or Cloud Composer. Experience of building ETL or ELT data pipelines. Exposure to orchestration tooling for scheduled and dependency‐driven workloads. Familiarity with SQL Server in support of legacy systems. Experience with ...