Azure Data Engineer

Morgan Hunt are working with a housing association in London to recruit an Azure Data Engineer on a permanent basis. The purpose of the role is to lead the implementation of data flows to connect operational systems, data for analytics and BI systems. You will be responsible for designing, implementing, and maintaining the organisations data infrastructure to ensure the efficient processing, storage, and retrieval of large datasets.The role requires a deep understanding of data engineering best practices, data architecture, and the ability to work with cross-functional teams to deliver high-quality data solutions.Key Responsibilities:Design and Development:

• Design, develop and optimise ETL processes using Databricks
• Develop data pipelines to extract, transform, and load into Data lakes and other data storage solutions
• Implement best practices for data management, data governance and data quality assurance

Data Infrastructure:

• Build and maintain scalable data architectures, ensuring data is organised, secure and accessible
• Integrate various data platforms and tools to create a cohesive and robust data ecosystem
• Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability of data services

Collaboration:

• Work closely with data scientists, analysts and other stakeholders to understand data requirements and deliver solutions
• Have a good understanding of the business problems and use cases
• Work with the IT, Digital and DevOps teams to ensure deployment and integration of data solutions

Innovation:

• Stay up to date with the latest trends and advancements in data engineering, data platforms and big data technologies
• Evaluate and improved existing data processes

Essential Experiences:

• Advanced SQL skills
• Proficient in Databricks, including Spark, Delta Lake and MLflow
• Experience migrating SSIS to Databricks
• In depth understanding of ETL/ELT processes
• Knowledge of data warehousing concepts and architectures
• Able to produce relevant data models across multiple subject areas
• Experience implementing data governance, data security and compliance standards to ensure data accuracy

£70,000 per yearHybrid working (2 days in London office)Permanent positionPlease get in touch for further information.

Morgan Hunt is a multi-award-winning recruitment business for interim, contract and temporary recruitment and acts as an Employment Agency in relation to permanent vacancies. Morgan Hunt is an equal opportunities employer. Job suitability is assessed on merit in accordance with the individual's skills, qualifications and abilities to perform the relevant duties required in a particular role.