Senior Data Engineer
Job summary
This is an exciting opportunity to join a leading health data engineering team, based at GSTT, but operating across the wider region as part of the AI Centre for Value-Based Healthcare.
We are looking for a motivated individual with excellent data engineering skills, with enthusiasm and passion to learn.
The successful applicant will use a professional tool stack (including Snowflake, SQL, dbt, Python, and Natural Language Processing/Language AI tools) for building and orchestrating data pipelines, from source systems to analyst ready marts which will be used in clinical analytics, research, population health, and to support Real-World Data functions.
The postholder will work closely with a range of key stakeholders across GSTT and other NHS Trusts (including Lewisham and Greenwich, King's College Hospital, and others).
Essential Criteria
- Proficient in SQL for data engineering and analytics
- Experience of using dbt
- Experience of EHR data and its meaning in clinical contexts
-Experience with best development practices, CI, and tools such as git
Desirable Criteria
- Experience in Snowflake or other cloud platforms
- Proficiency in Python
- Experience of NLP / LLM usage in production settings
- Experience in data pipeline orchestration
Main duties of the job
In this role, you will be expected to develop modern data pipelines built to production standards (using Snowflake / SQL / dbt / Python) to harmonise data from highly complex transactional data, including from Cerner and Epic systems. You will be responsible for owning pipeline functions, and regularly engaging with data product end-users to update and improve functionality. Understanding of clinical data quality, best practices in testing and monitoring, and ability to build visualisation applications, are all highly desirable.
Your work will be the foundation for a wide variety of valuable use-cases, from clinical care to research, and from audit to innovation and AI projects.
You will have the opportunity to deploy Large Language Model and NLP tooling orchestration frameworks as part of standard data transformation pipelines, for extracting information from unstructured free text.
You will be supported in continuing your personal development within a friendly and highly expert team, particular in areas such as data orchestration, cloud infrastructure, and continuous integration/deployment. There will also be opportunity to mentor and supervise more junior technical staff.
About us
The AI, Data & Digital Innovation directorate is made up of data and technology experts - based in GSTT but working closely as a team with KCH and KCL.
The team forms part of the Artificial Intelligence Centre for Value-Based Healthcare - a consortium of NHS, academic, and industry partners from across the UK. This consortium offers expert professional technical delivery across data engineering, data science & AI development, and software engineering. Programmes include region-wide infrastructure delivery of cloud and federated platforms, multi-modal Real-World Data engineering, foundation model development, and development of different Language AI solutions.
Job description
Job responsibilities
- Developing modern data pipelines built to production standards, that harmonises data from highly complex transactional data, including from Cerner and Epic systems
- Designing and building data products that serve frequent end-user needs across clinical analytics, research, and population health
- Develop pipelines that drive value for a wide range of end-user stakeholders, including clinicians, researchers, delivery teams, and population health analysts
- To use expertise in data modelling and health data to build deep insights into health data quality, including implementing tests and real-time monitoring, and producing impactful visualisations and reports
- Lead collaborations with multi-disciplinary teams to discover and extract data from new sources
- Co-ordinate and support local analysts to ensure alignment on projects and timelines
- Develop, deploy, and maintain Large Language Model and NLP tooling orchestration frameworks as part of standard data transformation pipelines, for extracting information from unstructured free text
- Contribute to a culture of shared learning, and support capability building around engineering and analytics within the team
- To use understanding of health data to act as subject matter expert for other teams
- Help to scale exemplar product solutions across other NHS Trusts
- Conduct stakeholder presentations and help to produce materials that support engagement across the Trust and the wider region
Person Specification
Education, Qualifications, and Training
- Educated to degree level, or equivalent experience of working in a complex organisation
- Evidence of continuing professional development
- Clinical experience
- Technical degree (e.g. computer / data science)
Previous Experience
- Extensive experience in technical roles involving health data engineering
- Highly developed understanding of NHS
- Extensive experience working in a delivery setting, including conducting end-user engagement
- Strong track record of effective written and spoken communication
- Extensive experience working in large complex organisations, including in a health data or health technology environment
- Knowledge and understanding of risk management and mitigation strategies in technical projects
- Knowledge of EHR systems and processes within healthcare
- Previous experience working in, or collaborating with, academia in high impact and multi-institutional settings
- Knowledge of sprint methodologies
Technical Skills
- Substantial Medical/healthcare data experience
- Expert SQL proficiency
- Proficiency in dbt
- Exceptional analytical, problem-solving, and judgement skills
- Detailed understanding of reproducible code and software development / CI/CD practices
- Proficiency with Snowflake
- Proficiency with Python
- Epic / Cerner data experience
- Skills in advanced data visualisation / presentation / frontend tools
Disclosure and Barring Service Check
This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.
Certificate of Sponsorship
Applications from job seekers who require current Skilled worker sponsorship to work in the UK are welcome and will be considered alongside all other applications. For further information visit the UK Visas and Immigration website.
From 6 April 2017, skilled worker applicants, applying for entry clearance into the UK, have had to present a criminal record certificate from each country they have resided continuously or cumulatively for 12 months or more in the past 10 years. Adult dependants (over 18 years old) are also subject to this requirement. Guidance can be found here Criminal records checks for overseas applicants.
Employer details
Employer name
Guy's and St Thomas' NHS Foundation Trust
Address
London Institute for Healthcare Engineering
100 Lambeth Palace Rd
London
SE1 7AR
Employer's website
https://www.guysandstthomas.nhs.uk/careers/careers.aspx