Data Engineer

JOB PURPOSE AND SCOPE  
The Data Engineer is responsible for designing, building, and maintaining our data infrastructure, ensuring the efficient and reliable processing, storage, and analysis of large datasets.
 The role will develop and deploy ETL processes from various systems to enhance the data capabilities of the force.
 They will contribute to ensuring our data, and data shared by others is of the highest quality, available and usable.
 The role requires a good level of knowledge, functional understanding of modern data technologies and platforms.
They will cultivate strong and effective working relationships with individuals, teams and colleagues, both internal and external to the Force. In particular they will work with performance, information and data analysts to build products and services which deliver insight and value to the Force and its ability to make Lincolnshire a safe place to live, work and visit.
 Demonstrate the Force values at all times. All staff involved in carrying out functions in this role will do so in accordance with the principles of the Code of Ethics. The aim of the Code of Ethics is to support each member of the policing profession to deliver the highest professional standards in their service to the public.
 
CORE WORK AREAS

1. Design and implement scalable, efficient, and robust data pipelines to extract, transform, and load (ETL) data from various sources into databases.
2. Build and link data sets so that the Force can easily derive insights from its data.
3. Write ETL scripts to make sure the ETL process performs optimally, leading on extracting data, joining and helping analysts to derive insights.
4. Troubleshooting and resolving issues as they arise, to ensure the timely delivery of high- quality data.
5. Identify and automate manual data flows and reports to enable scaling and repeatable use, working with colleagues across the organisation.
6. Engage and collaborate with teams across the Force, to understand their data requirements and translate them into technical solutions.
7. Work closely with data stakeholders to provide support and guidance on how to use the data engineering infrastructure effectively.
8. Work with information analysts and BI developers to build optimal and reusable visualisations and reports
9. Extract, clean, combine and store data in a way that allows it to be exploited and as an asset to help service delivery and decision making.
10. Support the Information Management Unit (IMU) colleagues implementing data quality checks, data validation, and data cleansing processes to help ensure the accuracy, consistency, and integrity of the data. This may include purging data in accordance with regulation and internal policy.
11. Adhere to data governance policies and standards, ensuring compliance with data privacy and security regulations.
12. Anonymise data wherever necessary, working within the principles of GDPR and with suitable data protection and sharing agreements in place.
13. Document data engineering processes, workflows, and technical specifications to facilitate knowledge sharing and ensure reproducibility of analyses.
14. Stay up-to-date with emerging trends and advancements in data engineering, evaluating and recommending new tools, techniques, and best practices that will enhance the data engineering infrastructure.
15. Contribute to data strategy, writing good standards and practice for data engineering.