Data Engineer - DV Cleared
- Location
- City Of London, England, United Kingdom
models to production and implement measures to monitor their performance Implement ETL pipelines and orchestrate data flows using batch and streaming technologies based on software engineering best practice Define, document and iterate data mappings based on concepts and principles of data modelling Re-engineer data pipelines … Data warehousing and lake architectures o ETL/ELT pipeline development o SQL and NoSQL databases o Distributed computing frameworks (Spark, Kinesis etc) o Software development best practices including CI/CD, TDD and version control. o Containerisation tools like Docker or Kubernetes o Experience with Infrastructure as Code ...