Senior Data Platform Engineer - Compliance Assurance
- Hiring Organisation
- Pontoon
- Location
- Chester, Cheshire, United Kingdom
- Employment Type
- Contract
developing data engineering solutions using Python and PySpark. Practical experience with Apache Kafka and Apache NiFi for streaming, ingestion and orchestration. Experience working with Hadoop, Hive or comparable enterprise data-lake technologies. Solid knowledge of SQL, data transformation, data modelling and large-scale data processing. Experience integrating REST APIs … Apache NiFi. Integrate vendor platforms and enterprise systems through REST APIs and secure authentication mechanisms. Build data processing and storage solutions using Spark, Hadoop and Hive. Develop automated reconciliation, data quality and compliance controls to identify missing, duplicate or inconsistent records. Implement robust data flows with retry mechanisms, error ...