Data Engineer
- Hiring Organisation
- HCLTech
- Location
- London Area, United Kingdom
Implement workflow orchestration (Airflow/Cloud Composer) including retries, dependencies, and monitoring. Ensure data quality, validation, and governance mechanisms. Design and manage streaming pipelines (Kafka/Pub-Sub) for near real-time data processing. Optimize data pipelines for performance, scalability, and cost efficiency. Monitor, troubleshoot, and enhance pipeline reliability ...