Data Support Engineer
Sr Data Support Engineer
Senior Data Support Engineer will provide technical leadership for support operations ensuring seamless data pipeline performance and timely resolution of issues to maintain business continuity. The role is responsible for overseeing the health and reliability of data systems, driving root cause analysis, and implementing proactive measures to minimize disruptions. This position combines managerial responsibilities with hands-on technical expertise.
KEY ACCOUNTABILITIES
- Provide technical leadership for team of Production support engineers assigned to data platform
- Monitor and manage the health and performance of Azure Data Factory pipelines, Databricks workflows, and SQL databases to ensure seamless data processing.
- Troubleshoot and resolve production incidents in Azure-based data pipelines, conducting root cause analysis and implementing preventive measures.
- Oversee and optimize the performance of Databricks notebooks and clusters to support efficient data transformations and analytics.
- Ensure the reliability and scalability of data integration workflows by leveraging Azure-native monitoring tools and alerts.
- Collaborate with development teams to deploy and support new Azure Data Factory pipelines, SQL scripts, and Databricks jobs into production.
- Maintain compliance with data governance, security, and backup policies across the Azure platform.
- Lead the implementation of proactive monitoring and logging solutions using Azure Monitor, Log Analytics, and other tools to minimize downtime.
- Coordinate with stakeholders to provide clear updates on production incidents, resolutions, and performance improvements
- Plan and execute disaster recovery and failover strategies for Azure Data Factory, Databricks, and SQL components to ensure business continuity.
- Document operational processes, troubleshooting steps, and best practices for the Azure platform to build a comprehensive knowledge base.