Lead Data Engineer
Leeds, England, United Kingdom
HCLTech
need good experience with Pyspark, Python, SQL, ADF and preferably Databricks experience Job description: Building new data pipelines and optimizing data flows using the Azure cloud stack. Building data products from scratch. Support Business Analysts and Data Architects with discovery and best practices. Self-directed and comfortable supporting the … data needs of multiple teams, systems and products. What’s required ? Advanced Azure knowledge and experience working and migrating data products from on-prem to Azure. Experience building and optimizing ‘big data' data pipelines, architectures and data sets using Py-spark. Experience building real-time data pipelines using Event … hub, storage queues and Azure stream analysis. Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management. Cloud Big Data Analytics in Azure Synapse Analytics, Azure Analysis Services Data Ingestion and Storage including more »
Posted: