Senior Cloud Data Engineer
LSEG (London Stock Exchange Group) is a world-leading financial markets infrastructure and data business. We are dedicated, open-access partners with a commitment to excellence in delivering services across Data & Analytics , Capital Markets , and Post Trade . Backed by three hundred years of experience, innovative technologies, and a team of over 23,000 people in 70 countries, our purpose is driving financial stability, empowering economies, and enabling customers to create sustainable growth. Role Description : As a Senior Cloud Data Engineer, you’ll design and implement functionalities, focusing on Data Engineering tasks. You’ll be working with semi-structured and non-structured data to ingest and distribute in cloud environments to modernize data products and distribution channels. You’ll drive the software development lifecycle for continuous data delivery and lead the evaluation and adoption of emerging technologies. Key Responsibilities :
- Create and maintain optimal data pipeline architecture,
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Work with stakeholders, including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Depending on seniority relevant experience in Data Platforms (in Financial Services industry), Azure’s PaaS/SaaS offerings (Fabric, Synapse, Purview, ADF, Azure Data Lake Storage etc.)
- Demonstrable experience in a similar role, with a focus on cloud distributed data processing platform for spark.
- Solid experience with Azure: Synapse Analytics, Data Factory, Data Lake, Databricks, Microsoft Purview, Monitor, SQL Database, SQL Managed Instance, Stream Analytics, Cosmos DB, Storage Services, Azure Functions, Log Analytics, Serverless Architecture
- Strong proficiency in Spark, SQL, and Python/scala/Java.
- Knowledge and understanding of Snowflake
- Knowledge of security best practices (e.g., using Azure Key Vault, IAM, RBAC, Monitor etc.).
- Proficient in integrating, transforming, and consolidating data from various structured and unstructured data systems into a structure that is suitable for building analytics solutions.
- Understand the data through exploration, experience with processes related to data retention, validation, visualization, preparation, matching, fragmentation, segmentation, and enhancement.
- Demonstrates ability to understand business requirements and the implications of those requirements on current and future roadmaps.