Data Engineer
Reading, England, United Kingdom
Mastek
Lake . Design, implement, and optimize data transformations and treatments for structured and unstructured data. Work with Hive Metastore and Unity Catalog for metadata management and access control. Implement State Store mechanisms for maintaining stateful processing in Spark Structured Streaming . Handle DataFrames efficiently for large-scale data processing … cloud platforms like Azure Databricks, AWS, or Google Cloud . Experience with Databricks Notebooks for development and testing. Familiarity with AWS S3 for data storage and management. Understanding the platform migrations and dependency requirements, challenges and dependency deliverables. Understanding of data privacy principles and ability to implement privacy-aware More ❯
Posted: