Data Engineer
- Hiring Organisation
- Quantum World Technologies Inc
- Location
- Cambridge, Cambridgeshire, UK
- Employment Type
- Full-time
Lake for optimized storage and analytics. Data Governance & Cataloging: Establish data cataloging, lineage, and metadata management for improved discoverability. Performance Optimization: Tune Spark/PySpark jobs for efficiency in large-scale data processing. Data Modelling & Quality: Develop dimensional/data vault models and enforce data quality checks. Collaboration: Work … integrate with Azure/AWS/GCP data ecosystems. Primary Skills (Must-Have): Databricks – Architecture, Delta Lake, Lakehouse, Unity Catalog/Data Catalog PySpark (optimization, UDFs, Delta operations) SQL (advanced querying, performance tuning) Data Lake/Warehouse best practices Secondary Skills (Nice-to-Have): Python (for scripting & automation) Data ...