2 of 2 Contract Parquet Jobs

scalable data pipelines using PySpark 3/4 and Python 3. * Contribute to the creation of a unified data lake following medallion architecture principles. * Leverage Databricks and Delta Lake (Parquet format) for efficient, reliable data processing. * Apply BDD testing practices using Python Behave and ensure code quality with Python Coverage. * Collaborate with cross-functional teams and participate in Agile More ❯

Job Description AWS Stack, data being landed in S3, Lambda triggers, Data Quality, data written back out to AWS S3(Parquet Formats), Snowflake for dimensional model. Design and build the data pipelines, work with someone around understanding data transformation, this is supported by BA's, building out the data pipelines, moving into layers in the data architecture (Medallion architecture More ❯