Cloud Data Platform Engineer
- Hiring Organisation
- Capgemini
- Location
- Glasgow, Scotland, United Kingdom
validation (e.g., schema tests, data quality checks) Manage data documentation and lineage within DBT Design and develop scalable data pipelines using Databricks and Apache Spark Build and optimize data workflows for batch and real-time processing Transform and process large datasets using PySpark, SQL, and Delta Lake Implement data … terraform enterprise , Snowflake terraform provider knowledge. Snowflake : Snow sight , Python SDK for snowflake , Stage , storage integration , Data marketplace , direct shares , listings , catalog linked database , Apache Iceberg , external tables , Catalog integrations , Snowpipe , external volume , Cortex ai ,privatelink. Airflow : Dag creation , operators , connections , deployment on Kubernetes , Astronomer. ...