1 of 1 Permanent Delta Lake Jobs in Wigan

Data Engineer

Location
Wigan, England, United Kingdom
detail at interview. What You’ll Be Doing: Designing, building, and optimising streaming and batch pipelines in Databricks Develop robust bronze, silver, and gold delta tables using the Medallion Architecture to serve downstream analytics and APIs Implement and manage integrations from AWS MSK (Kafka), S3, and external APIs into … PySpark, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Declarative Pipelines (DLT) and the Medallion Architecture. A strong AWS background is important, particularly with services such as S3. You’ll need ...