Senior Technology Architect Databricks
- Hiring Organisation
- Infosys Technologies
- Location
- London, UK
- Employment Type
- Full-time
owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe) and SQL Server. Working knowledge of AWS data services, especially S3. Understanding … data pipelines from legacy mainframe DB2 and on prem SQL Server systems into an AWS-based data lake. Implement real-time ingestion using Confluent Kafka and land data in Amazon S3. Develop data transformations and analytics pipelines in Databricks using the medallion architecture (Bronze, Silver, Gold). Design ...