2 of 2 PySpark Jobs in the North East

services, data modeling, ETL/ELT development, and collaborative engineering practices. Key Responsibilities * Design, develop, and maintain scalable data pipelines using Azure Databricks (Python, PySpark, SQL). * Build and optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services … standards and best practices. Required Skills and Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data ...

play a critical role in supporting a broader initiative to migrate the ETL engine for the data set from Oracle PL/SQL to PySpark on Microsoft Fabric. Key Responsibilities: - Collaborate closely with the existing team to ensure a smooth transition of data during the migration process. - Optimise data … data engineering roles, with a proven track record of data migration projects. - Familiarity with Oracle and Microsoft Fabric environments is essential. - Proficiency in PySpark is required. - Excellent analytical and problem-solving skills, with attention to detail. - Strong communication skills to effectively collaborate with team members and stakeholders. ...