2 of 2 Spark SQL Jobs in South Yorkshire

Your Profile Essential Skills/Knowledge/Experience Strong working and technical knowledge of GCP (GCS Buckets, BigQuery, Dataproc, Pub/Sub), Airflow, Spark, PySpark, Spark SQL, Hive. Good knowledge of Python (Pandas, NumPy, Wiresafe), SQL, Linux CLI, BigQuery. Experience with DBMS … Delta Tables, Oracle, MS Access, SQL Server. Skills in Data Modelling, ETL/ELT Data Pipelines, Python scripting, Data Warehousing. Desirable Skills/Knowledge/Experience Experience in production support, troubleshooting data pipeline issues, and ensuring timely and accurate business reporting. Expertise in Data Modelling, ETL/ ...

domain experience is beneficial but not essential. Key Responsibilities Design and deliver scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Build and maintain ETL/ELT processes and data quality frameworks Develop consistent data models and analytics-ready datasets Partner with stakeholders to deliver … years' experience in data engineering within enterprise environments Strong expertise in Azure Databricks, ADF, Synapse, and Azure Data Lake Hands-on experience with Spark (PySpark/Spark SQL) and orchestration Strong data modelling and data warehousing knowledge Proficiency in SQL ...