2 of 2 Permanent Spark SQL Jobs in East London

Pipeline Development Design and maintain ETL pipelines for diverse data sources (APIs, databases, file systems). Ensure reliability, scalability, and performance. Data Transformation & Processing Implement data transformation using Spark (PySpark/Scala) and related tools. Conduct data cleaning, validation, and enrichment. Azure Databricks Implementation Work with Unity Catalog, Delta Lake, Spark SQL . Optimize … and follow best practices in Databricks development. Program using SQL, Python, R, YAML, JavaScript . Data Integration Integrate from various structured/unstructured sources (CSV, JSON, Parquet, Delta). Enable downstream access for dashboards and reporting. Data Quality & Monitoring Use Azure Purview for governance and quality. Develop and maintain monitoring and validation processes. Collaboration & Communication Work closely with … workflows. Support automation across engineering tasks. Essential Skills & Experience: 5+ years in data engineering or related roles. Strong grasp of data engineering concepts and principles. Proficiency in Python, SQL, and ideally Spark (PySpark). Experience with Azure Databricks, Delta Lake , and data architecture . Familiarity with Azure cloud , version control (e.g., Git), and DevOps pipelines . More ❯

AWS, Snowflake, etc. Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong communicator and team player More ❯