3 of 3 PySpark Jobs in East London

treating data pipelines as production-grade software. A significant portion of the work involves designing and maintaining complex, test-driven Python data flows, with PySpark used as the execution engine rather than the primary focus. Strong Python fundamentals, testing discipline, and code quality are critical to success in this … strong emphasis on clean architecture, reusable Python modules, and testability Writing comprehensive unit tests and BDD tests (Behave), including mocking and patching Using PySpark to process large-scale datasets while keeping business logic in Python Creating, maintaining, and optimising Delta Lake tables for performance and reliability Building and running ...

Cleared Python Data Engineer Azure & PySpark Contract | £400£458 per day (Inside IR35) SC Clearance is essential We are looking for an experienced Data Engineer with strong Python skills to join a delivery-focused team working on complex data platforms in Azure. Although this is a data engineering role … work involves developing and maintaining complex, test-driven data flows. You will be responsible for building scalable data pipelines using Python and PySpark, optimizing Delta Lake storage, and deploying solutions in a containerised Azure environment. Quality, maintainability, and automated testing are key to this role. What youll be doing ...

building common functionality for distributed product teams. Early work will focus on: Abstracting existing codebases into shared Python libraries Developing data-driven solutions using PySpark and DataFrames Building and extending Python-based microservices using Azure Functions Creating scalable, reusable services that solve common challenges across teams Beyond the initial … essential, as youll be working closely with engineers and technical leads across five time zones . Essential skills Strong commercial experience with Python PySpark and data-frame based processing Solid SQL capability Experience working with Azure infrastructure Good understanding of containers, microservices, and functional design patterns Comfortable working ...