7 of 7 Python Data Engineer Jobs in the UK

Python Data Engineer Azure & PySpark - SC Cleared Contract £400-£458pd (Inside IR35) SC Clearance is Essential Summary Were looking for a Python Data Engineer skilled in PySpark, Delta Lake, Azure services, containerized development, and Behave-based testing. Youll design and build scalable data pipelines and maintain high-quality, test-driven code in a … cloud environment. What youll do Build and maintain Python/PySpark pipelines for data ingestion, processing, and validation. Write unit and BDD tests using Behave, including mocking and patching. Create and optimize Delta Lake tables for reliable, performant data storage. Use Docker to manage consistent development, testing, and deployment environments. Build configurable, parameter-driven code for modular … data solutions. Work with Azure Functions, Key Vault, and Blob Storage for cloud-based workflows. Collaborate with architects, data scientists, and DevOps on CI/CD and deployment. Tune and troubleshoot Spark jobs in production. Document solutions and follow cloud security and governance best practices. Skills you need Strong Python skills with a focus on clean, test More ❯

Data Engineer - Security Clearance (SC), Python, Azure, BDD Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who require a Security Cleared Data Engineer, with Active Security Clearance (SC), and strong Python skills to design and deploy scalable Data solutions in a containerized Azure … environment. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong Python skills with modular, test-driven design Experience with Behave for unit and BDD testing (mocking, patching) Proficiency in PySpark and distributed Data processing Solid understanding of Delta Lake (design and maintenance) Hands-on with Docker for development and deployment Familiarity … CI/CD pipelines (ideally Azure DevOps) and cloud security best practices Strong collaboration and communication skills Nice to have: Immediate availability Experience with Databricks or Synapse Knowledge of Data governance in Azure ecosystems Infrastructure as Code (IaC) tooling If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV More ❯

Data Engineer - Security Clearance (SC), Python, Azure, BDD Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who require a Security Cleared Data Engineer, with Active Security Clearance (SC), and strong Python skills to design and deploy scalable Data solutions in a containerized Azure … environment. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong Python skills with modular, test-driven design Experience with Behave for unit and BDD testing (mocking, patching) Proficiency in PySpark and distributed Data processing Solid understanding of Delta Lake (design and maintenance) Hands-on with Docker for development and deployment Familiarity … CI/CD pipelines (ideally Azure DevOps) and cloud security best practices Strong collaboration and communication skills Nice to have: Immediate availability Experience with Databricks or Synapse Knowledge of Data governance in Azure ecosystems Infrastructure as Code (IaC) tooling If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV More ❯

Data Engineer (Python) - Leading Asset Manager | London (Hybrid) Brand-new team. High autonomy. Big impact and high bonus! A top Asset Manager is building a new data function from scratch , and they need a sharp Data Engineer to help define and deliver their future data strategy. You'll work directly with senior stakeholders … environment. You will be joining a small but growing team! Hybrid: 3 days in London office, 2 days WFH (flexible when needed). The Role Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations … and high-quality data validation Translate business needs into technical solutions with full ownership What You Need STEM degree (Computer Science, Engineering, Maths, etc.) 2+ years in data engineering/analytics/software Strong Python + SQL 2+ years ETL, APIs, CI/CD, Docker, AWS Ideally but not essential experience with Airflow/Prefect/Dagster More ❯

About the RoleWe are looking for a Python Data Engineer with strong hands-on experience in Behave-based unit testing, PySpark development, Delta Lake optimisation, and Azure cloud services. This role focusses on designing and deploying scalable data processing solutions in a containerised environment, emphasising maintainable, configurable, and test-driven code delivery.Key Responsibilities Develop and maintain … data ingestion, transformation, and validation pipelines using Python and PySpark. Implement unit and behavior-driven testing with Behave, ensuring robust mocking and patching of dependencies. Design and maintain Delta Lake tables for optimised query performance, ACID compliance, and incremental data loads. Build and manage containerised environments using Docker for consistent development, testing, and deployment. Develop configurable, parameter … driven codebases to support modular and reusable data solutions. Integrate Azure services, including: Azure Functions for serverless transformation logic Azure Key Vault for secure credential management Azure Blob Storage for data lake operations What We're Looking For Proven experience in Python, PySpark, and Delta Lake. SC Cleared Strong knowledge of Behave for test-driven development. Experience More ❯

About the RoleWe are looking for a Python Data Engineer with strong hands-on experience in Behave-based unit testing, PySpark development, Delta Lake optimisation, and Azure cloud services. This role focusses on designing and deploying scalable data processing solutions in a containerised environment, emphasising maintainable, configurable, and test-driven code delivery. Key Responsibilities Develop and … maintain data ingestion, transformation, and validation pipelines using Python and PySpark. Implement unit and behavior-driven testing with Behave, ensuring robust mocking and patching of dependencies. Design and maintain Delta Lake tables for optimised query performance, ACID compliance, and incremental data loads. Build and manage containerised environments using Docker for consistent development, testing, and deployment. Develop configurable … parameter-driven codebases to support modular and reusable data solutions. Integrate Azure services, including: Azure Functions for serverless transformation logic Azure Key Vault for secure credential management Azure Blob Storage for data lake operations What We're Looking For Proven experience in Python, PySpark, and Delta Lake. SC Cleared Strong knowledge of Behave for test-driven development. More ❯

At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for jobseekers and companies around the world to find a perfect match in fair and equitable way. With over 20 brands across 30+ countries, we strive for fair and unbiased hiring. Join our … platform. Job Description Join our team and youll be responsible for working with one of our scrum teams to help us keep our customers happy, solve the challenges of data warehousing in a rapidly growing platform whilst in an evolving marketplace. Working in the commercial data solutions team, we are making significant investment in business intelligence, data products and data warehousing at scale; and we will invest in you. This is so important to us. By joining our team, you will be playing a vital role as together we reimagine the labour market to make it work for everybody. Start your day with a stand up, then spin up your environment and bham, you're More ❯