1 to 25 of 53 Remote PySpark Jobs in England

UK. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have a passion for data engineering and a … desire to make a significant impact, we encourage you to apply! Job Responsibilities Data Engineering & Data Pipeline Development Design, develop, and optimize scalable DATA workflows using Python, PySpark, and Airflow Implement real-time and batch data processing using Spark Enforce best practices for data quality, governance, and security throughout the data lifecycle Ensure data availability, reliability and performance through … Implement CI/CD pipelines for data workflows to ensure smooth and reliable deployments. Big Data & Analytics: Build and optimize large-scale data processing pipelines using Apache Spark and PySpark Implement data partitioning, caching, and performance tuning for Spark-based workloads. Work with diverse data formats (structured and unstructured) to support advanced analytics and machine learning initiatives. Workflow Orchestration More ❯

business intelligence, reporting, and regulatory needs Lead the integration and optimisation of large-scale data platforms using Azure Synapse and Databricks Build and maintain robust data pipelines using Python (PySpark) and SQL Collaborate with data engineers, analysts, and stakeholders to ensure data quality, governance, and security Ensure all solutions adhere to financial regulations and internal compliance standards Key Skills … Experience: Proven experience as a Data Architect within the financial services sector Hands-on expertise with Azure Synapse Analytics and Databricks Strong programming and data engineering skills in Python (PySpark) and SQL Solid understanding of financial data and regulatory compliance requirements Excellent stakeholder communication and documentation skills More ❯

We're looking for someone with strong technical expertise and a passion for solving complex business problems. You'll bring: Strong experience with SQL, SQL Server DB, Python, and PySpark Proficiency in Azure Data Factory, Databricks is a must, and Cloudsmith Background in data warehousing and data engineering Solid project management capabilities Outstanding communication skills, translating technical concepts into More ❯

s/PhD in Computer Science, Data Science, Mathematics, or related field. 5+ years of experience in ML modeling, ranking, or recommendation systems . Proficiency in Python, SQL, Spark, PySpark, TensorFlow . Strong knowledge of LLM algorithms and training techniques . Experience deploying models in production environments. Nice to Have: Experience in GenAI/LLMs Familiarity with distributed computing More ❯

team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have a passion for data engineering and a … make a significant impact, we encourage you to apply! Job Responsibilities ETL/ELT Pipeline Development: Design, develop, and optimize efficient and scalable ETL/ELT pipelines using Python, PySpark, and Apache Airflow. Implement batch and real-time data processing solutions using Apache Spark. Ensure data quality, governance, and security throughout the data lifecycle. Cloud Data Engineering: Manage and … and documentation. Required profile: Requirements Client facing role so strong communication and collaboration skills are vital Proven experience in data engineering, with hands-on expertise in Azure Data Services, PySpark, Apache Spark, and Apache Airflow. Strong programming skills in Python and SQL, with the ability to write efficient and maintainable code. Deep understanding of Spark internals, including RDDs, DataFrames More ❯

infrastructure Excellent communication and collaboration skills Experience working with Git, practicing code reviews and branching strategies, CI/CD and testing in software solutions Proficiency in SQL, Python, and PySpark Ability to translate marketing needs into well-structured data products Deep understanding of data modeling concepts and building scalable data marts Basic experience with frontend technologies is a plus More ❯

SQL Server. Understanding of applying master data management principles, data quality frameworks and data governance best practices. Understanding of Azure Data Factory, Fabric and similar technologies Tech Stack – Python, PySpark, SQL, Xpath, XML, Azure-based Data Science tools, BI tools, Data Visualisation, Agile. The company have an excellent reputation within their sector and have shown consistent growth year-on More ❯

SQL Server. Understanding of applying master data management principles, data quality frameworks and data governance best practices. Understanding of Azure Data Factory, Fabric and similar technologies Tech Stack Python, PySpark, SQL, Xpath, XML, Azure-based Data Science tools, BI tools, Data Visualisation, Agile. The company have an excellent reputation within their sector and have shown consistent growth year-on More ❯

Strong analytical and troubleshooting skills. Desirable Skills Familiarity with state management libraries (MobX, Redux). Exposure to financial data or market analytics projects. Experience with data engineering tools (DuckDB, PySpark, etc.). Knowledge of automated testing frameworks (Playwright, Cypress). Experience of WebAssembly. Python programming experience for data manipulation or API development. Use of AI for creating visualisations. Soft More ❯

Data Engineering Manager £110,000-£115,000 + 10% bonus ️Databricks, Snowflake, Terraform, Pyspark, Azure London, hybrid working (2 days in office) Leading property data & risk software company We are partnered with a leading property data & risk software company that contributes valuations, insights, and decisioning technology to over 1 million mortgage approvals each year. They are looking for a … visualization, and data modeling. Engage in projects that influence the company's bottom line. Drive the business forward by enabling better decision-making processes. Tech Stack : Databricks, Azure, Python, Pyspark, Terraform. What's in it for you 7.5% pension contribution by the company Discretionary annual bonus up to 10% of base salary 25 days annual leave plus extra days More ❯

Head of Data Platform and Services, you'll not only maintain and optimize our data infrastructure but also spearhead its evolution. Built predominantly on Databricks, and utilizing technologies like Pyspark and Delta Lake, our infrastructure is designed for scalability, robustness, and efficiency. You'll take charge of developing sophisticated data integrations with various advertising platforms, empowering our teams with … and informed decision-making What you'll be doing for us Leadership in Design and Development : Lead in the architecture, development, and upkeep of our Databricks-based infrastructure, harnessing Pyspark and Delta Lake. CI/CD Pipeline Mastery : Create and manage CI/CD pipelines, ensuring automated deployments and system health monitoring. Advanced Data Integration : Develop sophisticated strategies for … standards. Data-Driven Culture Champion : Advocate for the strategic use of data across the organization. Skills-wise, you'll definitely: Expertise in Apache Spark Advanced proficiency in Python and Pyspark Extensive experience with Databricks Advanced SQL knowledge Proven leadership abilities in data engineering Strong experience in building and managing CI/CD pipelines. Experience in implementing data integrations with More ❯

office 2/3 days in a week) Duration: 6 months Inside IR35 Job description: This will be a Tech Lead who is proficient in developing complex logic using pyspark in AWS along with helping/leading the team. 7+ years of experienced in designing, developing complex logic for data pipelines using pyspark in AWS along with helping …/leading the team. He/she needs to experienced and skilled in PySpark, Glue, Python, SQL and Data processing . This involves designing ETL processes, ensuring data security, and collaborating with other teams for data analysis and business requirements. Skilled in scalable, reliable, and efficient data solutions , often using AWS services like S3, Redshift, EMR, Glue, and Kinesis More ❯

lead in the engineering function of the data team Skills, Knowledge and Expertise Skills: Proficiency in cloud based data engineering tools (ADF, Synapse Analytics, S3, Lamda) Proficiency in using PySpark notebooks for ELT. Fostering and cultivating a culture of best practices Strong analytical and problem-solving skills. Ability to work independently and as part of a functional and cross More ❯

Data Scientist, you will work using data engineering, statistical, and ML/AI approaches to uncover data patterns and build models. We use Microsoft tech stack, including Azure Databricks (Pyspark, python), and we are expanding our data science capabilities. To be successful in the role, you will need to have extensive experience in data science projects and have built More ❯

months Location: London JOB DETAILS Role Title: Senior Data Engineer Note: (Please do not submit the same profiles as for 111721-1) Required Core Skills: Databricks, AWS, Python, Pyspark, data modelling Minimum years of experience: 7 years Job Description: Must have hands-on experience in designing, developing, and maintaining data pipelines and data streams. Must have a strong working … knowledge of moving/transforming data across layers (Bronze, Silver, Gold) using ADF, Python, and PySpark. Must have hands-on experience with PySpark, Python, AWS, data modelling. Must have experience in ETL processes. Must have hands-on experience in Databricks development. Good to have experience in developing and maintaining data integrity and accuracy, data governance, and data security policies More ❯

data engineering and reporting. Including storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You're a problem-solver, pragmatically exploring options and finding effective solutions. An understanding of how to design and build well-structured, maintainable systems. Strong communication skills More ❯

deduplication, adept at transforming raw data into a unified and reliable asset. Projects natural confidence in communication and has strong stakeholder management skills. Has strong proficiency with Pandas, Numpy, PySpark or similar for Data Analysis & Cleaning Python Maintains a solid knowledge of both Automation tools, & other emerging technologies including AI platforms & LLM's. Organised, meticulous & attention to detail orientated More ❯

to performance optimisation and cost efficiency across data solutions. Required Skills & Experience: Proven hands-on experience with Azure Databricks, Data Factory, Delta Lake, and Synapse. Strong proficiency in Python, PySpark, and advanced SQL. Understanding of Lakehouse architecture and medallion data patterns. Familiarity with data governance, lineage, and access control tools. Experience in Agile environments, with solid CI/CD More ❯

security What We're Looking For: Strong experience with AWS Glue , S3, Lambda, and related services Proven track record in data engineering or ETL development Proficiency in Python or PySpark for data processing Familiarity with data lake, data warehouse, and cloud-first environments Comfortable working in agile teams and hybrid work settings Location: Hybrid - 2-3 days a week More ❯

Knowledge of the technical differences between different packages for some of these model types would be an advantage. Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar software is preferred Proficient at communicating results in a concise More ❯

Knowledge of the technical differences between different packages for some of these model types would be an advantage. Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW's Radar software is preferred Proficient at communicating results in a concise More ❯

inclusive, forward-thinking culture, and help drive the energy transition for the UK. Code & create - Develop complex SQL and ABAP CDS views for analytics and reporting Transform & optimise - Use PySpark and Databricks to manipulate big data efficiently Automate & schedule - Manage workflows, jobs and clusters for scalable data processing Collaborate & deliver - Engage across agile teams to build high-impact solutions … Experience in building data pipelines and models in SAP Datasphere or SAP BW4/Hana Advanced skills in SQL, data modelling, and data transformation Familiarity with Databricks, Apache Spark, PySpark, and Delta Lake Agile mindset with experience in DevOps and iterative delivery Excellent communication and stakeholder engagement abilities Sound like a fit? Let's build the future of data More ❯

and value creation from data curation activities. Agile mindset with the ability to deliver prototypes quickly and iterate improvements based on stakeholder feedback Experience in Python, Databricks, Delta Lake, PySpark, Pandas, other data engineering frameworks and applying them to achieve industry standards-compliant datasets Strong communication skills and expertise to translate business needs into technical data requirements and processes More ❯

ownership, lineage, sensitivity and definitions. Ensure compliance with GDPR and other data regulations when handling sensitive information. Support the stability and performance of enterprise data platforms. Requirements: Proficient with PySpark, Delta Lake, Unity Catalog and Python (including unit and integration testing). Deep understanding of software development principles (SOLID, testing, CI/CD, version control). Strong knowledge of More ❯

our machine learning and analytics workloads to support the companies growth. Our data stack: We work with a modern data stack built on Databricks and AWS with python and pyspark as our primary tools. In this role, you'll get to: Own business critical components and perform meaningful work with an impact on our company and our customers Design … expand your skillset About you We believe that no one is the finished article, however, some experience in the following is important for this role: Proficient with Python and PySpark Experience working with a modern data stack is beneficial but not required Experience with AWS is beneficial but not required You enjoy learning new technologies and are passionate about More ❯