21 of 21 Remote/Hybrid PySpark Jobs in London

Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes of Python, Pyspark, SQL, Scala or Java. Experience operating in a cloud-native environment such as Azure, AWS, GCP etc ( Fabric experience would be beneficial ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology ...

Responsibilities Design, develop, and optimize end-to-end data pipelines using Azure services Build and maintain scalable ETL/ELT workflows using Azure Databricks (PySpark/SQL) Develop and manage data warehouses and analytics solutions using Azure Synapse Analytics Ingest data from multiple sources (APIs, databases, files, streaming sources … pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer or in a similar role Strong experience with Azure Databricks (PySpark, Spark SQL) Hands-on experience with Azure Synapse Analytics (dedicated and/or serverless pools) Solid understanding of data warehousing concepts and dimensional modeling ...

agentic capabilities, able to build production-grade applications not just prototypes. Strong proficiency in at least one relevant programming language such as Python or PySpark, Java, Typescript or SQL. Solid understanding of data engineering, data modelling, security and governance in enterprise settings. Experience with software engineering best practices including ...

performance, efficiency, and cost-effectiveness . Implement data quality checks and validation rules within pipelines. Data Transformation & Processing Implement complex transformations using Spark (PySpark or Scala) and related technologies. Develop logic for data cleaning, enrichment, and aggregation . Ensure data consistency and accuracy throughout the lifecycle. Azure Databricks Implementation … practices . Essential Skills & Experience 10+ years in data engineering, with 3+ years hands-on Azure Databricks experience. Strong proficiency in Python and Spark (PySpark or Scala) . Deep understanding of data warehousing, modelling, and integration patterns . Extensive experience with Azure services (Data Factory, Blob Storage, SQL Database ...

build skills and knowledge relevant to the role. Essential Skills Experience of using a unified engine for large-scale data analytics (e.g. Spark/PySpark) Experience in writing, testing and implementing scripts (e.g. Python, Scala) Experience of cloud data stack use (e.g. SageMaker Notebooks, S3, Glue, Athena) Communicating technical ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

trading and risk management. Document system architecture, data flows, and technical processes for transparency and reproducibility. Requirements Strong proficiency in Python (pandas, NumPy, PySpark) and ETL development. Hands-on experience with AWS services (S3, Glue, Lambda) and Databricks. Solid understanding of financial market data, particularly time-series. Knowledge ...

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/ ...

troubleshooting and incident response for deployed ML systems Required Skills and Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark, or PyTorch Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer Familiarity with cloud platforms (AWS, Azure) and ML services ...

troubleshooting and incident response for deployed ML systems Required Skills and Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark, or PyTorch Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer Familiarity with cloud platforms (AWS, Azure) and ML services ...

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

solutions. Hands-on knowledge of Palantir Foundry (Ontology, pipelines, apps, AIP); Palantir certifications desirable. Proficiency in at least one coding language (e.g., Python/PySpark, R, Java, SQL, Typescript) and experience with object-oriented design or software engineering. Strong understanding of data engineering, modelling, platform integration, UX design ...

Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred ...

across modern BI platforms, while supporting decision-making in Financial Advice and Wealth Management. Key Responsibilities Build and deploy data models and reports using PySpark, Microsoft Fabric or Databricks, and Power BI Apply relational data modelling and data warehousing techniques (Kimball methodology or similar) Manage version control and collaboration … business, particularly in Financial Advice and Wealth Management Requirements 3-5 years' experience in relational data modelling and data warehousing Strong experience with PySpark, Power BI, Microsoft Fabric/Databricks Knowledge of Git and CI/CD pipelines Understanding of the UK Financial Advice & Wealth Management markets highly advantageous ...

Practice Lead – Defence Data Transformation Location: London/Hybrid Salary: £90,000 – £110,000 + Bonus & Benefits Palantir Foundry | Python | PySpark | Ontology | Data Engineering*Eligibility for clearance is required Join a £20m Programme Driving the Future of Defence Data Our client has just secured a £20 million programme … Stakeholders —translate technical concepts into operational impact. What We’re Looking For Essential: Palantir Foundry experience, ontology modelling, Agile leadership. Tech Stack: Python, SQL, PySpark, strong data engineering principles. Bonus Skills: Graph technologies (RDF, OWL, SPARQL), cloud platforms (AWS/Azure/GCP), integration tools (Airflow, Kafka, Elasticsearch). ...