1 to 25 of 32 PySpark Jobs in the Thames Valley

Spark Programming Ability to write Spark code for large scale data processing, including RDDs, Data Frames, and Spark SQL Python/Notebook programming PySpark programming Spark Streaming/batch processing Delta table Optimization Fabric spark jobs Java programming language, OOP knowledge Database knowledge, including Relational Database ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services (AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage). Exposure ...

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Commercial experience with Microsoft Fabric would be advantageous Some of the package/role details include: Salary ...

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Commercial experience with Microsoft Fabric would be advantageous Some of the package/role details include: Salary ...

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Commercial experience with Microsoft Fabric would be advantageous Some of the package/role details include: Salary ...

Considerable experience designing and building operationally efficient pipelines, utilising core Cloud components, such as Azure Data Factory, Big Query, AirFlow, Google Cloud Composer and Pyspark etc Proven experience in modelling data through a medallion-based architecture, with curated dimensional models in the gold layer built for analytical use Strong ...

overview of many different sectors. What We're Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience ...

overview of many different sectors. What We're Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout … years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data ...

Looking For: Proven hands-on experience in data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like ...

applied data science. Hands-on experience with cloud data platforms, including AWS, Azure, GCP, and/or Databricks. Proficiency in Python (and ideally PySpark); experience with R beneficial but not essential. Experience building end-to-end data pipelines, feature engineering, and production ML workflows. Exposure to Generative AI, LLMs ...

applied data science. Hands-on experience with cloud data platforms, including AWS, Azure, GCP, and/or Databricks. Proficiency in Python (and ideally PySpark); experience with R beneficial but not essential. Experience building end-to-end data pipelines, feature engineering, and production ML workflows. Exposure to Generative AI, LLMs ...

applied data science. Hands-on experience with cloud data platforms, including AWS, Azure, GCP, and/or Databricks. Proficiency in Python (and ideally PySpark); experience with R beneficial but not essential. Experience building end-to-end data pipelines, feature engineering, and production ML workflows. Exposure to Generative AI, LLMs ...

Salary range: £75,000-£85,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities ...

optimising large-scale data pipelines Practical expertise with Databricks and Azure services Confident communicator with strong problem-solving ability Core Technologies Databricks DBT Python PySpark SQL Azure If you are interested in this role then please apply via this platform or email me a copy of your most ...

organisation ahead of the curve. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Hands-on experience with Databricks and PySpark/Spark SQL. Strong expertise with the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture and performance optimisation. What ...

ground running Comfortable operating in high-pressure, delivery-focused environments Able to explain how you moved data, not just which tools you used Python (PySpark/Pandas/NumPy) and SQL using practically for data transformation Desirable Experience migrating data into Dynamics (or similar ERP platforms) Experience working with ...

troubleshooting and incident response for deployed ML systems Required Skills and Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark, or PyTorch Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer Familiarity with cloud platforms (AWS, Azure) and ML services ...

Title: Azure Databricks Data Engineer Primary skills: Advanced SQL, Azure Databricks, Azure Data Factory, Azure Datalake. Secondary skills: Azure SQL, PySpark, Azure Synapse. Experience: 10+ Years of Experience Hybrid - 3 Days/Week onsite is MUST About the job We are looking for an experienced Databricks Data Engineer … ingestion and transformation. • Collaborate with Azure stack modules like Data Lakes and SQL DW to handle large volumes of data. • Write SQL, Python, and PySpark code to meet data processing and transformation needs. • Understand business requirements and create data flow processes that meet them. • Develop mapping documents and transformation ...

What We're Looking For Strong data modelling expertise and experience with Data Lineage Maps. Hands-on skills with SQL/Python, Synapse Notebooks, PySpark, and Delta Lake. Familiarity with MS Fabric and medallion architecture. Excellent communication and stakeholder management skills. Agile mindset and ability to thrive in cross ...

processing, and analytics • Designing scalable, secure, and high performance architectures using Databricks and distributed data systems • Building and overseeing robust data ingestion pipelines using PySpark, with a strong focus on reliability and accuracy • Ensuring end to end data quality from raw ingestion through to curated datasets used for reporting … wider regulatory requirements • Evaluating and introducing new technologies where they add clear value to the data ecosystem What you will bring • Deep expertise in PySpark and distributed data processing at scale • Strong hands on experience designing, building, and optimising Databricks based platforms • Advanced SQL skills including performance optimisation ...