1 to 25 of 32 Contract PySpark Jobs in the UK

complex Power BI reports and dashboards. Proficiency in writing complex DAX formulas and strong understanding of Power Query. Programming Skills: Requires strong Python/PySpark for data manipulation, analysis, and automation, alongside advanced T-SQL. Performance Optimisation: Experience of monitoring, troubleshooting, and optimising data pipeline and storage performance within ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure ...

Lead the migration of existing data pipelines and workloads from Azure Synapse into Microsoft Fabric Build and optimise ETL pipelines using Python, SQL, and PySpark within the Fabric environment Work across OneLake, Data Factory, Power BI, and Microsoft Purview to support end-to-end data workflows Assist with ...

trials to validate model performance and business impact. Perform data analysis on structured and unstructured datasets using Python, SQL, and big data frameworks (Hadoop, PySpark). Develop and optimize data retrieval systems using RAG, FAISS, Elasticsearch, and other retrieval algorithms. Implement time-series forecasting and anomaly detection for performance ...

Responsibilities Design, develop, and optimize end-to-end data pipelines using Azure services Build and maintain scalable ETL/ELT workflows using Azure Databricks (PySpark/SQL) Develop and manage data warehouses and analytics solutions using Azure Synapse Analytics Ingest data from multiple sources (APIs, databases, files, streaming sources … pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer or in a similar role Strong experience with Azure Databricks (PySpark, Spark SQL) Hands-on experience with Azure Synapse Analytics (dedicated and/or serverless pools) Solid understanding of data warehousing concepts and dimensional modeling ...

performance, efficiency, and cost-effectiveness . Implement data quality checks and validation rules within pipelines. Data Transformation & Processing Implement complex transformations using Spark (PySpark or Scala) and related technologies. Develop logic for data cleaning, enrichment, and aggregation . Ensure data consistency and accuracy throughout the lifecycle. Azure Databricks Implementation … practices . Essential Skills & Experience 10+ years in data engineering, with 3+ years hands-on Azure Databricks experience. Strong proficiency in Python and Spark (PySpark or Scala) . Deep understanding of data warehousing, modelling, and integration patterns . Extensive experience with Azure services (Data Factory, Blob Storage, SQL Database ...

services, data modeling, ETL/ELT development, and collaborative engineering practices. Key Responsibilities * Design, develop, and maintain scalable data pipelines using Azure Databricks (Python, PySpark, SQL). * Build and optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services … standards and best practices. Required Skills and Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

optimising large-scale data pipelines Practical expertise with Databricks and Azure services Confident communicator with strong problem-solving ability Core Technologies Databricks DBT Python PySpark SQL Azure If you are interested in this role then please apply via this platform or email me a copy of your most ...

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/ ...

reservations, Event Hub TUs, serverless compute clusters, and Terraform deployments Required Skills * Azure Databricks * Git/GitHub * DevOps & Cloud Operations * Azure Functions * Data Streaming * PySpark, Python, SQL * Automation & Public Cloud * Cloud Infrastructure * Terraform * Palantir Foundry * PowerBI * Software Lifecycle Management If you meet the requirements please send me a copy ...

reservations, Event Hub TUs, serverless compute clusters, and Terraform deployments Required Skills * Azure Databricks* Git/GitHub* DevOps & Cloud Operations* Azure Functions* Data Streaming* PySpark, Python, SQL* Automation & Public Cloud* Cloud Infrastructure* Terraform* Palantir Foundry* PowerBI* Software Lifecycle Management If you meet the requirements please send me a copy ...

deployed ML systems. Your Profile Essential Skills/Knowledge/Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark or PyTorch. Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer. Familiarity with cloud platforms (AWS, Azure) and ML services (e.g. ...

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Worki click apply for full job details ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred) Click apply now to be considered for the Data Annotator Analyst London role ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

Dataflow Gen2. Design, develop, and implement complex Power BI reports and dashboards, including writing complex DAX formulas and utilising Power Query. Use Python/PySpark for data manipulation, analysis, and automation, alongside advanced T-SQL for database management. Monitor, troubleshoot, and optimise data pipeline and storage performance within … processes using SQL and Azure technologies. Experienced in Power BI development, including complex DAX and Power Query. Strong programming skills in Python/PySpark and advanced T-SQL. Skilled in performance optimisation within Microsoft Fabric. Excellent communication skills, both written and verbal in English, and willingness to travel ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2 … Required Skills & Experience - Senior Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment, ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

junior & Mid Data Engineers on the best practices when building and managing data infrastructure, including data lakes, warehouses, and distributed processing systems (e.g., PySpark, Hadoop). The Role As a Tech Lead , you will play a critical role in designing, building, and maintaining data pipelines and infrastructure that enable … junior & Mid Data Engineers on the best practices when building and managing data infrastructure, including data lakes, warehouses, and distributed processing systems (e.g., PySpark, Hadoop). Data Preparation: Collaborate with data scientists to prepare and transform raw data into formats suitable for machine learning, including feature engineering and data ...