1 to 25 of 27 Contract PySpark Jobs in England

trials to validate model performance and business impact. Perform data analysis on structured and unstructured datasets using Python, SQL, and big data frameworks (Hadoop, PySpark). Develop and optimize data retrieval systems using RAG, FAISS, Elasticsearch, and other retrieval algorithms. Implement time-series forecasting and anomaly detection for performance ...

Responsibilities Design, develop, and optimize end-to-end data pipelines using Azure services Build and maintain scalable ETL/ELT workflows using Azure Databricks (PySpark/SQL) Develop and manage data warehouses and analytics solutions using Azure Synapse Analytics Ingest data from multiple sources (APIs, databases, files, streaming sources … pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer or in a similar role Strong experience with Azure Databricks (PySpark, Spark SQL) Hands-on experience with Azure Synapse Analytics (dedicated and/or serverless pools) Solid understanding of data warehousing concepts and dimensional modeling ...

performance, efficiency, and cost-effectiveness . Implement data quality checks and validation rules within pipelines. Data Transformation & Processing Implement complex transformations using Spark (PySpark or Scala) and related technologies. Develop logic for data cleaning, enrichment, and aggregation . Ensure data consistency and accuracy throughout the lifecycle. Azure Databricks Implementation … practices . Essential Skills & Experience 10+ years in data engineering, with 3+ years hands-on Azure Databricks experience. Strong proficiency in Python and Spark (PySpark or Scala) . Deep understanding of data warehousing, modelling, and integration patterns . Extensive experience with Azure services (Data Factory, Blob Storage, SQL Database ...

services, data modeling, ETL/ELT development, and collaborative engineering practices. Key Responsibilities * Design, develop, and maintain scalable data pipelines using Azure Databricks (Python, PySpark, SQL). * Build and optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services … standards and best practices. Required Skills and Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

optimising large-scale data pipelines Practical expertise with Databricks and Azure services Confident communicator with strong problem-solving ability Core Technologies Databricks DBT Python PySpark SQL Azure If you are interested in this role then please apply via this platform or email me a copy of your most ...

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/ ...

troubleshooting and incident response for deployed ML systems Required Skills and Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark, or PyTorch Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer Familiarity with cloud platforms (AWS, Azure) and ML services ...

deployed ML systems. Your Profile Essential Skills/Knowledge/Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark or PyTorch. Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer. Familiarity with cloud platforms (AWS, Azure) and ML services (e.g. ...

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Worki click apply for full job details ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred) Click apply now to be considered for the Data Annotator Analyst London role ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

Dataflow Gen2. Design, develop, and implement complex Power BI reports and dashboards, including writing complex DAX formulas and utilising Power Query. Use Python/PySpark for data manipulation, analysis, and automation, alongside advanced T-SQL for database management. Monitor, troubleshoot, and optimise data pipeline and storage performance within … processes using SQL and Azure technologies. Experienced in Power BI development, including complex DAX and Power Query. Strong programming skills in Python/PySpark and advanced T-SQL. Skilled in performance optimisation within Microsoft Fabric. Excellent communication skills, both written and verbal in English, and willingness to travel ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2 … Required Skills & Experience - Senior Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment, ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

junior & Mid Data Engineers on the best practices when building and managing data infrastructure, including data lakes, warehouses, and distributed processing systems (e.g., PySpark, Hadoop). The Role As a Tech Lead , you will play a critical role in designing, building, and maintaining data pipelines and infrastructure that enable … junior & Mid Data Engineers on the best practices when building and managing data infrastructure, including data lakes, warehouses, and distributed processing systems (e.g., PySpark, Hadoop). Data Preparation: Collaborate with data scientists to prepare and transform raw data into formats suitable for machine learning, including feature engineering and data ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format … Ideal Candidate Profile 310+ years of experience in AWS Data Engineering/Support Engineering. Strong telecom domain understanding. Hands-on with Iceberg, Athena, Glue (PySpark), Python, Redshift, S3, ETL frameworks. Strong troubleshooting mindset and ability to work in 247 or rotational support environments (if required). ...

Cleared Python Data Engineer Azure & PySpark Contract | £400£458 per day (Inside IR35) SC Clearance is essential We are looking for an experienced Data Engineer with strong Python skills to join a delivery-focused team working on complex data platforms in Azure. Although this is a data engineering role … work involves developing and maintaining complex, test-driven data flows. You will be responsible for building scalable data pipelines using Python and PySpark, optimizing Delta Lake storage, and deploying solutions in a containerised Azure environment. Quality, maintainability, and automated testing are key to this role. What youll be doing ...

generative AI applications. Perform Exploratory Data Analysis (EDA), data mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using … systems. Required Skills: Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

Job Title: AI Technical Architect Location: London, UK (2 days in a week from office, Hybrid) Job type: 6 Months contract with possible extension Active SC Cleared - Inside IR35 15 years of IT experience currently ...