201 to 218 of 218 PySpark Jobs in England

that powers analytics and AI-driven solutions. What You'll Do Build and maintain scalable data pipelines in a cloud environment. Develop transformations using PySpark, Python, and SQL. Model data for analytics and reporting. Ensure data quality and performance. Opportunity to engage with stakeholders and challenge external partners. What … Looking For Solid foundations in data engineering with experience in cloud-based platforms. Hands-on experience with Databricks or Fabric. Strong SQL and PySpark skills. Understanding of ETL and dimensional modelling. Excellent communication and problem-solving mindset. Why Join? Work on modernisation projects including real-time reporting and AI. ...

across modern BI platforms, while supporting decision-making in Financial Advice and Wealth Management. Key Responsibilities Build and deploy data models and reports using PySpark, Microsoft Fabric or Databricks, and Power BI Apply relational data modelling and data warehousing techniques (Kimball methodology or similar) Manage version control and collaboration … business, particularly in Financial Advice and Wealth Management Requirements 3-5 years' experience in relational data modelling and data warehousing Strong experience with PySpark, Power BI, Microsoft Fabric/Databricks Knowledge of Git and CI/CD pipelines Understanding of the UK Financial Advice & Wealth Management markets highly advantageous ...

doing: Building and maintaining a unified data platform for global data processing. Developing reusable and scalable data solutions using Databricks, Python/PySpark, and Azure. Working closely with the data visualisation team to align back-end and front-end needs. Designing secure data access models and integrating external data … Strong understanding of data engineering principles, Lakehouse architecture, ETL/ELT, warehousing, and stream processing. Hands-on experience with Azure Databricks, Python/PySpark, SQL Server, and Azure Blob Storage. A track record of building secure, scalable data pipelines and solving complex problems. Great communication and documentation skills with ...

Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. Leading client workshops and guiding technical … technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. CI/CD expertise with Azure DevOps ...

Practice Lead – Defence Data Transformation Location: London/Hybrid Salary: £90,000 – £110,000 + Bonus & Benefits Palantir Foundry | Python | PySpark | Ontology | Data Engineering*Eligibility for clearance is required Join a £20m Programme Driving the Future of Defence Data Our client has just secured a £20 million programme … Stakeholders —translate technical concepts into operational impact. What We’re Looking For Essential: Palantir Foundry experience, ontology modelling, Agile leadership. Tech Stack: Python, SQL, PySpark, strong data engineering principles. Bonus Skills: Graph technologies (RDF, OWL, SPARQL), cloud platforms (AWS/Azure/GCP), integration tools (Airflow, Kafka, Elasticsearch). ...

Cleared Python Data Engineer Azure & PySpark Contract | £400£458 per day (Inside IR35) SC Clearance is essential We are looking for an experienced Data Engineer with strong Python skills to join a delivery-focused team working on complex data platforms in Azure. Although this is a data engineering role … work involves developing and maintaining complex, test-driven data flows. You will be responsible for building scalable data pipelines using Python and PySpark, optimizing Delta Lake storage, and deploying solutions in a containerised Azure environment. Quality, maintainability, and automated testing are key to this role. What youll be doing ...

generative AI applications. Perform Exploratory Data Analysis (EDA), data mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using … systems. Required Skills: Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

days per week onsite What you'll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you'll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...

days per week onsite What you’ll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you’ll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...

days per week onsite What you’ll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you’ll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...

Pyspark Engineer- (Data Engineering, AWS GLUE) SC Cleared OR EligibleStevenage (Hybrid) 2-3 days onsite Up to £80,000High-impact programme - Revolutionary platform I am looking for a Security-Cleared Pyspark expert to take the reins on a range of highly ambitious Data Migration projects supporting a range … databases and AWS cloud environments. Strong hands-on experience with ETL processes and tools (Talend, Informatica, Matillion, Pentaho, MuleSoft, Boomi) or scripting using Python, PySpark, and SQL. Solid understanding of data warehousing and modelling techniques (Star Schema, Snowflake Schema). Familiarity with security frameworks such as GDPR, HIPAA ...

development and optimisation of scalable data and pipelines in a cloud-based environment, building and managing ETL processes using tools such as Azure, Databricks, PySpark and SQL. The role would suit a strong Data Engineer with 2+ years' commercial experience working with cloud-based data tech (Azure & AWS), Python … relevant languages for ETL), relational and non-relational databases. Key skills required: Proficiency in implementing ETL pipelines Azure/AWS cloud proficiency Python/PySpark Strong SQL skills for database management SSRS/SSIS Ability to develop, optimise & maintain complex SQL queries & stored procedures (T-SQL) The role ...

responsible for: Designing end-to-end data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions … producing professional documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud data ...

price scraping outputs, mystery shopping insights) into practical pricing frameworks Work with Python-based pricing models and support their migration into scalable SQL/PySpark environments Translate complex analysis into clear, commercially relevant insight for non-technical stakeholders Act as a trusted interface between Pricing, Finance, Commercial teams, Category … closely with business stakeholders, not purely in a technical silo Comfort operating with imperfect or noisy data in real-world commercial environments Desirable Python, PySpark, or similar analytical/model-driven tools Exposure to B2B pricing, wholesale, distribution, or multi-site trading environments Understanding of ERP-driven pricing processes ...

Job Title: AI Technical Architect Location: London, UK (2 days in a week from office, Hybrid) Job type: 6 Months contract with possible extension Active SC Cleared - Inside IR35 15 years of IT experience currently ...

effectively. The role You will: Customise and implement product solutions for enterprise clients Own delivery across multiple engagements Work directly with client stakeholders Use PySpark, Python and SQL for data engineering tasks Develop an understanding of the Strategy OS platform Support modelling and transformation work What we're looking … years' experience Proficient in PySpark, Python and SQL Client-facing delivery experience Able to work in a fast-paced environment STEM academics preferred Nice to have: Snowflake, Databricks, dbt, BigQuery Palantir Foundry Experience in startups or niche data consultancies Working pattern Soho office (WeWork) Four days onsite per week ...

solutions. GenAI Implementation: Hands-on deployment of Azure OpenAI, Vector Databases, and RAG patterns. Data Engineering: Oversee large-scale data processing using Python and PySpark to support AI model fine-tuning. Leadership: Manage technical delivery teams and deliver upskilling sessions to the customer. Stakeholder Management: Gain consensus on complex … roadmaps with diverse client groups. Technical Skills & Experience Core AI: Azure OpenAI, Prompt Engineering, Cognitive Search, Form Recognizer. Data/Coding: Expert-level Python, PySpark, and Vector DB integration. Ops: Practical knowledge of MLOps and LLMOps lifecycles. Security: Must hold active UK Government SC Clearance. Certifications Mandatory ...

solutions. GenAI Implementation: Hands-on deployment of Azure OpenAI, Vector Databases, and RAG patterns. Data Engineering: Oversee large-scale data processing using Python and PySpark to support AI model fine-tuning. Leadership: Manage technical delivery teams and deliver upskilling sessions to the customer. Stakeholder Management: Gain consensus on complex … roadmaps with diverse client groups. Technical Skills & Experience Core AI: Azure OpenAI, Prompt Engineering, Cognitive Search, Form Recognizer. Data/Coding: Expert-level Python, PySpark, and Vector DB integration. Ops: Practical knowledge of MLOps and LLMOps lifecycles. Security: Must hold active UK Government SC Clearance. Certifications Mandatory ...