101 to 125 of 279 PySpark Jobs in the UK

Gen2), Purview, Cosmos DB, OpenAI, Azure ML, and AI Foundry Solid understanding of software engineering concepts, with hands‐on experience using Python, Scala or PySpark, data modelling techniques, database technologies, and SQL Confident communicator, able to explain technical concepts to non‐technical audiences and mentor junior colleagues For Senior ...

data integrity Ability to use data science techniques and statistics, including use of Python/R Ability to manipulate data in data lakes using PySpark/Experience with cloud-based data architectures and scalable data solutions Highly data rational with the ability to use quantitative and qualitative information ...

optimisation and leading delivery teams. Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments. Advanced experience with PySpark, SparkSQL and large-scale distributed data processing. Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling. Excellent SQL skills ...

Please note, this vacancy closes at 12pm, midday, on Monday 7th September. We reserve the right to close this vacancy early if we receive a high volume of suitable applications. Interviews will take place as ...

data processing & Spark Amazon Bedrock - Generative AI & foundation models AWS Lambda - Serverless data & AI workflows Amazon QuickSight - BI, dashboards & visualisation Python, SQL, R & PySpark - Data science & analytics As an equal opportunities’ employer, we welcome applications from individuals of all backgrounds. However, for you to be eligible for this role ...

knowledge (you don't need everything) Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

client sites or in our office, with hybrid work plan of at least two days per week. Use tools such as Data Bricks, Python, PySpark, R, Excel, Palantir Foundry and PowerBI to deliver scalable, high‐performance applications. Qualifications Essential requirements A degree, Masters or PhD in data science, mathematics ...

Data Lake Storage (Gen2), Purview, Cosmos DB, OpenAI, AzureML, AIFoundry, Kubernetes Understanding of software engineering tools and concepts including experience in Python, Scala or PySpark, database technologies, data modelling and SQL Confident communicator who is able to explain technical terms to non‐technical audiences and mentor junior colleagues Leads ...

fields.· Proven track record of designing, developing, and deploying AI systems in production environments.· Proficient in Python and key ML libraries (e.g. PyTorch, PySpark, scikit-learn, Hugging Face Transformers).· Hands-on experience with modern data platforms and AI tooling such as Azure ML, Databricks, MLflow, LangChain, LangGraph.· Proven ...

ISx4 Group is seeking a Data Engineer specialized in Databricks to design and build scalable data pipelines. The role involves collaborating within an agile team to deliver high-quality data solutions for enterprise clients in ...

cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

securityPractical cloud-native experiencePreferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or SnowflakeHands-on experience with Spark/PySpark and other big data processing technologiesDemonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine learning ...

looking for a Senior Data Engineer – Tech Lead (Databricks, PySpark) to join our team in London, UK, in a hybrid working mode.In this role, you will lead the design, development and optimization of scalable cloud-native data architectures, focusing on Azure Databricks, PySpark and Lakehouse principles. You will … architecture, design and build of large-scale data platforms using Azure Databricks and modern cloud technologiesImplement and optimize ETL workflows and streaming pipelines with PySpark and Delta Live Tables following Lakehouse principlesEnhance performance, manage cloud costs and ensure platform reliability for structured streaming workloadsDefine data governance, security and quality ...

knowledge (you don't need everything) Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

knowledge (you don't need everything) Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

complex data led organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design ...

complex data led organisation Deep understanding & hands‐on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

complex data led organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

management, network design and access control; Databricks and OCI, or comparable platforms Strong Python and SQL, and experience building and monitoring production pipelines using PySpark or Spark Experience with infrastructure as code, CI/CD and a Git and GitHub workflow A track record of setting engineering standards ...

Experience Strong experience across both Data Science and Data Engineering Hands-on expertise with Microsoft Fabric and Databricks Advanced Python and SQL skills, with PySpark desirable Experience building data pipelines and machine learning models Knowledge of NLP, entity resolution, or document processing is advantageous Strong communication skills and commercial ...

ability to work independently, challenge assumptions constructively and influence decision-making through evidence It would be beneficial if you also have: Experience with Databricks, PySpark, Azure or similar cloud-based data environments Experience within operational, retail, logistics or customer-focused businesses Exposure to AI solutions or emerging machine learning ...

experience with Databricks — running queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large-scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

language models Experience with experiment design and conducting A/B tests Experience building shared or platform-style ML systems Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with ...