51 to 74 of 74 PySpark Jobs in the UK excluding London

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

Data Engineers. What You Bring: Extensive hands-on experience with Databricks and Synapse. Strong proficiency in Azure DevOps. Solid knowledge of SQL and PySpark within a Databricks environment. Experience using Databricks in a federated cloud setting. Proven leadership experience with small engineering teams. Skills in configuration management and documentation. ...

migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards … reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines ...

TensorFlow Serving, TorchServe, ONNX, and TensorRT. Build and manage ML pipelines using MLflow, Kubeflow, and Azure ML Pipelines. Work with large‐scale data using PySpark and integrate models into production environments. Monitor model performance and retrain as needed to ensure accuracy and efficiency. Collaborate with cross‐functional teams ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

help deliver reliable, high-quality data to support business insights. Key Responsibilities Build and maintain scalable ETL/ELT pipelines using Python and PySpark Ingest and integrate data from multiple sources including trading, finance, and operational systems Work within a Databricks Lakehouse architecture to transform and deliver data Optimise … stakeholders to ensure data is accessible and usable Key Skills & Experience Strong experience in Data Engineering (3–6 years) Proficiency in Python and PySpark for large-scale data processing Hands-on experience with Databricks (Delta Lake, Workflows, Lakehouse architecture) Solid knowledge of AWS services (e.g. S3, Kinesis, Lambda ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

internal teams. Ideal candidates will have strong programming skills in Python, a solid background in databases, and excellent collaboration abilities. Experience with AWS, PySpark, or Terraform is advantageous. We promote an inclusive work environment and provide accommodations for individuals with disabilities. #J-18808-Ljbffr ...

making across the organization. Embrace Cutting-Edge Technology: Collaborate with a tech-savvy team to leverage the latest tools and platforms, including Microsoft Fabric, PySpark, and Power BI, to build scalable, efficient, and secure data solutions. Elevate the Customer Experience: Ensure reliable data flow between core systems, empowering teams … insights to valued clients. Preferred Requirements: Proven expertise in building and operating production-ready data pipelines, with a strong background in SQL, Python, and PySpark/Spark. Proficiency in building REST API integrations, with a solid understanding of data modelling, ETL/ELT patterns, and modern data warehousing concepts. ...

build team resilience and engineering capability What the tech stack looks like Databricks (primary — strong experience essential) Azure Data Factory Azure Data Lake Storage PySpark/Spark SQL CI/CD with Azure DevOps Dimensional modelling/Kimball methodology What you'll need Strong, demonstrable Databricks experience - hands … theoretical Proven pipeline development skills - you set up pipelines, you don't just work inside them Experience with PySpark/Spark SQL Background in medallion architecture and data warehouse design Comfortable in an Agile, CI/CD environment Financial services experience Why this one is worth a look This ...

Data Architect (Databricks or Microsoft Fabric) £110,000–£125,000 Hybrid – 2 days a week in London, Manchester, or Edinburgh We’re looking for a twoexperienced Data Architects to design and deliver modern, scalable data ...

Data Engineering Lead Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract till the end of the year and is required to be onsite 3 ...

Data Engineer Hadoop/Cloudera/PySpark/Python Location: Sheffield (Hybrid 3 days per week preferred) c.£600 per day Inside IR35 Initial 6-month contract with strong likelihood of extension We are supporting a major global banking organisation on the growth of its Data Technology function … already based in the local area. Key Responsibilities Design, develop and maintain scalable data pipelines and ETL processes Develop robust solutions using Python, PySpark and SQL Work with large-scale datasets in distributed data environments Develop and support data solutions within on-prem Hadoop/Cloudera environments Support data ...

We’re looking for a highly skilled Lead Data Engineer to provide technical leadership across data engineering, delivery execution and project governance. This role ensures development standards are upheld while delivering high‐quality, reliable solutions ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...

ETL Data Engineer/Data Architect - Azure Stack We are looking for an experienced ETL Data Engineer/Data Architect to deliver projects in collaboration with our IT partners and internal stakeholders that will transform ...

Skills You’ll need to have experience with some of the following and be able to learn new technologies or concepts quickly: Python/PySpark - Experience using Python and PySpark to connect to APIs and extract data, with strong API design skills and familiarity with LLM orchestration frameworks ...

challenges Communicate findings and recommendations to stakeholders in a clear and structured manner Learning & Professional Development Build practical expertise in React, TypeScript, Python, and PySpark within a Palantir Foundry environment Apply and develop best practices in data engineering, visualisation, and business intelligence Engage with mentorship and guidance from senior … appreciation for how data can be presented in ways that are both intuitive and impactful Working knowledge of SQL, with exposure to Python or PySpark desirable but not essential Strong analytical and problem-solving skills, with a passion for extracting meaning from complex datasets Effective communication skills, both written ...

Senior Test Engineer Identifies and raises awareness of risks arising from automation test results. Identifies new and implements existing test frameworks to improve confidence in testing. Proficient in coding languages such as Python, Java, SQL ...

also partner directly with teams across the company—from building embedded dashboards to driving strategic acquisition‐related projects. We work in Python, PySpark, AWS, and Terraform using Agile methodologies, but we welcome strong engineers with experience in other backend stacks who are excited to learn. What … working with large datasets (e.g. Spark, other big data frameworks) Excellent communication and cross‐functional collaboration skills Nice‐to‐Haves Experience with AWS, PySpark, or Terraform Strong problem‐solving skills related to data architecture and data analysis In addition to our core values, Addepar is proud ...

ownership, visibility, and influence. What You’ll Be Doing Building scalable data pipelines and integrations Working with modern tools including Microsoft Fabric, Python/PySpark, and SQL Turning complex business challenges into smart data solutions Collaborating with stakeholders across the organisation Helping shape best practices within a growing data … team What They’re Looking For Strong experience in data engineering and cloud data platforms Python/PySpark and SQL skills Experience integrating APIs and SaaS platforms Someone who enjoys solving problems and building things properly Why This Role Stands Out Influence a company-wide data platform High-impact ...

models. This isn't just a "high-level" role-we need someone who can design the blueprint and then dive into the Python/PySpark code to ensure the foundation is rock solid. Key Responsibilities: Architect & Design: Lead the implementation of on-premises, cloud, and hybrid AI solutions. GenAI … certification. AI Specialization: Proven experience with Prompt Engineering, Azure OpenAI, Document Intelligence (Form Recognizer), and Cognitive Search. Data Engineering: Expert proficiency in Python and PySpark for large-scale data processing. Operations: Familiarity with MLOps or LLMOps frameworks to support production-grade AI. Seniority: 15+ years of total IT experience ...

calculations, including netting sets, supervisory factors and collateral/CSA logic.* **Design a modern Databricks architecture** using Delta Lake, Unity Catalog and Spark/PySpark to support versioned, reproducible and regulator‐defensible exposure calculations.* **Develop high‐performance compute pipelines** that support intraday recalculation, scenario testing and rapid exposure analytics … exposure methodology, netting sets, supervisory parameters and collateral/CSA design.* **Strong Databricks engineering capability**, including Delta Lake versioning, Unity Catalog governance, Spark/PySpark optimisation and secure data architecture.* **Experience designing governed, audit‐ready systems**, with a focus on lineage, reproducibility, evidence trails and internal/external audit ...

Data Engineer PySpark/Python Location: Sheffield (Hybrid 3 days per week preferred) c.£600 per day Inside IR35 Initial 6-month contract with strong likelihood of extension We are supporting a major global banking organisation on the growth of its Data Technology function and are looking ...