26 to 50 of 58 PySpark Jobs in the UK

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

leading Professional Services firm is looking to hire a Finance Data Engineering & Analytics Lead to join their team. You will develop Python/PySpark-based data pipelines, build data models and datasets for finance, actuarial and risk reporting. You will also develop analytics, dashboards and data validation tooling. ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards … reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

data platforms and cloud environments Collaborate with engineering and architecture teams to deliver high-quality solutions Key Skills & Experience Strong experience with Python and PySpark Hands-on expertise in AWS Experience working with Databricks Ideally 5+ years working with this tech stack as a data engineer Proven background delivering ...

Technical Environment : Palantir Foundry, Foundry Virtual Tables, Ontology, Workshop, Pipeline Builder, Code Repositories, AWS, Databricks, Terraform, IaC, Delta Lake, Parquet, Iceberg, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...

Data Engineers. What You Bring: Extensive hands-on experience with Databricks and Synapse. Strong proficiency in Azure DevOps. Solid knowledge of SQL and PySpark within a Databricks environment. Experience using Databricks in a federated cloud setting. Proven leadership experience with small engineering teams. Skills in configuration management and documentation. ...

assurance activities Collaborate with enterprise architects, cloud teams, and delivery stakeholders Technology Stack Strong experience with: Azure Data Factory (ADF) Azure Databricks (ADB) PySpark Synapse Analytics Python Modern data patterns (Lakehouse/Medallion architecture) What We're Looking For Proven experience in Data Architecture within cloud environments Strong ...

impact. Key Responsibilities Design and deliver end-to-end technical solutions across complex data environments Build and maintain applications using technologies such as Python, PySpark, TypeScript, and React Develop AI-driven workflows and operational tooling Work directly with client stakeholders in fast-paced delivery environments Present demos, gather feedback ...

connectivity issues across AWS, Databricks and downstream integrations. Technical Environment: AWS, Databricks, Unity Catalog, Palantir Foundry, Terraform, IaC, Delta, Parquet, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

help deliver reliable, high-quality data to support business insights. Key Responsibilities Build and maintain scalable ETL/ELT pipelines using Python and PySpark Ingest and integrate data from multiple sources including trading, finance, and operational systems Work within a Databricks Lakehouse architecture to transform and deliver data Optimise … stakeholders to ensure data is accessible and usable Key Skills & Experience Strong experience in Data Engineering (3–6 years) Proficiency in Python and PySpark for large-scale data processing Hands-on experience with Databricks (Delta Lake, Workflows, Lakehouse architecture) Solid knowledge of AWS services (e.g. S3, Kinesis, Lambda ...

strategies for performance and scalability Collaborate with data scientists, analysts and DevOps teams to deliver end-to-end solutions Contribute hands-on development using PySpark, SQL and Python Support CI/CD and DevOps practices using Git-based version control Present architectural designs to governance and review forums Essential … governance, metadata and security frameworks Experience working across enterprise-scale cloud data landscapes Strong understanding of performance optimisation and data management strategies Experience with PySpark, SQL and Python Spark Desirable Skills for the Databricks Solution Architect: Strong stakeholder management and communication skills Experience working within Agile delivery environments Experience ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

making across the organization. Embrace Cutting-Edge Technology: Collaborate with a tech-savvy team to leverage the latest tools and platforms, including Microsoft Fabric, PySpark, and Power BI, to build scalable, efficient, and secure data solutions. Elevate the Customer Experience: Ensure reliable data flow between core systems, empowering teams … insights to valued clients. Preferred Requirements: Proven expertise in building and operating production-ready data pipelines, with a strong background in SQL, Python, and PySpark/Spark. Proficiency in building REST API integrations, with a solid understanding of data modelling, ETL/ELT patterns, and modern data warehousing concepts. ...

Data Engineering Lead Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract till the end of the year and is required to be onsite 3 ...

Data Engineer PySpark/Python Location: Sheffield (Hybrid 3 days per week preferred) c.£600 per day Inside IR35 Initial 6-month contract with strong likelihood of extension We are supporting a major global banking organisation on the growth of its Data Technology function and are looking … regulatory functions across a complex global environment. Key Responsibilities Design, develop and maintain scalable data pipelines and ETL processes Develop robust solutions using Python, PySpark and SQL Work with large-scale datasets in distributed data environments Contribute to data modelling and data warehousing initiatives Support CI/ ...

Data Engineer PySpark/Python Location: Sheffield (Hybrid 3 days per week preferred) c.£600 per day Inside IR35 Initial 6-month contract with strong likelihood of extension We are supporting a major global banking organisation on the growth of its Data Technology function and are looking … already based in the local area. Key Responsibilities Design, develop and maintain scalable data pipelines and ETL processes Develop robust solutions using Python, PySpark and SQL Work with large-scale datasets in distributed data environments Contribute to data modelling and data warehousing initiatives Support CI/CD and automation ...

working with Databricks** You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … Develop automation for cluster management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor ...