1 to 25 of 58 PySpark Jobs in the UK

addressing any concerns and ensuring incidents are effectively managed. What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark). - A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control). - Proven ability to design data ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

cloud, S3, cross-cloud connectivity, API Gateways GCP (BigQuery, GCS, Dataflow) Deep expertise in: Databricks (Lakehouse architecture, Delta Lake, Unity Catalog, Delta Sharing, Pyspark, Spark SQL etc.) Snowflake (data warehousing & data sharing, Iceberg, ) Strong understanding of: Data architecture patterns, industry benchmarking, Data integration patterns (ETL/ELT, streaming ...

essential - More to React JS and Java script Back-end: Expertise in C# .NET Core, REST APIs, and microservices-based development is essential Python (pyspark): Experience with data pipelines and analytics is a plus. Power BI: Familiarity with developing dashboards and data models. Architecture: Knowledge of microservices, API design ...

working independently and collaboratively Ability to manage priorities across multiple projects and clients Bonus Skills Experience with any of the following would be beneficial: PySpark Power BI Power Apps GCP What We’re Looking For We’re looking for someone who: Is comfortable moving between different client environments ...

engagement skills, including requirements gathering, expectation management and delivery planning. Desirable Knowledge and practical experience of hospital systems and datasets (where relevant locally). PySpark experience. Experience with NLP and/or computer vision techniques (where relevant to Trust priorities). Experience using deep learning frameworks such as TensorFlow ...

with Databricks SQL endpoints for data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/ ...

programming, concurrency, multithreading) Expertise in RESTful API design, OpenAPI/Swagger documentation and API lifecycle best practices Hands-on experience with ML libraries: PyTorch, PySpark, scikit-learn, Hugging Face Transformers Proficiency in AI tooling: Azure ML, Databricks, MLflow, LangChain, LangGraph Proven experience with Git, unit testing, containerisation and agile ...

data providers-ensuring data is reliable, auditable, and delivered at scale. Data Engineer Responsibilities Act as the subject matter expert across ADF, Databricks, Python, PySpark, and Spark SQL Manage the daily ingestion and processing of raw data into enterprise data platforms Collaborate with the wider team to define data ...

Experience Technical Skills Strong SQL and data modelling experience Azure data stack (ADF, Synapse, Fabric, Databricks) Experience building ETL/ELT pipelines Python/PySpark (desirable) DBT or similar transformation tools Experience with orchestration tools (ADF, Airflow, Prefect) CI/CD, Git, and Azure DevOps ...

Hands-on experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory • Hands-on data engineering experience (Spark/PySpark, Delta Lake) • Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC • Experience leading Agile teams and managing risks, dependencies ...

Databricks Defining data models and Databricks architecture standards Establishing data governance and security frameworks Aligning data architecture with enterprise cloud strategy Supporting development using PySpark, SQL and Python within Databricks Working within CI/CD and DevOps environments (Git-based) About You: Strong technical background in Databricks and Azure ...

Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Configured and validated Foundry Databricks connector. ...

Looking For: Proven experience working with Palantir Foundry in production environments Strong engineering background with expertise in at least one of: Python, SQL, PySpark, or TypeScript Strong understanding of modern data architectures & distributed systems Ability to work in fast-paced, ambiguous environments Excellent stakeholder engagement and communication skills Experience ...

query performance Strong analytical and problem-solving skills Desirable Skills Knowledge of modern data architectures Experience with SQL, SSRS, SSIS and SSAS Experience using PySpark Familiarity with KQL Experience with data cataloguing tools Exposure to C# What you'll get in return Competitive Salary - 52,750 with a generous ...

someone who combines strong engineering capability with a solid understanding of the cyber domain. Key skills required: Strong Python development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

Engineer with a strong modelling focus Hands-on Databricks experience - certifications are a plus Solid understanding of Medallion/multi-hop architecture Proficiency in PySpark, Spark SQL, and Delta Lake Experience with Azure (ADF, Synapse, or similar) or AWS data services Strong communication skills - you'll be working closely ...

Palantir Foundry experience, especially Databricks connector, Data Connection, datasets, syncs, projects and permissions. Secure data-sharing, data governance, lineage and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Secure AWS S3 and Databricks access ...

within NHS England, UK Health Security Agency, Department of Health and Social Care, or another central government data environment (Essential) Solid experience with Python, PySpark, Power BI, and AWS Strong communication skills - able to engage confidently with both technical peers and non-technical clinical or operational stakeholders Available ...

join their Defence & National Security team. In order to be successful, you will have the following experience: Experienced in the Python data stack (PySpark, numpy, pandas, scikit-learn, etc.) Experienced within AWS-based infrastructure and services Excellent communication skills, able to work with customers to shape requirements Able ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...