1 to 25 of 60 Permanent PySpark Jobs in the UK excluding London

such as Databricks, Azure Data Factory, Snowflake, or similar Good programming skills in Python and SQL, with familiarity with data engineering libraries (e.g., pandas, PySpark, dbt) Solid understanding of data modelling, ETL/ELT processes, and Lakehouse concepts Experience with data quality frameworks, data governance, and compliance requirements. Familiarity ...

from relational databases or operational systems 2. Modern data platforms and engineering tools: Practical knowledge of data engineering tools or approaches such as Python, PySpark, Microsoft Fabric, Azure Data Factory, Synapse, Lakehouse, OneLake or equivalent platforms 3. Data quality, validation and reconciliation: Good understanding of data quality, validation, reconciliation ...

addressing any concerns and ensuring incidents are effectively managed. What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark). - A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control). - Proven ability to design data ...

quality and security best practices Excellent stakeholder management and communication skills Experience operating within Agile delivery environments Desirable Experience Microsoft Fabric Power BI development PySpark REST API integrations Lakehouse architectures AI and Machine Learning platforms Retrieval Augmented Generation (RAG) Vector databases Generative AI solutions Power Platform technologies Data transformation ...

such as AWS, Azure, Microsoft Fabric and Databricks, as well as big data technologies like Hadoop and Spark Skilled in programming languages , including Python, PySpark and Scala, with extensive experience developing robust ETL pipelines and ensuring scalable deployments Experienced in mentoring and developing less experienced Data Engineers , guiding them ...

working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

technical architecture and detailed design. Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines. Strong programming skills in PySpark, SQL, Python. Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines. Hands‐on experience with DQ frameworks, validation rules and reconciliation logic. Ability ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

from different sources. Good communication and teamwork skills. Experience with cloud data platforms such as Databricks or Snowflake on AWS. Strong skills in Python, PySpark, and SQL. Good understanding of ETL/ELT pipelines and data transformation. Experience with data modelling and database optimisation. Strong knowledge ...

/ELT pipelines using Azure Data Factory to ingest data from operational systems, sensors, maintenance logs, and regulatory datasets. Use Azure Databricks (SQL/PySpark) to transform, cleanse, and model large datasets for analytical consumption. Build interactive dashboards and reports in Power BI to support operational performance, asset health ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

data lineage. Experience managing compute across Fabric and Databricks environments. CI/CD knowledge using Azure DevOps, GitHub, and Databricks Asset Bundles. Proficiency in PySpark, Databricks SQL, T‐SQL, and Delta Lake internals. Strong understanding of Medallion Architecture and modern data modelling (Star Schema, Data Vault 2.0). Experience ...

including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/ ...

practitioner level. Knowledge Essential Extensive knowledge with integration and transformation within Cloud Data Warehouses (e.g Azure, AWS, GCP) including languages such as Python, SQL, PySpark, Java Knowledge and experience with programming languages such as React, Typescript, Javascript is desirable. Skills and Experience Essential Able to design appropriate solutions with ...

Pipelines Databricks Data engineering, including data enrichment and validation. Database management, including partitioning of data ETL processes Azure - pipelines, data factory, function apps SQL Pyspark HDFS Big data, including how to optimise operations Core Competencies & Technical Skills (AI and emerging technology): Designs, integrates and operates AI-enabled solutions within ...

including: Azure Synapse Databricks Microsoft Fabric Azure SQL Azure Data Factory (ADF) Cosmos DB Power BI Strong hands-on capability with: SQL/NoSQL PySpark Data streaming technologies Strong understanding of: Modern data architectures Lakehouse and cloud-native platforms Data integration patterns Enterprise cloud design Excellent communication and stakeholder ...

experience. Knowledge Essential Good working knowledge with integration and transformation within Cloud Data Warehouses (e.g Azure, AWS, GCP) including languages such as Python, SQL, PySpark, Java Knowledge and experience with programming languages such as React, Typescript, Javascript is desirable Skills and Experience Essential Can deliver well-scoped work independently ...

specialist area.. Knowledge Essential Extensive knowledge with integration and transformation within Cloud Data Warehouses (e.g Azure, AWS, GCP) including languages such as Python, SQL, PySpark, Java Knowledge and experience with programming languages such as React, Typescript, Javascript is desirable. Skills and Experience Essential Proven ability to use data engineering ...

data providers-ensuring data is reliable, auditable, and delivered at scale. Data Engineer Responsibilities Act as the subject matter expert across ADF, Databricks, Python, PySpark, and Spark SQL Manage the daily ingestion and processing of raw data into enterprise data platforms Collaborate with the wider team to define data ...

Databricks Defining data models and Databricks architecture standards Establishing data governance and security frameworks Aligning data architecture with enterprise cloud strategy Supporting development using PySpark, SQL and Python within Databricks Working within CI/CD and DevOps environments (Git-based) About You: Strong technical background in Databricks and Azure ...

Looking For: Proven experience working with Palantir Foundry in production environments Strong engineering background with expertise in at least one of: Python, SQL, PySpark, or TypeScript Strong understanding of modern data architectures & distributed systems Ability to work in fast-paced, ambiguous environments Excellent stakeholder engagement and communication skills Experience ...

query performance Strong analytical and problem-solving skills Desirable Skills Knowledge of modern data architectures Experience with SQL, SSRS, SSIS and SSAS Experience using PySpark Familiarity with KQL Experience with data cataloguing tools Exposure to C# What you'll get in return Competitive Salary - 52,750 with a generous ...

genuinely influence engineering direction within a scaling business What they’re looking for 👇 😍 Strong hands-on Data Engineering experience across Python, Spark, SQL and PySpark 😍 Experience working with hyperscaler cloud platforms such as AWS, Azure or GCP 😍 Exposure to modern tooling including Databricks, Microsoft Fabric and PowerBI 😍 Previous experience ...

someone who combines strong engineering capability with a solid understanding of the cyber domain. Key skills required: Strong Python development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...