26 to 50 of 53 PySpark Jobs in the UK excluding London

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query ...

Strong, demonstrable experience running A/B testing and causal inference in a product-focused environment. Core Technical Stack: Proficiency with SQL , Python/PySpark , and Snowflake . Experience with data visualisation tools (e.g. Tableau, Power BI) Hands-on experience with A/B testing and experimentation Experience with ...

IR35 for initially 6 months. Hybrid to Glasgow (2 days a week in the office) 450 - 525 a day Technology: Python & SQL development (ideally PySpark) Databricks or similar cloud platforms Git, CI/CD, Azure DevOps (ideal) or GitHub Optimiser apps & battery storage systems (optional) Software quality assurance Unit ...

unexpected backgrounds. Desirable Experience Any exposure to Palantir Foundry or AIP is a bonus, but not required - this is where you will build it PySpark or experience with large-scale data processing Experience with ERP, supply chain, finance or commercial data in an industrial setting Consultancy or startup experience ...

Experience: 6-10+ Years Key Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance Role Summary We are looking for an experienced MDM/Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse click apply for full ...

Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance Role Summary We are looking for an experienced MDM/Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse. The role will focus on common IDs, Golden Records, entity ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

deliver high-quality data products and insights. We are looking for people with extensive commercial experience with AWS Glue, developing Glue Jobs using PySpark and Python, AWS Glue Data Catalog, Crawlers, Workflows and triggers, Amazon S3 data lake implementation and management, AWS IAM roles, policies and security best practice. ...

small team of data engineers and own work end to end: talking to stakeholders, designing models, building and testing pipelines in Python and PySpark, and supporting BI. It's essentially remote, and the platform will grow from one business to the whole group, so there's plenty … conceptual, logical and physical data models for the new platform Building metadata-driven pipelines across Azure Synapse, Data Factory and PostgreSQL using Python and PySpark Migrating data from legacy reporting solutions onto the platform while protecting data integrity Releasing code through Azure DevOps and Git, and keeping documentation ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

comfortable with large datasets, distributed processing and applying software engineering discipline to data products; Foundry experience is highly desirable but strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. Extensive experience with Python and PySpark/Apache Spark. Proven track record designing ...

integration and multi-market SCI knowledge desirable. Role 2: BNA Production-Readiness, MLOps & Integration Engineer Must Have: Strong hands-on background in Databricks, PySpark, Python and SQL Production pipeline engineering and troubleshooting Workflow orchestration and dependency management Idempotency, retry/restart and failure-recovery patterns CI/CD, version … platforms Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer Must Have worked with: Strong hands-on background with Databricks, SQL, Python/PySpark ETL/ELT Data-pipeline migration Legacy-to-cloud modernisation Production troubleshooting Dependency analysis Pipeline repointing Error handling and recovery Data validation/reconciliation ...

requirements into scalable data solutions. Key Responsibilities Design scalable enterprise data models and lakehouse architectures Build production-grade data pipelines using SQL, Python/PySpark and Databricks Implement data governance, security and access controls through Unity Catalog Work directly with business stakeholders to understand and prioritise requirements Establish data … data engineering and advanced data modelling Hands-on expertise in Databricks, Delta Lake, Unity Catalog and Databricks Workflows Production experience with SQL, Python/PySpark and dbt and/or Spark Declarative Pipelines Strong understanding of data governance, security and access-control design Excellent stakeholder-management and business-communication ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

Senior Data Engineer - Python, SQL, Airflow, Azure, Pyspark, Kubernetes Salary: Up to £80,000 per annum Benefits: Excellent benefits package including annual bonus, free onsite gym, and company pension scheme. About the Role We are seeking an experienced Senior Data Engineer to join an innovative SaaS business that … Leadership & Mentorship Technology Stack The successful candidate will have experience working with some or all of the following technologies: Data Processing: Apache Spark/PySpark Workflow Orchestration: Apache Airflow Containerisation: Kubernetes Cloud Platform: Microsoft Azure Data APIs & Semantic Layer: CubeJS Programming Languages: Python, SQL Essential Skills & Experience Degree ...

data science, statistical modelling and using data to solve problems. Some experience with programming languages or analytical tools, such as Python, SQL, SAS or PySpark, gained through university projects, placements, internships or employment. An understanding of statistical concepts and an interest in predictive modelling or machine learning. Strong analytical ...

ensure reporting is accurate and useful, and improve engineering standards while supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures … documentation, Git and CI/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with ...

Azure Data Engineer - 1/2 days onsite 18 month FTC Summary: Join a team building a modern Azure-based data platform. This hands-on engineering role involves designing and developing scalable, automated data pipelines ...

deploying machine learning models in production, alongside strong hands-on Python skills. You should be comfortable with distributed computing tools such as Spark or PySpark, cloud platforms and modern MLOps practices. Experience managing and developing data scientists is essential, as is the ability to explain technical decisions and work ...

progressing its implementation and ongoing development. Developing and extending a medallion lakehouse architecture across bronze, silver and gold data layers. Working hands-on with PySpark notebooks, SQL, Dataflows Gen2 and data integration technologies. Developing robust ETL/ELT pipelines, bringing data together from multiple business systems and sources. Applying … ideally bring: Strong hands-on experience with Microsoft Fabric, including Lakehouse architecture and its wider data engineering capabilities. Good working knowledge of PySpark and Spark notebooks, with the ability to understand, troubleshoot and extend existing code. A solid understanding of data warehousing, ETL/ELT processes, data integration ...

Description Join us, be part of more. We're so much more than an energy company. We're a family of brands revolutionising how we power the planet. We're energisers. One team of 21 ...

DV Cleared Data Engineer Location: West Midlands - Hybrid working Rate: Up to £600 per day Inside IR35 Contract: Until 31 March 2027, with extensions expected Clearance: Active DV Clearance essential The Role We are seeking ...

Snowflake - Warehouses, Streams/Tasks, Snowpipe, Snowpark, UDFs/Stored Procedures, Time Travel and Cloning. Snowpark and end-to-end pipeline development. Experience with PySpark, Polars or Dask is also valuable. Python, including testing, build tooling and engineering best practices. SQL, including Snowflake-optimised SQL. dbt, including Jinja templating … champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great opportunity to join a collaborative ...

blend of experience is expected as the business transitions to Microsoft Fabric, with key skills and core technologies including: SQL, Microsoft Fabric, Python/PySpark, Data Factory, etc. Exposure to and working with AI based tools such as Microsoft Copilot will certainly enhance candidate applications. Managerial and leadership experience … forward your most recent CV for consideration. BIRMINGHAM/NOTTINGHAM/LEICESTER/MIDLANDS/DATA/SQL/MICROSOFT FABRIC/PYTHON/PYSPARK/DATA WAREHOUSE/DATA LAKE/DATA FACTORY/HOME/BIRMINGHAM/NOTTINGHAM/LEICESTER/MIDLANDS/DATA/SQL/ ...