26 to 50 of 101 PySpark Jobs

design decisions. Required Skills & Experience Strong analytical skills with experience working with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow ...

Significant hands-on experience in data engineering, analytics engineering, or related disciplines delivering enterprise data solutions Strong proficiency in Databricks, SQL, and Python/PySpark, including building, optimizing, and troubleshooting ETL pipelines Experience designing scalable, maintainable data architectures and dimensional models for BI, reporting, and analytical use cases Proven ...

good understanding of Azure PaaS management and resource monitoring CI/CD experience and DevOps Working knowledge of SQL and experience in Python/PySpark Microsoft Power Platform Ability to collect, organize, and analyze data from multiple sources to solve business challenges. Knowledge in testing and validating Analytical solutions ...

looking for: Strong background in data engineering and software development Proven experience working with Databricks (essential) Hands-on expertise with tools such as Python, PySpark, Spark, and SQL Experience building and maintaining data pipelines and data platforms Solid understanding of data modelling and data architecture principles Experience working with ...

looking for: Strong background in data engineering and software development Proven experience working with Databricks (essential) Hands-on expertise with tools such as Python, PySpark, Spark, and SQL Experience building and maintaining data pipelines and data platforms Solid understanding of data modelling and data architecture principles Experience working with ...

Strong experience building and delivering data pipelines using cloud platforms (AWS preferred) and big data tools like Kafka or Spark. Confident using Python (or PySpark) and SQL to process, transform and analyse data, and able to guide others in doing the same. Good understanding of modern data platforms (such ...

Strong experience building and delivering data pipelines using cloud platforms (AWS preferred) and big data tools like Kafka or Spark. Confident using Python (or PySpark) and SQL to process, transform and analyse data, and able to guide others in doing the same. Good understanding of modern data platforms (such ...

Strong experience building and delivering data pipelines using cloud platforms (AWS preferred) and big data tools like Kafka or Spark. Confident using Python (or PySpark) and SQL to process, transform and analyse data, and able to guide others in doing the same. Good understanding of modern data platforms (such ...

Strong experience building and delivering data pipelines using cloud platforms (AWS preferred) and big data tools like Kafka or Spark. Confident using Python (or PySpark) and SQL to process, transform and analyse data, and able to guide others in doing the same. Good understanding of modern data platforms (such ...

with Databricks SQL endpoints for data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/ ...

practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence You're collaborative with excellent ...

engineers on new capabilities Opportunity to gain exposure to data modelling and machine learning projects YOUR SKILLS AND EXPERIENCE: Experience with Python and PySpark (commercial, academic, or project-based) Exposure to Databricks or similar data platforms Strong desire to learn and develop within a collaborative team Understanding of data ...

Looking For: Proven experience working with Palantir Foundry in production environments Strong engineering background with expertise in at least one of: Python, SQL, PySpark, or TypeScript Strong understanding of modern data architectures & distributed systems Ability to work in fast-paced, ambiguous environments Excellent stakeholder engagement and communication skills Experience ...

query performance Strong analytical and problem-solving skills Desirable Skills Knowledge of modern data architectures Experience with SQL, SSRS, SSIS and SSAS Experience using PySpark Familiarity with KQL Experience with data cataloguing tools Exposure to C# What you'll get in return Competitive Salary - £52,750 with a generous ...

someone who combines strong engineering capability with a solid understanding of the cyber domain. Key skills required: Strong Python development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security ...

someone who combines strong engineering capability with a solid understanding of the cyber domain. Key skills required: Strong Python development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security ...

programming, concurrency, multithreading) Expertise in RESTful API design, OpenAPI/Swagger documentation and API lifecycle best practices Hands-on experience with ML libraries: PyTorch, PySpark, scikit-learn, Hugging Face Transformers Proficiency in AI tooling: Azure ML, Databricks, MLflow, LangChain, LangGraph Proven experience with Git, unit testing, containerisation and agile ...

DataBricks, Snowflake or MS Fabric. Comfortable engaging directly with stakeholders, taking ownership of technical decisions, and contributing to the broader data strategy. Essential SQL PySpark/Python DBT Azure knowledge (i.e. Logic apps) Power BI/Fabric Semantic Models Ability to work with stakeholders with their own operational pressures. ...

DataBricks, Snowflake or MS Fabric. Comfortable engaging directly with stakeholders, taking ownership of technical decisions, and contributing to the broader data strategy. Essential SQL PySpark/Python DBT Azure knowledge (i.e. Logic apps) Power BI/Fabric Semantic Models Ability to work with stakeholders with their own operational pressures. ...

travel to Leeds/Newcastle) Clearance: Active SC Clearance required Rate: Up to £425/day (Inside IR35) Tech Stack: AWS (EMR, S3, Lambda), PySpark, Python, SQL, CI/CD Focus: Building scalable pipelines, modernising ETL, supporting MI/Power BI Level: Mid to Senior - mentoring & architectural input expected ...

Databricks, Data Factory, and Data Lake. Strong understanding of Azure PaaS management, resource monitoring, and CI/CD processes. Proficiency in SQL, Python/PySpark, and the Microsoft Power Platform. This role offers a competitive annual salary of £70,000 and excellent employee benefits, including 16% non contributory pension ...

Databricks, Data Factory, and Data Lake. Strong understanding of Azure PaaS management, resource monitoring, and CI/CD processes. Proficiency in SQL, Python/PySpark, and the Microsoft Power Platform. This role offers a competitive annual salary of £70,000 and excellent employee benefits, including 16% non contributory pension ...

within NHS England, UK Health Security Agency, Department of Health and Social Care, or another central government data environment (Essential) Solid experience with Python, PySpark, Power BI, and AWS Strong communication skills - able to engage confidently with both technical peers and non-technical clinical or operational stakeholders Available ...

Skills & Experience we’re looking for: Commercial experience in data engineering role Hands-on experience with tools such as Python, SQL, Spark/PySpark Understanding of data pipelines, ETL/ELT processes, and data storage solutions Exposure to data modelling and designing scalable data structures Background or grounding ...

performance, cost efficiency, and operational stability Advanced programming skills in Python and SQL, including working with large, distributed datasets (experience with Spark/PySpark is a strong advantage) Hands-on experience with Infrastructure as Code and CI/CD in cloud environments (CDK, Terraform, or CloudFormation; automated deployments ...