126 to 150 of 509 PySpark Jobs in the UK

AWS cloud ecosystems. Proven track record of designing and implementing data architectures that support large-scale data pipelines and analytics. Strong proficiency in Python, PySpark, and SQL for data processing and manipulation. Extensive experience in creating ETL pipelines from scratch, handling large datasets, and developing solutions that align with More ❯

their data. Overview of responsibilities in the role: Design and maintain scalable, high-performance data pipelines using Azure Data Platform tools such as Databricks (PySpark), Data Factory, and Data Lake Gen2. Develop curated data layers (bronze, silver, gold) optimised for analytics, reporting, and AI/ML, ensuring they meet More ❯

their data. Overview of responsibilities in the role: Design and maintain scalable, high-performance data pipelines using Azure Data Platform tools such as Databricks (PySpark), Data Factory, and Data Lake Gen2. Develop curated data layers (bronze, silver, gold) optimised for analytics, reporting, and AI/ML, ensuring they meet More ❯

their data. Overview of responsibilities in the role: Design and maintain scalable, high-performance data pipelines using Azure Data Platform tools such as Databricks (PySpark), Data Factory, and Data Lake Gen2. Develop curated data layers (bronze, silver, gold) optimised for analytics, reporting, and AI/ML, ensuring they meet More ❯

AWS: S3, Lambda, EMR, SMS, SQS, and additional services related to data infrastructure Terraform Databricks Data Lake, Warehouse, Lakehouse architecture and design Python/Pyspark Data platforms and notebooks: Jupyter, Databricks, Azure Gitlab: repository and CI/CD Java (Spring Boot) experience is a plus Benefits: Join a rapidly More ❯

and data lake patterns , including ingestion, governance, and quality. Strong technical skills in BigQuery, DataProc, Dataform, Composer, Pub/Sub . Fluent in Python, PySpark, and SQL . Experience with BI tools like Looker or Power BI. Strong client-facing and communication skills — able to lead conversations with both More ❯

Platforms Hands-on practical experience delivering system design, application development, testing, and operational stability Advanced knowledge in one or more programming language(s) - Python, PySpark, SQL Advanced knowledge of software applications and technical processes with considerable in-depth knowledge in one or more technical disciplines (e.g., cloud, artificial intelligence More ❯

analytics. Input into high level design and responsibility for low level design • Hands on development of data pipelines using Step functions, Glue, Python/Pyspark and DBT(Redshift). Thorough and high-quality automated Unit testing • Creation of accurate, insightful & informative technical documentation • Performance analysis & improvement Handover and upskill More ❯

their data. Overview of responsibilities in the role: Design and maintain scalable, high-performance data pipelines using Azure Data Platform tools such as Databricks (PySpark), Data Factory, and Data Lake Gen2. Develop curated data layers (bronze, silver, gold) optimised for analytics, reporting, and AI/ML, ensuring they meet More ❯

team that's transforming how data powers retail, this is your opportunity. Your Role (Key Responsibilities) Design, build, and optimise robust data pipelines using PySpark, SparkSQL, and Databricks to ingest, transform, and enrich data from a variety of sources. Translate business requirements into scalable and performant data engineering solutions More ❯

Databases such as MongoDB Experience in various messaging technologies such as Kafka Cloud Certifications including AWS Developer Associate, AWS Solutions Architect Associate Experience with PySpark Good understanding of event based architecture AI/ML field knowledge and trends Experience with Java, Big Data technologies will be a strong plus More ❯

team that’s transforming how data powers retail, this is your opportunity. Your Role (Key Responsibilities) Design, build, and optimise robust data pipelines using PySpark, SparkSQL, and Databricks to ingest, transform, and enrich data from a variety of sources. Translate business requirements into scalable and performant data engineering solutions More ❯

Mathematics, Statistics, or strong quantitative and software background. SKILLS 6+ years of hands-on experience in Python & 3+ years of hands-on experience in PySpark 6+ years of hands-on experience in using advanced SQL queries (analytical functions), experience in writing and optimizing highly efficient SQL queries Proven ability More ❯

Have experience in containerization, Azure - Azure service bus, Function Apps, ADFs Possesses knowledge on data related technologies like - Data Warehouse, snowflake, ETL, Data pipelines, pyspark, delta tables, file formats - parquet, columnar Have a good understanding of SQL, stored procedures Be able to lead development and execution of performance and More ❯

to engage both technical and non-technical stakeholders. Desirable Criteria Experience with Matillion products and competitive ETL solutions. Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza). Familiarity with version control tools such as Git, and experience with Python. Degree in Computer More ❯

and managing data sets from multiple sources. 4+ years hands-on experience in key data management technologies, including Python but not limited to SQL, PySpark, Scoop, etc. Experience working on use cases for different file formats and database management systems including NOSQL databases Conceptual understanding of data management including More ❯

and agent-based systems Our technology stack Python and associated ML/DS libraries (scikit-learn, numpy, pandas, LightGBM, LangChain/LangGraph, TensorFlow, etc ) PySpark AWS cloud infrastructure: EMR, ECS, ECR, Athena, etc. MLOps: Terraform, Docker, Spacelift, Airflow, MLFlow Monitoring: New Relic CI/CD: Jenkins, Github Actions More More ❯

understanding of DevOps and CI/CD - Understanding of compliance and lifecycle of data management Desirable Skills - Coding experience in any relevant language (Python, Pyspark etc.) - Experience working with Azure Synapse, Databricks, and/or similar technology - Experience automating test cases - Experience working with non-relational databases (Cassandra, MongoDB More ❯

and/or algorithms Our technology stack Python and associated ML/DS libraries (Scikit-learn, Numpy, LightlGBM, Pandas, LangChain/LangGraph, TensorFlow, etc ) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous More ❯

Technologies (awareness of) Azure Databricks, Data Factory, Storage, Key Vault Source control systems, such as Git dbt (Data Build Tool), SQL (Spark SQL), Python (PySpark) Certifications (Ideal) SAFe POPM or Scrum PSP Microsoft Certified: Azure Fundamentals (AZ-900) Microsoft Certified: Azure Data Fundamentals (DP-900) What’s in it More ❯

data science expertise with code-based model development e.g. R, Python Strong knowledge of deploying end-to-end machine learning models in Databricks utilizing Pyspark, MLflow and workflows Strong knowledge of data platforms and tools, including Hadoop, Spark, SQL, and NoSQL databases Communicate algorithmic solutions in a clear, understandable More ❯

We’re looking for someone who has these abilities and skills: Well established Data & Analytics work experience. Sound understanding/experience of Python, Databricks, PySpark, Spark SQL and best practices. Expertise in Star Schema data modelling. Expertise in the design, creation and management of large datasets/data models. More ❯

Ideal background: 3+ years in a Data Engineering position. Self-starter, who has experience with building data pipelines in the cloud. Excellent knowledge of PySpark, Python and SQL fundamentals. Familiar with Airflow, Databricks & BigQuery. Exposure with Palantir Foundry would be a huge plus but not essential. Ability to work More ❯

Experience: Strong proficiency in SQL and Python. Experience in cloud data solutions (AWS, GCP, or Azure). Experience in AI/ML. Experience with PySpark or equivalent. Strong problem-solving and analytical skills. Excellent attention to detail. Ability to manage stakeholder relationships effectively. Strong communication skills and a collaborative More ❯

Experience: Strong proficiency in SQL and Python. Experience in cloud data solutions (AWS, GCP, or Azure). Experience in AI/ML. Experience with PySpark or equivalent. Strong problem-solving and analytical skills. Excellent attention to detail. Ability to manage stakeholder relationships effectively. Strong communication skills and a collaborative More ❯