1 to 25 of 42 Permanent PySpark Jobs in the South East

working with unstructured data and NLP-related datasets. Proficiency in one programming language, preferably Python with experience in data processing libraries such as Pandas, PySpark, or Dask. Familiarity with MLOps and deploying AI/ML models into production environments. Knowledge of Retrieval-Augmented Generation (RAG) frameworks or interest in More ❯

Proven experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python, SQL, PowerShell, PySpark, Spark SQL. Good working knowledge of data warehouse and data mart architectures. Good experience in Data Governance, including Unity Catalog, Metadata Management, Data Lineage More ❯

. Some experience with Infrastructure-As-Code (Terraform). Strong skills in SQL and Python. Experience in ETL practices. Good knowledge of Apache Spark (PySpark, Spark Streaming is a plus). Familiarity with modern software engineering practices (CI/CD, Testing, Version Control). Personal Skills Strong communication and More ❯

working in cloud-native environments (AWS preferred) Strong proficiency with Python and SQL Extensive hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Familiarity with DevOps practices and infrastructure-as-code (e.g., Terraform, CloudFormation) Solid understanding More ❯

machine learning, with a strong portfolio of relevant projects. Proficiency in Python with libraries like TensorFlow, PyTorch, or Scikit-learn for ML, and Pandas, PySpark, or similar for data processing. Experience designing and orchestrating data pipelines with tools like Apache Airflow, Spark, or Kafka. Strong understanding of SQL, NoSQL More ❯

System Integration, Application Development or Data-Warehouse projects, across technologies used in the enterprise space. Software development experience using: Object-oriented languages (e.g., Python, PySpark,) and frameworks Stakeholder Management Expertise in relational and dimensional modelling, including big data technologies. Exposure across all the SDLC process, including testing and deployment. More ❯

similar tools is valuable (Collibra, Informatica Data Quality/MDM/Axon etc.) Data Architecture experience is a bonus Python, Scala, Databricks Spark and Pyspark with Data Engineering skills Ownership and ability to drive implementation/solution design More ❯

practices in data management and governance, guide in structuring cloud environments, and support data initiatives and future projects. Qualifications: Proficiency in Databricks, Python/PySpark, and SQL/SparkSQL. Experience with Big Data/ETL processes, preferably Spark and Databricks. Expertise in Azure cloud platform. Knowledge of version control More ❯

able to engage directly with clients and senior stakeholders. Cloud-native development – Azure, AWS, or GCP. Expertise in data pipeline design and implementation using: PySpark Databricks Snowflake Experience working in data science, AI or machine learning environments is a big plus. Understanding of data governance, quality, and security in More ❯

relational databases Experience of Azure Fabric and its use in Data engineering and Data management. A high degree of proficiency with tools like Terraform, PySpark, and Databricks. Understanding of data migration concepts, including mapping, transformation, cleansing, and validation. Strong attention to detail and problem-solving ability. Must be comfortable More ❯

relational databases Experience of Azure Fabric and its use in Data engineering and Data management. A high degree of proficiency with tools like Terraform, PySpark, and Databricks. Understanding of data migration concepts, including mapping, transformation, cleansing, and validation. Strong attention to detail and problem-solving ability. Must be comfortable More ❯

relational databases Experience of Azure Fabric and its use in Data engineering and Data management. A high degree of proficiency with tools like Terraform, PySpark, and Databricks. Understanding of data migration concepts, including mapping, transformation, cleansing, and validation. Strong attention to detail and problem-solving ability. Must be comfortable More ❯

Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong More ❯

coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques More ❯

key role in building their scalable data capabilities; designing ingestion pipelines, high-performance APIs, and real-time data processing systems. Key Responsibilities Stack: Python, PySpark, Linux, PostgreSQL, SQL Server, Databricks, Azure, AWS Design and implement large-scale data ingestion, transformation, and analysis solutions Model datasets and develop indicators to More ❯

queries for huge datasets. Has a solid understanding of blockchain ecosystem elements like DeFi, Exchanges, Wallets, Smart Contracts, mixers and privacy services. Databricks and PySpark Analysing blockchain data Building and maintaining data pipelines Deploying machine learning models Use of graph analytics and graph neural networks If this sounds like More ❯

skills: Typical Data Engineering Experience required (3+ years): Strong knowledge and experience: Azure Data Factory and Synapse data solution provision Azure Devops PowerBi Python PySpark (Preference will be given to those who hold relevant certifications) Proficient in SQL. Knowledge of Terraform Ability to develop and deliver complex visualisation, reporting More ❯

in programming languages and data structures such as SAS, Python, R, SQL is key. With Python background, particularly familiarity with pandas/polars/pyspark, pytest; understanding of OOP principles; git version control; knowledge of the following frameworks a plus: pydantic, pandera, sphinx Additionally, experience in any or all More ❯

Engineering Experience required ACTIVE SC is mandatory Essential requirement: Azure Data Factory and Synapse data solution provision Azure DevOps Microsoft Azure PowerBi Python misson Pyspark Dimension Data Model Semantic Data Models, including integration to Power BI Data Engineering Capabilities Business analysis to understand service needs and and documents accurately More ❯

effectiveness, including Azure DevOps. Considerable experience designing and building operationally efficient pipelines, utilising core Azure components, such as Azure Data Factory, Azure Databricks and Pyspark etc. Proven experience in modelling data through a medallion-based architecture, with curated dimensional models in the gold layer built for analytical use. Strong More ❯

in Computer Science, Statistics, Data Science, or a related field. Extensive experience in clinical data programming and Data Visualization. Strong experience with Python programming (PySpark, Databricks, pandas, Numpy) is mandatory for this role. Good SDTM Knowledge. Strong SAS/R Programming and SQL expertise. Proficiency in CDISC SDTM standards. More ❯

Knowledge of Data Warehouse/Data Lake architectures and technologies. Strong working knowledge of a language for data analysis and scripting, such as Python, Pyspark, R, Java, or Scala. Experience with any of the following would be desirable but not essential; Microsoft's Fabric data platform, Experience with ADF More ❯

Analyst: Must hold experience with data analytic techniques, ideally applied to real-life objects. Must hold a year's professional experience using Python/Pyspark/Pandas. Experience with processing data and working with databases/datalakes (SQL). Strong understanding of data manipulation, analysis and processing. Ability to More ❯

following areas: Data Warehousing (Databricks) Data Modelling (Medallion Arch, Facts/Dimensions) Azure Data Stack (DataFactory/Synapse) Visualisation (PowerBI) Coding Best Practice (Python, PySpark) Real-Time Processing (SQL) Insurance/Finance Experience Startup experience Leadership/Line Management experience. What’s in it for you? Remote First Working More ❯

sprint planning sessions. Monitor data pipeline executions and investigate test failures or anomalies. Document test results, defects, and quality metrics. Preferred qualifications: Experience with PySpark or notebooks in Databricks. Exposure to Azure DevOps, Unit Testing frameworks, or Great Expectations for data testing. Knowledge of data warehousing or medallion architecture More ❯