18 of 18 Remote/Hybrid Contract PySpark Jobs

data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience with modern data warehousing ...

Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data ...

+ environment configurations. Analysing current Data Engineering solutions + identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps ...

enterprise-scale analytics environments Technical Skills Advanced SQL/T-SQL Python for data engineering, automation, and transformation DAX for Power BI semantic modelling PySpark/Spark (highly desirable) Microsoft Azure ecosystem knowledge Our client is looking for someone who combines strong technical depth with excellent communication and stakeholder ...

enterprise-scale analytics environments Technical Skills Advanced SQL/T-SQL Python for data engineering, automation, and transformation DAX for Power BI semantic modelling PySpark/Spark (highly desirable) Microsoft Azure ecosystem knowledge Our client is looking for someone who combines strong technical depth with excellent communication and stakeholder ...

Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Configured and validated Foundry Databricks connector. ...

travel to Leeds/Newcastle) Clearance: Active SC Clearance required Rate: Up to £425/day (Inside IR35) Tech Stack: AWS (EMR, S3, Lambda), PySpark, Python, SQL, CI/CD Focus: Building scalable pipelines, modernising ETL, supporting MI/Power BI Level: Mid to Senior - mentoring & architectural input expected ...

Palantir Foundry experience, especially Databricks connector, Data Connection, datasets, syncs, projects and permissions. Secure data-sharing, data governance, lineage and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Secure AWS S3 and Databricks access ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

schema design Develop API connectors and ingestion pipelines using Python Implement integration services feeding into the Databricks environment Build data transformation logic using PySpark/Python (data cleansing, standardisation, validation) Configure monitoring, logging, and alerting Execute unit and integration testing across pipelines Experience: Azure Databricks Python Financial Services sector ...

schema design Develop API connectors and ingestion pipelines using Python Implement integration services feeding into the Databricks environment Build data transformation logic using PySpark/Python (data cleansing, standardisation, validation) Configure monitoring, logging, and alerting Execute unit and integration testing across pipelines Experience: Azure Databricks Python Desirable - any exposure ...

mentoring, and peer-reviewing the work of junior colleagues. What We Are Looking For The Core Tech Stack: Advanced proficiency in Python (most critical), PySpark, SQL, and Snowflake. Experimentation Expertise: Strong, demonstrable working knowledge of experiment design, A/B testing methodologies, and randomisation. Data Visualisation: Expert capability ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

connectivity issues across AWS, Databricks and downstream integrations. Technical Environment: AWS, Databricks, Unity Catalog, Palantir Foundry, Terraform, IaC, Delta, Parquet, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

Job Title: Technical Delivery Lead - Data Engineering Lead (Hands-on Python/Databricks) Contract: 6 months - Potentially extension. Location: London Hybrid (5 days in 10 in the office) Rate: Highly competitive Status: Inside IR35 Working ...

within Azure cloud environments. Proven hands-on expertise with Azure Databricks in enterprise-scale production environments. Strong programming skills using: Python/SQL/PySpark/Spark SQL. Experience designing and building scalable data pipelines and data processing frameworks. Strong understanding of modern Lakehouse architecture principles. Experience working with … technical teams to understand data requirements and translate them into scalable engineering solutions. Design, develop and maintain high-performance data pipelines using Azure Databricks, PySpark, Spark SQL and Python. Contribute to the design and implementation of Lakehouse architecture using Delta Lake and Databricks best practices. Develop and manage data ...

reporting perspective *Design and build data pipelines and datasets for dashboards *Identify and mitigate data quality risks *Develop reusable code in Python and PySpark for data prep and analysis *Collaborate closely with Data Engineers, Analysts and UCD professionals *Work within modern engineering practices including CI/… strong engineering culture What I need from you (essential) To be considered, your CV must clearly show experience with the following: *Palantir (FDP) *Python *PySpark *Exploratory Data Analysis *Data pipeline design/data engineering concepts *Dashboard/reporting dataset development *BI tools (Redshift, Databricks or similar) *Agile delivery environments ...

digital Screening service. You will build and maintain data pipelines that ingest, transform and prepare data for reporting, developing ETL processes using PySpark and platform tooling. You will work within an Agile multidisciplinary team alongside Data Scientists, Analysts and UCD professionals. This is a largely remote role. Essential skills … experience Strong experience in data pipeline engineering, ETL development and PySpark Experience with Palantir products, including the NHS Federated Data Platform (FDP) Ability to ensure data quality, reliability and performance across reporting pipelines Experience supporting testing and validation of dashboards and underlying datasets Proven ability to work independently ...