20 of 20 Contract PySpark Jobs in London

Analytics Azure Data Lake Storage Gen2 Delta Lake Lakehouse Architecture Data Engineering ETL/ELT pipeline design and implementation Data transformation using SQL and PySpark Large-scale structured and unstructured data processing Data modelling and warehouse design Real Time and batch processing architectures Cloud Architecture End-to-end solution ...

projects. Hands-on AWS experience. Experience working within Azure environments. Strong SQL skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure ...

Duration: 5 months Essential Skills: Informatica Data project experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data ...

analytics development (circa 4+ years) Hands-on expertise with Microsoft Fabric or closely related technologies (Synapse, ADF, Power BI Premium) Strong Python or PySpark skills for large-scale data processing Solid SQL/T-SQL capabilities and experience working with data warehouses Good understanding of P&C insurance data ...

essential - More to React JS and Java script Back-end: Expertise in C# .NET Core, REST APIs, and microservices-based development is essential Python (pyspark): Experience with data pipelines and analytics is a plus. Power BI: Familiarity with developing dashboards and data models. Architecture: Knowledge of microservices, API design ...

enterprise-scale analytics environments Technical Skills Advanced SQL/T-SQL Python for data engineering, automation, and transformation DAX for Power BI semantic modelling PySpark/Spark (highly desirable) Microsoft Azure ecosystem knowledge Our client is looking for someone who combines strong technical depth with excellent communication and stakeholder ...

Palantir Foundry experience, especially Databricks connector, Data Connection, datasets, syncs, projects and permissions. Secure data-sharing, data governance, lineage and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Secure AWS S3 and Databricks access ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

leading Professional Services firm is looking to hire a Finance Data Engineering & Analytics Lead to join their team. You will develop Python/PySpark-based data pipelines, build data models and datasets for finance, actuarial and risk reporting. You will also develop analytics, dashboards and data validation tooling. ...

data platforms and cloud environments Collaborate with engineering and architecture teams to deliver high-quality solutions Key Skills & Experience Strong experience with Python and PySpark Hands-on expertise in AWS Experience working with Databricks Ideally 5+ years working with this tech stack as a data engineer Proven background delivering ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

Technical Environment : Palantir Foundry, Foundry Virtual Tables, Ontology, Workshop, Pipeline Builder, Code Repositories, AWS, Databricks, Terraform, IaC, Delta Lake, Parquet, Iceberg, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

connectivity issues across AWS, Databricks and downstream integrations. Technical Environment: AWS, Databricks, Unity Catalog, Palantir Foundry, Terraform, IaC, Delta, Parquet, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

connectivity issues across AWS, Databricks and downstream integrations. Technical Environment: AWS, Databricks, Unity Catalog, Palantir Foundry, Terraform, IaC, Delta, Parquet, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

ingestion frameworks for enterprise data sources, including: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Develop and optimize Apache Spark workloads using PySpark or Scala, ensuring performance, scalability, and operational efficiency. Assess existing or partially completed data platform implementations and provide remediation, optimization, and modernization recommendations. Collaborate … Warehouses Experience integrating enterprise data sources such as: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Strong Apache Spark expertise using PySpark and/or Scala, including performance tuning, partitioning strategies, and workload orchestration. Experience evaluating and remediating incomplete, underperforming, or legacy data platform implementations. Strong stakeholder ...

digital Screening service. You will build and maintain data pipelines that ingest, transform and prepare data for reporting, developing ETL processes using PySpark and platform tooling. You will work within an Agile multidisciplinary team alongside Data Scientists, Analysts and UCD professionals. This is a largely remote role. Essential skills … experience Strong experience in data pipeline engineering, ETL development and PySpark Experience with Palantir products, including the NHS Federated Data Platform (FDP) Ability to ensure data quality, reliability and performance across reporting pipelines Experience supporting testing and validation of dashboards and underlying datasets Proven ability to work independently ...

role in developing and deploying finance and actuarial data pipelines, supporting reporting, analytics and reconciliation capabilities. Key responsibilities include: Developing Python/PySpark data pipelines within cloud platforms Building datasets and models for finance, actuarial and risk reporting Integrating with existing data ingestion and orchestration frameworks Supporting reconciliation, validation … Collaborating with design and architecture teams to deliver aligned solutions Experience Required Strong experience as a Data Engineer within insurance (essential) Proficiency in Python, PySpark and Spark-based processing Experience working with Databricks or similar cloud data platforms Exposure to finance, actuarial or regulatory reporting data Ability to operate ...

models. This isn't just a 'high-level' role-we need someone who can design the blueprint and then dive into the Python/PySpark code to ensure the foundation is rock solid. Key Responsibilities: Architect & Design: Lead the implementation of on-premises, cloud, and hybrid AI solutions. GenAI … certification. AI Specialization: Proven experience with Prompt Engineering, Azure OpenAI, Document Intelligence (Form Recognizer), and Cognitive Search. Data Engineering: Expert proficiency in Python and PySpark for large-scale data processing. Operations: Familiarity with MLOps or LLMOps frameworks to support production-grade AI. Seniority: 15+ years of total IT experience ...

models. This isn't just a "high-level" role-we need someone who can design the blueprint and then dive into the Python/PySpark code to ensure the foundation is rock solid. Key Responsibilities: Architect & Design: Lead the implementation of on-premises, cloud, and hybrid AI solutions. GenAI … certification. AI Specialization: Proven experience with Prompt Engineering, Azure OpenAI, Document Intelligence (Form Recognizer), and Cognitive Search. Data Engineering: Expert proficiency in Python and PySpark for large-scale data processing. Operations: Familiarity with MLOps or LLMOps frameworks to support production-grade AI. Seniority: 15+ years of total IT experience ...