26 to 50 of 344 PySpark Jobs

Azure Data Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

data platforms, lakehouses and data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

scale data platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

Senior Data Engineer (Azure | Databricks | Fabric | PySpark | SQL) Permanent | Hybrid | UK Public Sector Project Overview We are seeking a hands-on Senior Data Engineer to join a growing Data & Analytics practice. This role focuses on designing, building, optimising, and maintaining enterprise-level data solutions. You will work across ...

Engineering across GCP, Azure and/or AWS . Hands-on experience with BigQuery, Databricks, Snowflake or Microsoft Fabric . Strong Python, SQL and PySpark development skills. Experience building scalable ETL/ELT pipelines and data integration solutions. Experience with DBT, Airflow, Azure Data Factory or Dataflow . Knowledge ...

process requirements into technical solutions and working across technical and non-technical teams. Nice To Have Skills and Experience Experience with Databricks, Spark/PySpark, Delta Lake or comparable technologies. Experience with Celonis, process mining or process intelligence. Experience with enterprise platforms such as Salesforce, SAP or ServiceNow. Exposure ...

Python, plus hands-on Git Production experience with at least one of Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow or PySpark Cloud-native orchestration and distributed data processing Infrastructure as code (e.g. Terraform) CI/CD and DevOps or DevSecOps practice A sound grasp of data architecture ...

Azure Data Lake Storage (ADLS) Azure SQL Database and SQL Server Microsoft Fabric (preferred) Advanced SQL development and performance tuning skills Experience with Python, PySpark, or Scala for data processing Knowledge of data warehousing concepts and dimensional modelling Experience with CI/CD pipelines using Azure DevOps or GitHub ...

enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform implementations Proven ability ...

enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform implementations Proven ability ...

vs. centralized models, capacity (F-SKU) sizing and licensingDesign end-to-end Microsoft Fabric solutions: OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL), Real-Time Intelligence (Eventstreams, Eventhouse/KQL) and Power BI + Power BI CoPilotDesign governance, security and operating frameworks: data governance ...

Strong problem-solving and analytical skills. Desirable Experience with Microsoft Fabric. Experience with Azure Data Services. Working with multi-currency financial information. Knowledge of PySpark or Spark SQL. Understanding of Microsoft Entra ID and security frameworks. Experience with DevOps and automated deployment pipelines. Knowledge of financial, legal, or professional ...

data platform on Microsoft Fabric, Databricks, Snowflake or similar, using a medallion architecture, including managing ingestion performance and forecasting platform costs. Strong Python/PySpark and SQL, with a track record of production ETL/ELT pipelines integrating data from varied business systems, including less well-documented sources. Experience ...

/CD deployment practices. Strong problem-solving and analytical skills.DesirableExperience with Microsoft Fabric. Experience with Azure Data Services.Working with multi-currency financial informationKnowledge of PySpark or Spark SQL.Understanding of Microsoft Entra ID and security frameworks.Experience with DevOps and automated deployment pipelines.Knowledge of financial, legal, or professional services data environments.Elite ...

Strong problem-solving and analytical skills. Desirable Experience with Microsoft Fabric. Experience with Azure Data Services. Working with multi-currency financial information Knowledge of PySpark or Spark SQL. Understanding of Microsoft Entra ID and security frameworks. Experience with DevOps and automated deployment pipelines. Knowledge of financial, legal, or professional ...

Strong problem-solving and analytical skills. Desirable Experience with Microsoft Fabric. Experience with Azure Data Services. Working with multi-currency financial information Knowledge of PySpark or Spark SQL. Understanding of Microsoft Entra ID and security frameworks. Experience with DevOps and automated deployment pipelines. Knowledge of financial, legal, or professional ...

/CD deployment practices. Strong problem-solving and analytical skills.DesirableExperience with Microsoft Fabric. Experience with Azure Data Services.Working with multi-currency financial informationKnowledge of PySpark or Spark SQL.Understanding of Microsoft Entra ID and security frameworks.Experience with DevOps and automated deployment pipelines.Knowledge of financial, legal, or professional services data environments.Elite ...

scale data platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

diversity of views and opinions. Key Skills & Experience: Proven experience in a Data Engineering role within the insurance industry Advanced SQL and PySpark skills, with hands-on experience building data transformations and data products Demonstrable data modelling experience, including practical experience with medallion architecture, and comfortable contributing technical input ...

Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting structured/unstructured source data into validatd Delta tables. Using Power ...

Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting structured/unstructured source data into validatd Delta tables. Using Power ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You’ve worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...