51 to 75 of 104 Permanent PySpark Jobs

direction, maintaining standards, and driving output Designing and delivering data solutions across platforms, including low/no-code tooling and code-based approaches (Python, PySpark, SQL) Overseeing the development of user-facing applications and workflows using TypeScript and React Acting as the primary client interface, shaping requirements and translating ...

growing data team What they’re looking for: Strong experience with modern data platforms (Microsoft Fabric a big plus) Solid skills in Python/PySpark and SQL Experience integrating data from APIs and SaaS platforms A problem-solver who enjoys working at the intersection of tech and business ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/Silver/Gold data layers Embedding data quality, validation and monitoring within pipelines Supporting … Data Engineering roles Strong experience working with Azure or AWS cloud platforms Commercial, hands-on experience using Databricks in production Proven capability across: PySpark/Python/SQL Cloud data storage (ADLS or S3) ETL/ELT pipeline development Solid understanding of data architecture, modelling, and governance Experience building ...

governed data solutions using modern cloud technologies. The Role Own the end-to-end design, build, optimisation, and support of scalable Spark/PySpark data pipelines (batch and streaming) Define and implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design … solutions Required Skills & Experience Strong experience as a Senior or Lead Data Engineer with ownership of end-to-end data solutions Expertise in Databricks, PySpark/Spark, SQL, and Python Proven experience building and optimising large-scale data pipelines in production environments Strong knowledge of cloud data architectures , particularly ...

frameworks Engineering Delivery Design and evolve scalable data pipelines (batch and incremental watermark-based ingestion) using: Microsoft Fabric Pipelines Azure Data Factory PySpark/Python Fabric Notebooks Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses Ensure Direct Lake-ready semantic models for analytics Lead incident … Hands-on experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models Strong Python and PySpark development capability Solid SQL skills for warehouse development and metadata layers Proven ownership of production data systems end-to-end Wider Azure data platform ...

/PnL/Trade lifecycle/Reconciliation Understanding of financial products such as: Derivatives (swaps, futures, options), bonds Good-to-Have Exposure to PySpark or large-scale data processing Experience working with quants or pricing models Basic scripting (Unix/Bash) Prior experience in global banking environments Experience ...

individual ownership of client problems, handling everything from deep data engineering that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos – to building user‐facing front‐end applications and widgets in TypeScript or React Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER – ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

governed data solutions using modern cloud technologies. The Role Own the end-to-end design, build, optimisation, and support of scalable Spark/PySpark data pipelines (batch and streaming) Define and implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design … solutions Required Skills & Experience Strong experience as a Senior or Lead Data Engineer with ownership of end-to-end data solutions Expertise in Databricks, PySpark/Spark, SQL, and Python Proven experience building and optimising large-scale data pipelines in production environments Strong knowledge of cloud data architectures , particularly ...

data loss. Enable CDC-based replication and minimal downtime migrations across heterogeneous systems. Design and optimize ETL/data pipelines using AWS Glue, PySpark, or EMR. Architect and implement scalable, secure, and highly available data platforms on AWS. Drive schema conversion and modernization strategies (heterogeneous migrations where applicable). … cloud or cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data lakes and Glue Data Catalog. Strong ...

data loss Enable CDC-based replication and minimal downtime migrations across heterogeneous systems Design and optimize ETL/data pipelines using AWS Glue, PySpark, or EMR Architect and implement scalable, secure, and highly available data platforms on AWS Drive schema conversion and modernization strategies (heterogeneous migrations where applicable) Optimize … data integration), Amazon Redshift (data warehousing) Experience designing and implementing data validation and reconciliation frameworks Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent) Deep proficiency in SQL and data modeling concepts Experience working with S3-based data lakes and Glue Data Catalog Strong understanding ...

with ability to run various data analytics tasks and queries. Exposure to Java script technologies (preferably React) Exposure to Azure Databricks environment is desirable (PySpark, notebooks, integrate repos, job workflows, delta lake, unity catalog etc) Exposure to azure environment and various interconnected azure components (Functions, App Service, Containers etc. … Risk management is highly desirable Qualifications 8+ years of prior experience as a developer in the required technologies CORE TECHNOLOGIES Scripting : Python, SQL, Bash, PySpark Web frameworks: Django, Flask, or FastAPI. Frontend Technologies: ReactJS, Java Script Azure : Azure Databricks, Azure Synapse Analytics, Azure Functions, Azure Data Lake Storage Gen2 ...

ownership of data pipelines and automation of data workflows. Finally have 5 years Strong programming experience at an advanced level in Python, PySpark, and SQL scripting. Benefits Salary: 50,000 - 65,000 per annum 25 day's holiday Company car/Allowance Variable annual bonus based 5-15% Pension … leveraging tools such as Foundry Code Workbooks, Pipelines, and Object Explorer for efficient data processing and integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability ...

strategies for performance and scalability Collaborate with data scientists, analysts and DevOps teams to deliver end-to-end solutions Contribute hands-on development using PySpark, SQL and Python Support CI/CD and DevOps practices using Git-based version control Present architectural designs to governance and review forums Essential … governance, metadata and security frameworks Experience working across enterprise-scale cloud data landscapes Strong understanding of performance optimisation and data management strategies Experience with PySpark, SQL and Python Spark Desirable Skills for the Databricks Solution Architect: Strong stakeholder management and communication skills Experience working within Agile delivery environments Experience ...

properly, you’ll fit right in. You’ll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines using Azure DevOps Cross‐functional teams in underwriting, actuarial, delegated authority, exposure … have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge of data governance, lineage, access controls and FinOps Confident communication skills ...

scalable, high-quality data platform from the ground up. The Role Design and build scalable ETL/ELT pipelines in Microsoft Fabric using PySpark and SQL Develop transformation logic within Fabric Lakehouses using Delta Lake and medallion architecture principles Integrate data from ecommerce, ticketing, and marketing platforms into … best practices in data engineering, documentation, and version control Requirements Strong hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, pipelines) Advanced SQL and strong PySpark experience Proficiency in Python for data processing and API integration Experience building Power BI models and dashboards for business reporting Strong understanding ...

accurate documentation, lineage and operational run books to support ongoing platform support Key Responsibilities Develop and maintain data pipelines using Azure Databricks (Python, PySpark and SQL) Implement ingestion and transformation across Bronze, Silver and Gold layers using Delta Lake Support and operate Azure Databricks and Azure Synapse platforms, including … pipelines and code reviews What you'll need to succeed Essential Proven experience as a Data Engineer Strong hands-on experience with AzureDatabricks, Python, PySpark and SQL Experience supporting Azure data platforms in production Understanding of medallion architecture and Delta Lake Desirable Experience with Azure Synapse Analytics Experience with ...

Novatus Global is a Series B scale-up RegTech SaaS provider and boutique advisory firm, helping financial institutions manage their most complex regulatory requirements. We combine deep consulting expertise with cutting-edge SaaS solutions, enabling ...

solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards. Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services. Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing … Data Engineering, or a related field. Proven experience designing and building cloud-based data platforms, ideally within Azure. Strong hands-on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory. Solid understanding of Data Lakehouse architecture and modern data platform design. Proficiency in Python for data engineering, automation ...

Novatus Global is a Series B scale-up RegTech SaaS provider and boutique advisory firm, helping financial institutions manage their most complex regulatory requirements. We combine deep consulting expertise with cutting-edge SaaS solutions, enabling ...

Role Overview The role is responsible for designing, building & maintaining robust data architectures that integrate SAP and non-SAP systems to support enterprise-wide data management, analytics, and reporting needs. This role ensures seamless data ...