126 to 150 of 322 Permanent PySpark Jobs

making. Key Responsibilities: Lead the design and development of curated, analytics-ready dimensional and semantic data models. Build, maintain and optimise complex SQL and PySpark transformation pipelines within Databricks. Define and maintain KPI definitions, conformed dimensions and consistent business logic across data domains. Ensure data quality, integrity, testing, observability ...

optimisation and leading delivery teams. Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments. Advanced experience with PySpark, SparkSQL and large-scale distributed data processing. Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling. Excellent SQL skills ...

taking ownership of data products or deliveries, breaking down work and managing priorities across projects. A strong understanding of CI/CD processes, SQL, pySpark and experience of modern enterprise cloud-based data environments including Azure and Databricks. Sufficient technical understanding to guide engineers and translate delivery requirements into ...

grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build … ongoing evolution of Envitia’s data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ ...

building scalable data pipelines, Databricks lakehouse architectures, and robust data platforms in a modern cloud environment. Strong expertise in Python, Databricks, Spark/PySpark and SQL is required, with hands-on delivery across ETL/ELT, data quality and CI/CD practices. Banking domain experience is a plus ...

Science, Mathematics, or a related field. Applied experience in data science, machine learning, or related areas. Strong programming skills in Python with experience in PySpark, Spark SQL, and manipulating data. Experience LLM Orchestration and Agentic AI libraries Proficiency in using GenAI models (OpenAI or similar) to solve business problems. ...

engineering best practicesYou have experience with Databricks, ideally also AzureYou have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and PandasYou're excited to join a start-up in a role that you can shape and influenceYou're collaborative with excellent communication skillsApply ...

knowledge (you don't need everything) Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

management, network design and access control; Databricks and OCI, or comparable platforms Strong Python and SQL, and experience building and monitoring production pipelines using PySpark or Spark Experience with infrastructure as code, CI/CD and a Git and GitHub workflow A track record of setting engineering standards ...

that deeply cares about its values. To use a range of different tools and technologies depending on the specific task, these include: Databricks, Python, PySpark, R, Excel, Palantir Foundry, SQL and PowerBI. Qualifications Even if you don’t meet every requirement below, feel free to still apply ...

Python for data processing, pipelines and model serving, alongside the libraries and tooling used by our data science team (such as pandas, Polars and PySpark) What we’re looking for: Significant experience as a senior or principal engineer leading the design and delivery of data‐intensive systems A track ...

Data Lake Storage (Gen2), Purview, Cosmos DB, OpenAI, AzureML, AIFoundry, Kubernetes Understanding of software engineering tools and concepts including experience in Python, Scala or PySpark, database technologies, data modelling and SQL Confident communicator who is able to explain technical terms to non‐technical audiences and mentor junior colleagues Leads ...

business stakeholders to deliver dashboards that meet Barclays standards and business needs. Some other highly valued skills may include: Knowledge of Python/PySpark and data analytics. Experience in a Call Centre or Contact Centre environment. Strong communication and stakeholder management skills. Ability to collaborate effectively across technical ...

software or computer engineering, applied math, physics, statistics, or a related field or equivalent experience Technical Skills: Strong knowledge of distributed data processing (Spark, PySpark, SQL) Expertise in data governance, metadata management, lineage, and data modeling Experience with AI/ML platforms and AI‐driven architecture Proven ability ...

/CD, monitoring, deployment patterns such as blue-green and canary) Cloud and data platforms (Azure, AWS or GCP; Databricks, Snowflake); Python, SQL, PySpark Experience leading proposals, solution architecture and commercial estimation in a consulting or services environment Confident communicator able to explain complex AI concepts to senior business ...

Arrow Global Group in Manchester seeks an experienced Data Engineer to design and implement robust data pipelines using Microsoft Fabric, PySpark, and SQL across multiple data sources in a financial services setting. The role focuses on building scalable data ingestion, transformation, and validation workflows, while collaborating with Data Stewards ...

patterns. Proficiency in data quality assurance, data governance, and query optimization. Nice to have: Familiarity with the AWS data ecosystem (S3, Glue, EMR/PySpark, Apache Iceberg). Non-Technical Skills & Competencies Proven sense of ownership, commitment, and personal accountability. Meticulous attention to detail with a proactive problem-solving ...

patterns. Proficiency in data quality assurance, data governance, and query optimization. Nice to have: Familiarity with the AWS data ecosystem (S3, Glue, EMR/PySpark, Apache Iceberg). Non-Technical Skills & Competencies Proven sense of ownership, commitment, and personal accountability. Meticulous attention to detail with a proactive problem-solving ...

quality data products while ensuring compliance with security, governance, and operational standards. Key Responsibilities: Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake. Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads. Integrate data from multiple … engineering activities, including CI/CD implementation, environment promotion, performance optimisation, and operational support. Skills we can’t do without: Azure Databricks, Apache Spark, PySpark, and Spark SQL expertise. Experience building enterprise-scale ETL/ELT pipelines within Azure environments. Strong Python and SQL development skills. Experience with Azure ...

quality data products while ensuring compliance with security, governance, and operational standards. Key Responsibilities: Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake. Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads. Integrate data from multiple … engineering activities, including CI/CD implementation, environment promotion, performance optimisation, and operational support. Skills we can’t do without: Azure Databricks, Apache Spark, PySpark, and Spark SQL expertise. Experience building enterprise-scale ETL/ELT pipelines within Azure environments. Strong Python and SQL development skills. Experience with Azure ...

pipelines using modern cloud-native data engineering technologies. Provide technical direction across data modelling, ingestion, transformation, orchestration and integration. Apply practical experience of Spark, PySpark, Python and SQL within enterprise data environments. Governance, Security & Quality Embed effective data governance, security, quality and access controls within cloud data environments. Ensure … Fabric or another cloud data platform. Strong understanding of modern data architecture patterns, data modelling and ETL/ELT approaches. Practical experience with Spark, PySpark, Python and SQL and the design of scalable batch and real-time data pipelines. Experience designing appropriate data governance, security and data quality controls. ...

analytics and data science across Veezu. Reporting into the Director of Data Engineering, you’ll work hands‐on with Microsoft Fabric, SQL, Python/PySpark and Power BI, bringing data in from our booking systems, databases and third‐party APIs and shaping it into trusted, well‐modelled datasets that … more! What you’ll do: Design, build and maintain ETL/ELT pipelines in Microsoft Fabric using Pipelines, Notebooks, Lakehouses, Warehouses and OneLake. Write PySpark notebooks to ingest and transform data from SQL Server databases, REST APIs, SaaS applications and file‐based exports. Develop and optimise SQL transformations, stored ...

analytics and data science across Veezu. Reporting into the Director of Data Engineering, you'll work hands‐on with Microsoft Fabric, SQL, Python/PySpark and Power BI, bringing data in from our booking systems, databases and third‐party APIs and shaping it into trusted, well‐modelled datasets that … more! What you’ll do: Design, build and maintain ETL/ELT pipelines in Microsoft Fabric using Pipelines, Notebooks, Lakehouses, Warehouses and OneLake. Write PySpark notebooks to ingest and transform data from SQL Server databases, REST APIs, SaaS applications and file‐based exports. Develop and optimise SQL transformations, stored ...

management (Metadata Driven Ingestion). Good working experience of database technologies and database integration, such as SQL, Postgres, MySQL or Oracle. Experience in Python (PySpark) Experience in working with DevOps capabilities like version control, automated builds, testing and release management capabilities using Microsoft DevOps and Git repositories. Good numerical … technology roles, such as architecture, operations, and design related disciplines. At least three years of experience in Azure (Azure Synapse, Azure Databricks, Fabric, PySpark). Comprehensive experience with data modelling Experience across a range of other IT functional disciplines will be of benefit, including: Data Platform tools CI/ ...

resolve data engineering challenges, supporting continuous platform improvement. Migrate data from legacy systems into Azure-based data lakes and data warehouses. Develop and maintain PySpark notebooks and data processing solutions. Improve data quality, reliability, and performance across data pipelines. Leverage AI-driven tools to enhance coding, testing, and documentation … data lakes, and end-to-end data pipelines. Proficiency in SQL, stored procedures, and working with relational and NoSQL databases. Hands-on experience with PySpark and Microsoft Fabric. Knowledge of DevOps, CI/CD, Agile delivery, and data quality best practices. Strong analytical, problem-solving, and troubleshooting skills. Experience ...