1 to 25 of 29 PySpark Jobs in the South East

transformation and orchestration pipelines, integrating data from cloud, on‐premises and third‐party business systems. Strong hands‐on capability with SQL, Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation ...

Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats.Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview.Data Integration ...

standards and best practices. About you Extensive experience designing and delivering enterprise-scale data engineering solutions Strong hands-on experience with Azure Databricks, including PySpark, Delta Live Tables and Databricks Workflows and orchestration Microsoft Fabric experience, including Lakehouse, Data Factory, Warehouse and Spark Knowledge of Azure Data Platform services ...

pipelines, stored procedures and reporting solutions to cloud-based platforms. Exposure to Azure DevOps, Git and CI/CD practices. Experience with Python, PySpark, Power BI or real estate technology platforms would be advantageous. Previous exposure to commercial real estate, investment management or alternative asset management environments is highly ...

strong Python/Data Engineering professionals with a solid Azure foundation and the ability to learn and work across the wider platform. Databricks/PySpark Infrastructure as Code/DevOps automation AI/ML data integration Automotive or insurance industry experience #J-18808-Ljbffr ...

Python/Data Engineering professionals with a solid Azure foundation and the ability to learn and work across the wider platform. Desirable Databricks/PySpark NoSQL/Cosmos DB Infrastructure as Code/DevOps automation AI/ML data integration Automotive or insurance industry experience If this sounds ...

EXPERIENCE You should have: Previous experience leading or managing Data Engineers Strong consulting or client-facing experience Advanced SQL and Python skills Experience with PySpark Experience with Azure, Microsoft Fabric and Databricks (or strong cloud data engineering experience) CI/CD and Git experience Power BI knowledge Experience building ...

Strong hands-on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud experience across ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

solutions Strong stakeholder engagement and communication skills Agile delivery experience Desirable Experience Financial Services, Banking or Investment Management experience Microsoft Fabric Databricks Python or PySpark Data migration and transformation programmes Team leadership or mentoring experience Contract Details £650-£750 per day Inside IR35 6-Month Initial Contract Hybrid Working ...

development of scalable ETL/ELT pipelines. Proven experience working with SQL, relational databases and large datasets. Experience with data processing technologies such as PySpark, Spark or Pandas. Strong understanding of data quality, transformation and integration. Experience with Git, CI/CD and automated testing. Comfortable working in Linux ...

Looking For Strong SQL Data Engineering or Analytics Engineering experience Databricks or comparable platforms such as Snowflake, Hadoop or SQL Server Python and ideally PySpark ETL/ELT and data modelling Strong problem-solving and critical-thinking skills Good understanding of data architecture and infrastructure Ability to work independently ...

supporting ETL/ELT frameworks. Strong understanding of data warehousing concepts and dimensional modelling. AWS experience, including services such as S3, Glue, EMR, or PySpark . Knowledge of data governance, quality controls, and testing practices. Experience working within large-scale enterprise environments. Desirable Knowledge of Iceberg or modern data ...

management (Metadata Driven Ingestion). Good working experience of database technologies and database integration, such as SQL, Postgres, MySQL or Oracle. Experience in Python (PySpark) Experience in working with DevOps capabilities like version control, automated builds, testing and release management capabilities using Microsoft DevOps and Git repositories. Good numerical … technology roles, such as architecture, operations, and design related disciplines. At least three years of experience in Azure (Azure Synapse, Azure Databricks, Fabric, PySpark). Comprehensive experience with data modelling Experience across a range of other IT functional disciplines will be of benefit, including: Data Platform tools CI/ ...

data pipelines using Databricks, and develop ETL processes that prepare data for reporting and analytics. The role requires strong Databricks, SQL and Python/PySpark expertise, plus experience with relational databases and data modelling. #J-18808-Ljbffr ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

data modelling experience. Strong understanding of ETL/ELT architecture and data integration frameworks. AWS experience including services such as S3, Glue, EMR, and PySpark . Experience defining data standards, governance frameworks, and architectural roadmaps. Desirable Experience with Iceberg and modern lakehouse architectures. Exposure to AI-enabled data platforms ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark ark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish ...

maintaining data pipelines in Databricks Developing and optimising ETL/ELT processes across ingestion, transformation and integration Building data solutions in SQL, Python and PySpark Monitoring and improving data quality, reliability and platform performance Supporting SQL database administration, maintenance and optimisation Tuning queries, indexes and execution plans where performance … mature automation, documentation, development standards and CI/CD practice Essential Skills and Experience Strong experience with Databricks SQL server experience Solid Python and PySpark skills Experience using MS stack, Azure and Spark Familiarity with CI/CD and DevOps Why Apply? Base salary up to £60,000 Work ...

data modelling, semantic layers and Power BI delivery. You'll have the opportunity to: ·Design and build production-grade pipelines using Azure Databricks, Python, PySpark and SQL ·Develop scalable Lakehouse architecture using Delta Lake and Unity Catalog ·Build data and BI products from scratch and own them through … Claude and OpenRouter to support development, code review and documentation To succeed, you'll need: ·Strong production experience with Azure Databricks ·Advanced Python, PySpark and SQL skills ·Advanced Power BI, DAX and data-modelling experience ·Experience building and owning complex data or BI solutions end to end ·Confidence translating ...

using modern data engineering patterns Develop robust orchestration workflows for dependable data ingestion, transformation, quality checks, and downstream consumption Apply strong SQL, Python, and PySpark skills to transform complex data into reliable, reusable, and performant data products SQL, Data Modelling & Analytics Enablement Create well-modelled, trusted datasets that support … analytics foundations Advanced SQL skills with the ability to design performant queries, data models, and transformation logic Hands‐on knowledge of Python and PySpark for large‐scale data processing and automation Curious, hands‐on problem solver who can engage with business stakeholders to understand the real requirement and deliver ...

modelling, Kimball, or Data Vault Exposure to modern cloud data platforms, ideally within the Microsoft ecosystem Familiarity with tools such as DBT, Python, or PySpark is beneficial Experience working in data transformation or modernisation programmes Understanding of architecture principles or a clear interest in developing into a Data Architect ...

/ELT processes for data ingestion and transformation. Supporting and enhancing a business-critical reporting and analytics platform. Developing SQL and Python/PySpark-based data solutions. Managing and optimising SQL database environments, ensuring performance, reliability and scalability. Collaborating with Data Analysts, BI professionals and stakeholders across the organisation. … skills and experience working with complex relational databases. Hands-on experience building and supporting ETL/ELT pipelines. Strong Python and/or PySpark development experience. Experience with SQL Server or similar database technologies. Excellent problem-solving skills and the ability to engage effectively with both technical ...