1 to 25 of 382 Permanent PySpark Jobs

optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow Experience with containerisation ...

Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell) Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software ...

with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

transformation and orchestration pipelines, integrating data from cloud, on‐premises and third‐party business systems. Strong hands‐on capability with SQL, Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation ...

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

models in data warehouses, lakehouse environments or analytics tools such as Power BI. Familiarity with modern data engineering frameworks and tools (e.g. Spark/PySpark, Pandas, orchestration tools). Experience with cloud data platforms on AWS or Azure (e.g. Databricks, Redshift, Microsoft Fabric). Understanding of software engineering fundamentals ...

construction, programme controls, PMO, or consultancy environments. Experience with cloud-native architectures, DevOps, CI/CD pipelines, and DataOps practices. Knowledge of Python, Spark, PySpark, Databricks, or equivalent data engineering technologies. Information Management in a complex construction or infrastructure project/programme Experience developing enterprise data products and supporting ...

standards and best practices. About you Extensive experience designing and delivering enterprise-scale data engineering solutions Strong hands-on experience with Azure Databricks, including PySpark, Delta Live Tables and Databricks Workflows and orchestration Microsoft Fabric experience, including Lakehouse, Data Factory, Warehouse and Spark Knowledge of Azure Data Platform services ...

Good understanding of modern software platform architecture and the role of the data platform within the broader platform ecosystem Good Scala skills (or strong PySpark plus a genuine willingness to work primarily in Scala) Strong SQL skills for transformations, data modelling, and debugging data issues Experience building and supporting ...

/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview. ...

query and manipulate data. Proficiency in cloud data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Factory or AWS equivalents. Spark/Pyspark or Python skills. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. MS Fabric interest, skills or experience a bonus. Willingness ...

with Microsoft Fabric technologies including OneLake, Lakehouse, Warehouse, Data Factory, Spark and Power BI. Strong Engineering Skills - You're proficient in SQL, Python and PySpark, with experience developing scalable and maintainable data solutions. Modern Data Platform Experience - You've successfully built, enhanced or operated cloud-based data platforms, ideally ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

complex data workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development ...

pipelines, stored procedures and reporting solutions to cloud-based platforms. Exposure to Azure DevOps, Git and CI/CD practices. Experience with Python, PySpark, Power BI or real estate technology platforms would be advantageous. Previous exposure to commercial real estate, investment management or alternative asset management environments is highly ...

Quality, Data Integration, Data Analytics, Data Visualisation, Data Science and Business Intelligence. Confident in delivering solutions using Databricks & developing and optimising data pipelines using PySpark within Databricks environments. Proven experience building, publishing, and supporting dashboards and reports using Power BI Strong experience and programming skills in languages such ...

deep Azure Synapse/Databricks experience with clear ability to specialise quickly in Fabric. Expert SQL/T-SQL plus strong Python or PySpark, with a track record of building maintainable ELT/ETL pipelines and analytical data models. Strong Power BI and DAX experience, including semantic modelling, incremental ...

data engineering or integration environments. 8–10 years’ experience in data engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing ...

Lake Storage (ADLS). Azure SQL Database and SQL Server. Microsoft Fabric (preferred). Advanced SQL development and performance tuning skills. Experience with Python, PySpark, or Scala for data processing. Knowledge of data warehousing concepts and dimensional modelling. Experience with CI/CD pipelines using Azure DevOps or GitHub. ...

Senior Data Engineer (Azure | Databricks | Fabric | PySpark | SQL) Permanent | Hybrid | UK Public Sector Project Overview We are seeking a hands-on Senior Data Engineer to join a growing Data & Analytics practice. This role focuses on designing, building, optimising, and maintaining enterprise-level data solutions. You will work across ...

years of experience in data engineering, data platform delivery, software engineering, or related enterprise technology roles. Strong hands‐on experience with Databricks, Spark/PySpark, Python, SQL, Delta Lake, Azure, ADLS, and cloud‐based data engineering patterns. Experience designing scalable data pipelines, data products, and lakehouse solutions using modern ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

data-science role, including delivery of reliable pipelines, reusable data assets and useful analytical models in Microsoft Fabric and Databricks. Advanced Python, SQL and PySpark, including experience with data transformation, analysis and common machine-learning libraries. Hands-on experience designing and operating scalable data pipelines, lakehouse data models, Delta ...