51 to 75 of 322 Permanent PySpark Jobs

Data Lakes, Blob Storage, Lakehouses and Data Modelling concepts (Kimball, star schemas) Strong ability to write Python code, including experience with data manipulation libraries (PySpark, Pandas etc) Strong ability to write SQL queries for data retrieval and manipulation Strong problem-solving skills to address and resolve data-related issues. ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

platform such as Databricks, Snowflake or Fabric, and orchestration tooling, with practical experience of ETL/ELT, data testing, Git and CI/CD; PySpark or dbt experience is beneficial. Working knowledge of finance data structures, including chart of accounts, entity hierarchy, sub-ledger to ledger relationships, intercompany ...

relevant discipline or equivalent practical experience. Desirable Experience Strong hands-on SQL and Python skills, with experience in distributed data processing using PySpark and/or Databricks. Experience with AWS services, APIs, event-driven integration and infrastructure as code. Exposure to AI/ML data preparation, data observability, metadata ...

degree in a relevant discipline or equivalent practical experience. Strong hands-on SQL and Python skills, with experience in distributed data processing using PySpark and/or Databricks. Experience with AWS services, APIs, event-driven integration and infrastructure as code. Exposure to AI/ML data preparation, data observability ...

Looking For Proven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions. Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks. Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within ...

role: Strong experience with modern data engineering and reporting, data storage, data pipelines and analytical data reporting. Extensive knowledge using Python, Spark, SQL, PySpark, PowerBI Hands-on client facing roles, providing techical advise to clients or solving engineering challenges. Strong background in Software Engineering including Front-end technologies such ...

role: Strong experience with modern data engineering and reporting, data storage, data pipelines and analytical data reporting. Extensive knowledge using Python, Spark, SQL, PySpark, PowerBI Hands-on client facing roles, providing techical advise to clients or solving engineering challenges. Strong background in Software Engineering including Front-end technologies such ...

/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview. ...

Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - London Skills Python & SQL: Advanced Python (PySpark, Advanced SQL (CTEs, Databricks We are looking for an experienced Principal Data Engineer to take technical ownership of our customer decisioning pipeline architecture, end-to-end orchestration, and production ...

only on ETL. However, we will consider candidates who bring Cloud + CI/CD experience alongside technologies such as: Proficient SQL; familiarity with PySpark is beneficial dbt Microsoft Fabric SQLMesh 5. Core Engineering Skills Strong Python, with good documentation and unit testing practices Comfortable in an agile delivery ...

architecture Experience with cloud data platforms (AWS preferred - Redshift, Athena, S3, Glue. Azure accepted) Understanding of workflow orchestration and automation tools Python experience (including PySpark) is beneficial but not essential Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

consuming data for analytical purposes Awareness of CI/CD via pipelines, Infrastructure-as-Code principles Database performance tuning, query optimisation Knowledge of Python, PySpark Experience of working in an Agile environment Experience working with live production systems Understanding/experience of the insurance industry Experience with AI-assisted ...

successfully managing and leading multi-disciplinary engineering teams. Solid understanding of the software and data delivery lifecycle. Technical Skills Strong experience with Databricks (PySpark/Spark SQL, Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows). Working knowledge of Microsoft Dynamics 365 data structures, Dataverse, and common integration patterns ...

enterprise-scale data products, defining technical strategy, architectural patterns and engineering standards while providing technical leadership across teams. Expert programming experience with Python/PySpark and advanced proficiency in SQL for large-scale data transformation, optimisation and analytics workloads. Extensive experience designing and implementing scalable data solutions on cloud ...

database technologies including data lakes such as Azure SQL or SQL Server, PostgreSQL, Snowflake, MySQL, Cosmos DB or Mongo DB with Python and Pyspark Building data warehouse or data lakes with SCVs or in conjunction with CDP platforms Experience of ELT/ETL tools such as Databricks, Azure Data ...

Azure Synapse, Azure Data Factory, and Databricks. Knowledge of SSRS, SSAS, SSIS, DAX, or MDX. Experience with Azure DevOps and Git. Exposure to Databricks, PySpark, or advanced analytics technologies. Microsoft Data and AI certifications, particularly Microsoft Fabric certifications. What can we offer you? Healthcare cash plan. This will give ...

documentation Act as a technical leader and mentor , raising standards across the data engineering community Essential skills & experience Deep expertise in Databricks (Spark/PySpark, Delta Lake, Unity Catalog, Workflows) Strong experience building ETL/ELT pipelines , both batch and streaming Advanced Python and SQL skills Hands-on experience ...

have: Proven experience as a Data Engineer working with large, complex datasets. Hands-on expertise with Apache Spark, and SQL. Strong proficiency in Python (PySpark preferred). Experience with AWS cloud services including S3, Glue, Lambda, IAM. Familiarity with ETL design, data modelling, and data lake/lakehouse concepts. ...

experience in data engineering, including 3+ years in a tech lead capacity Strong hands-on expertise in Databricks for enterprise-scale workloads Proficiency in PySpark, Spark Structured Streaming, and Delta Lake Solid programming skills in Python and demonstrated SQL performance optimization experience Deep understanding of Azure Data Platform services ...

patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data ...

ecosystems, customer engagement platforms, event tracking, identity resolution, and customer data models. Experience building and optimising batch and streaming data pipelines using SQL, Python, PySpark, and Delta Lake. Familiarity with dbt for data transformations and orchestration tools such as Airflow, Dagster, or Prefect. Experience designing and delivering secure, resilient ...

Databricks Apps. Databricks certification at Professional level, or Associate level with the drive to progress (see Certifications). Technical skills Python and SQL, including PySpark and Spark SQL; scripting in Bash. Infrastructure as code with Terraform (Databricks and AzureRM providers), plus Declarative Automation Bundles, the Databricks ...

depth working knowledge of SQL, with the ability to write and optimise queries and stored procedures. Experience building robust data pipelines using Python, PySpark or other relevant programming languages. Solid hands-on experience of designing and developing Microsoft Fabric solutions across pipelines, lakehouses, warehouses and data layers. Strong data ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...