26 to 50 of 394 PySpark Jobs

data-science role, including delivery of reliable pipelines, reusable data assets and useful analytical models in Microsoft Fabric and Databricks. Advanced Python, SQL and PySpark, including experience with data transformation, analysis and common machine-learning libraries. Hands-on experience designing and operating scalable data pipelines, lakehouse data models, Delta ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

Azure Data Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You've got a background in software engineering, including Front End technologies like JavaScript. You're a problem-solver, pragmatically exploring options ...

notebooks, Delta Lake or Delta Live Tables Data lakehouse and medallion architecture Advanced data modelling, including relational, dimensional, Data Vault or similar approaches Python, PySpark or SQL Building or improving cloud-based data platforms Large-scale data migration or platform-modernisation projects Batch, streaming or event-driven data integration ...

Fabric, or Data Engineering are a plus. Technical Skills Required Preferred Required Preferred Power BI Microsoft Fabric SQL Azure Synapse Python Spark/PySpark Data Modeling Dataflows Gen2 DAX OneLake ETL/ELT Concepts Azure Data Factory Dashboard Development Azure Event Hub REST APIs AI/ML Tooling ...

process requirements into technical solutions and working across technical and non‐technical teams. “Nice To Have” Skills and Experience Experience with Databricks, Spark/PySpark, Delta Lake or comparable technologies. Experience with Celonis, process mining or process intelligence. Experience with enterprise platforms such as Salesforce, SAP or ServiceNow. Exposure ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You’ve worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering. You’re a problem-solver, pragmatically exploring options and finding effective solutions. An understanding ...

Experience leading or mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong ...

following technical expertise: Platform Experience: Strong, hands-on experience with Databricks or Snowflake on AWS. Programming & Orchestration: Proven Python skills, including deep expertise in PySpark (or Spark) and Airflow. Data Modelling: Advanced SQL skills and extensive experience with end-to-end data modelling. Data Integration: Proven experience building both ...

Experience leading or mentoring teams of engineers to deliver high‐quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong ...

Your responsibilities Design, develop and implement data pipelines to perform (ETL) from multiple sources into Azure Services. Proficiency in programming languages (e.g., Python, SQL, PySpark etc.) and data manipulation tools. Knowledge of DevOps tools Manage and monitor the health of data integration processes, troubleshooting Experience in Data Migrations (Perform ...

with both technical and non-technical audiences Nice to have: Production experience specifically in Microsoft Fabric Power BI, particularly Direct Lake mode Python/PySpark Data governance or security experience Experience enabling AI/ML workloads from a data platform, not just BI The details: Salary ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

Skills Significant experience leading the design, development, and operation of enterprise‐scale data engineering platforms and teams. Strong technical expertise in SQL and Python, PySpark, or equivalent technologies for data ingestion, transformation, and validation. Experience with Microsoft Fabric or similar modern cloud‐based analytics and data platforms. Strong experience ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview. ...

Bronze, Silver, Gold). Experience building reusable and scalable data frameworks. Understanding of data quality, monitoring, governance, and production support. Python and/or PySpark experience. Experience with Spark Notebooks. SSIS and SSRS experience. Experience within professional services or legal environments. If you’re a Data Engineer with strong ...

Looking ForProven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions. Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks. Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern ...

analytics libraries (e.g., pandas, numpy, matplotlib)Exposure to DevOps, DataOps, and MLOps concepts and best practicesExperience with core technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro, Dask/RAPIDS, Docker, Kubernetes, and cloud services (AWS/GCP/Azure)Experience with Generative AI (GenAI) or agentic systems ...

storage Proficiency with Terraform (ideally the Databricks provider) and Gitbased CI/CD workflows Solid programming skills in Python and SQL; comfortable with PySpark and shell scripting Experience tuning Spark jobs and Delta tables (partitioning, Working knowledge of SSO/SCIM, identity providers and secretsmanagement Understanding of data security ...

best practices, including DevOps, DataOps, and MLOps, will be advantageousWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such ...