26 to 50 of 126 PySpark Jobs

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

experience required Strong experience in data engineering Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

experience required Strong experience in data engineering Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

business needs Ensure data quality, governance and best practice Key Requirements Strong leadership experience (5+ engineers) Solid Data Engineering/DataOps background Python/PySpark for data pipelines AWS experience (design & deployment) Strong data modelling & warehousing knowledge Agile (Jira/Azure DevOps) experience Get in touch for more details ...

Customer Experience, Finance and Operations can trust. Responsibilities Build, optimise and operate ELT/ETL pipelines into our data platform using SQL and Python (PySpark), with a focus on reliability, performance and maintainability. Develop and maintain core data models (curated layers, dimensional models, shared definitions) that enable consistent ...

Data bricks). Experience with Power BI or other Azure Data Services is a significant advantage. Strong experience with SQL. Familiarity with Python/PySpark is a plus. Deep knowledge of modeling approaches such as Kimball, Inmon, and Data Vault 2.0, with the ability to select the most appropriate ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

sizing Fabric initiatives, including capacity needs, effort estimates, infrastructure requirements, team resourcing, and delivery timelines. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Lead architectural design, capacity planning, and performance optimisation efforts. Implement governance, security ...

data projects Key Requirements Proven leadership of multiple engineering teams Strong Data Engineering/DataOps background Experience in large, dataheavy environments Python/PySpark for endto-end pipelines Strong AWS experience (design & deployment) Deep knowledge of data modelling, warehousing & lakehouse concepts Strong stakeholder management and communication skills Agile delivery ...

Hands-on experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory• Hands-on data engineering experience (Spark/PySpark, Delta Lake)• Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC• Experience leading Agile teams and managing risks, dependencies ...

Collaborate with your peers on technical direction within your team Required Skills and Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

production. Key skills and experience: Strong experience as a Data Engineer Advanced Python and SQL skills Experience with cloud platforms (AWS preferred) Proficiency in PySpark and/or Pandas Familiarity with data quality tools (e.g., Great Expectations) Experience with infrastructure as code (Terraform or CloudFormation) Understanding of modern development ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

basis, clarifying ownership and integration points, and capturing this information in a way that supports risk assessment and decommissioning decisions. Responsibilities: Python and PySpark are required as supporting capabilities, used where needed to analyse data pipelines and confirm how data moves and transforms in practice. The role also requires ...

Strong knowledge of data modelling, database design, and enterprise data architecture Experience with ETL/ELT frameworks and data integration patterns using Python or PySpark Experience with structured and semi-structured data, preferably in life sciences (e.g., omics, health records) Ability to translate business requirements into technical solutions ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...