76 to 100 of 394 PySpark Jobs

Hands-on expertise in Snowflake and Databricks Proficient in Microsoft Azure, including Azure Data Factory and Azure Data Lake Advanced SQL and Python or PySpark skills Experience building enterprise-level ETL/ELT pipelines Knowledge of data modelling, governance, and data quality Previous consultancy or client-facing delivery experience ...

tools and approaches. Hands-on experience with Microsoft Azure technologies, including MS Fabric, Data Factory, Azure Synapse & RDS. Experience with Python or PySpark for scripting and automating data tasks. Knowledge of CI/CD approaches for Data Platforms Knowledge of MS SQL Server, SSIS, Visual Studio, and SSDT projects. ...

Experience designing data architectures that scale, and a good handle on integration patterns like ETL/ELT and batch or streaming. Desirable Technical Skills PySpark or a similar distributed data processing framework. CI/CD pipelines, observability tooling (logging, alerting, monitoring), and infrastructure‐as‐code. Experience working with cloud ...

mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...

batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL. Deliver solutions across the full data & AI engineering lifecycle, from requirements analysis and solution design through development, testing, deployment … data platforms, data products and analytics solutions using modern lake house or cloud data warehouse technologies Proficiency writing advanced, highly optimised SQL and Spark (PySpark/Scala), with a focus on performance optimisation and maintainability. Designing and building solutions on a cloud data warehouse or lakehouse platform (e.g., Databricks ...

development of scalable ETL/ELT pipelines. Proven experience working with SQL, relational databases and large datasets. Experience with data processing technologies such as PySpark, Spark or Pandas. Strong understanding of data quality, transformation and integration. Experience with Git, CI/CD and automated testing. Comfortable working in Linux ...

/ML and data engineering, with a strong foundation in enterprise‐scale solution development Strong hands‐on expertise in Python, Apache Spark, PySpark, and Databricks for large‐scale data processing and AI model development Proven ability to design, deploy, and evaluate machine learning and generative AI models using platforms ...

performance and automation Required Experience Strong Data Engineering background within Banking, Financial Services or Capital Markets. Expertise in Python and SQL. Experience with Spark, PySpark and streaming technologies Experience with cloud data platforms such as Databricks and Azure Data Services. Strong understanding of ETL/ELT development and modern ...

design Databricks and/or Microsoft Fabric at architecture level Azure Data Factory, Synapse, Azure SQL, Data Lake Storage Gen2 Advanced SQL, Python or PySpark, Spark optimisation, Delta Lake, Unity Catalog Azure DevOps or GitHub Actions, Terraform or Bicep Entra ID, Key Vault, private endpoints, RBAC UK GDPR ...

compliance, controls, remediation, or automation programs (preferably in financial services). Strong proficiency in Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases. Experience designing ...

Looking ForProven expertise with the Databricks platform, including Unity Catalog, Delta Lake, orchestration, and enterprise-scale Azure deployments. Strong technical skills in Python, PySpark, and distributed data processing frameworks, with experience building production-grade data platforms. Extensive experience implementing CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions ...

Azure Resource Manager/Bicep, or Google Cloud Deployment Manager) to build, maintain, and scale complex cloud environments Strong coding proficiency in Python and PySpark/Spark: you must demonstrate live coding, debugging, and solution-building skills Solid grounding in distributed data systems architecture: scalable pipelines, streaming architectures, lakehouse ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

Generation), Serverless, DWH models, Data Mesh, Data Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape ...

relevant to the business. What you'll be doing Design, build and optimise data pipelines and analytics solutions using Databricks Develop SQL, Python and PySpark solutions for transformation, analytics and automation Build and improve AI agents, reusable workflows, prompt templates and automation solutions Use AI development tools for coding ...

functional teams entrusted with business-critical platforms. Desirable skills & experienceWorking to an Agile methodology and familiarity with Azure DevOpsDeep automation knowledge with PythonSkilled in Pyspark and SynapseExperience with data modelling and visualisation in Power BI (or alternative)A strong understanding of architecting data platforms, BI, MI, or analytics solutionsStrong ...

methodologies. Требования: Experience in data engineering and data integration; Experience building and managing data pipelines on Azure or AWS; Hands-on experience with Notebook PySpark coding, preferably on Databricks; Advanced SQL skills for developing and optimizing data pipelines; Understanding of data modeling and data transformation techniques; Knowledge of maintaining ...

experience would be desirable Data warehousing, Power BI/Fabric administration or data governance experience would be advantageous DevOps, source control and Python/PySpark exposure would be useful but not essential The Opportunity There’s a lot happening across the organisation’s data landscape, including major data ...

relevant data solutions development qualifications (e.g. Microsoft certification; Databricks Certification; data apprenticeship qualifications)Additional InformationTech StackSQL Server, SSIS, T-SQLADF, ADLS, Azure SQL Database, PySpark, Spark SQL, Databricks, Delta lakeSummaryType: Full-timeFunction: Analyst ...

knowledge of Matillion’s Data Productivity Cloud/Matillion ETL, or primary competitive data integration platforms. Big Data & Analytics Ecosystems: Experience with Spark/PySpark, modern Data Lakes/Lakehouses, and MPP database architectures (Teradata, Vertica, Netezza). Education: Bachelor’s or Master’s degree in Computer Science, Data ...

principles. Strong understanding of data modelling, governance and data quality frameworks. Experience delivering large-scale data transformation initiatives. Knowledge of technologies such as SQL, PySpark, Snowflake, Kafka and modern BI platforms including Power BI or Tableau. Excellent communication skills with the ability to influence senior technical and business stakeholders. ...

actually trust and use. Essential Skills & Experience: Hands-on experience with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi ...

grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build … ongoing evolution of Envitia’s data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ ...

with Databricks SQL endpoints for data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/ ...