4 of 4 PySpark Jobs in Berkshire

with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

manage Ontology Objects, Links, and Actions to support downstream applications and workflows. Pipeline Development & Automation: Build scalable, fault-tolerant ETL/ELT pipelines using PySpark, SQL, and Foundry's proprietary tools like Pipeline Builder and Code Workbook. Productization & Scaling: Create Prototypes and Convert them into production-grade solutions ...

supporting/principles. Current Microsoft/Databricks certifications are useful but not mandatory – we'll help you get those! Core: Data Manipulation (SQL, Pandas, Pyspark) Azure AI (Azure AI Foundry, AI Search, Document Intelligence, AI Services) Data Science & Machine Learning (Databricks, Python, SKLearn, XGBoost, MLFlow, EDA) Familiarity with LLMs … Modelling Data Science Data Warehouse Theory Data Architecture Master Data Management Key Words: Azure - AI - Artificial Intelligence - ML - Machine Learning - Consultant - IT Consultancy - SQL - PySpark - Azure AI - AI Search - LLM - OpenAI - Data Science - Databricks - Python - AI Engineer - ML Engineer - AI Consultant - ML Consultant - Data Manipulation - SQL Databases Additional Information ...