4 of 4 PySpark Jobs in Basildon

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

Looking For: Proven hands-on experience in data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like ...

analysis, governance, reporting and analytics. Hands-on delivery experience working with data, analysis and BI tools e.g. SQL, PowerQuery, Excel VBA, Python (e.g. Pandas, PySpark), DataBricks, Power BI, Tableau etc. The salary on offer for this Data Business Analyst Role is up to £120,000 + Bonus and Benefits … define functional and system requirements Hands-on delivery experience working with data, analysis and BI tools e.g. SQL, PowerQuery, Excel VBA, Python (e.g. Pandas, PySpark), DataBricks, Power BI, Tableau etc. Essential Skills for the Data Business Analyst: Senior IT Business Analyst/Data Analyst with experience of a Commodities ...

open-source data validation frameworks to ensure data accuracy, schema integrity, and quality across ingestion and transformation pipelines Test and validate data pipelines and PySpark notebooks developed by Data Engineers, ensuring they meet quality, reliability, and validation standards Defining and standardizing monitoring, logging, alerting, and KPIs/SLAs across … setting up Alert rules, building dashboards using data queried (KQL) from Log Analytics. Experience with Fabric Data Factory, Azure Data Factory, Synapse pipelines, and PySpark notebooks Hands-on experience calling REST/OData APIs for validating data. Experience writing SQL and scripts for programmatically doing data validations and reconciliation ...