3 of 3 Remote/Hybrid Databricks Jobs in Lincolnshire

explain and implement the concepts and principles of data modelling. Ability to create and run simple unit tests. Desired Qualifications Certifications in AWS, Azure, Databricks, or related technologies. Example Experience with public sector data initiatives and compliance requirements. Knowledge of machine learning and artificial intelligence concepts. Where you’ll work ...

assessments of counterparts design Adhere to the organisations Release and Change process The individual: Expert knowledge of authoring/troubleshooting SQL/Azure/Databricks queries Expert knowledge and application of general and tool specific connectivity, data modeling, source data procurement strategies (Database, import/cached), security configuration, performance tuning ...

optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance … from raw ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets ...