9 of 9 Remote/Hybrid Databricks Jobs in Newcastle upon Tyne

explain and implement the concepts and principles of data modelling. Ability to create and run simple unit tests. Desired Qualifications Certifications in AWS, Azure, Databricks, or related technologies. Example Experience with public sector data initiatives and compliance requirements. Knowledge of machine learning and artificial intelligence concepts. Where you’ll work ...

understanding of the concepts of real time data streaming, batch data processing and data transformation processes Experience with core tools such as Data Factory, Databricks, Synapse, Kafka and Python Any exposure to data migration/ETL would be highly beneficial, with SQL/T-SQL, SSIS, SSRS and SSAS ...

integration, prompt engineering, RAG architectures, fine-tuning, Agentic AI frameworks (LangChain, LlamaIndex, AutoGen). Data Engineering : ETL/ELT patterns, data modeling, Snowflake, Databricks, dbt, Airflow, Kafka/streaming architectures. Cloud Platforms : AWS (SageMaker, Bedrock, Glue), Azure (OpenAI Service, Synapse), GCP (Vertex AI, BigQuery). MLOps : Model deployment, monitoring, versioning ...

Data Engineering Testing tools like Great Expectations, dbt, pytest and data validation frameworks Experience of testing data platforms such as Snowflake, BigQuery, Redshift, Databricks, Azure Synapse Experience of testing pipelines in things like Airflow, Azure Data Factory, AWS Glue, Kafka Data Concepts ETL/ELT, data modelling, data warehousing, data ...

Data Engineering Testing tools like Great Expectations, dbt, pytest and data validation frameworks Experience of testing data platforms such as Snowflake, BigQuery, Redshift, Databricks, Azure Synapse Experience of testing pipelines in things like Airflow, Azure Data Factory, AWS Glue, Kafka Data Concepts ETL/ELT, data modelling, data warehousing, data ...

Gold layers of our Medallion Architecture, and all the way to analytics, AI and self‐service reporting. Working within the Microsoft Azure and Databricks ecosystem, you’ll help build a data platform that’s scalable, flexible and built to last. Your work will directly support high‐impact use cases, including … zone into the Bronze layer, working in partnership with the Development and Application Management team. Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. Ensure Silver and Gold layer data products are fit for purpose for analytics, pricing, AI and ML model consumption. ...

A leading insurance company in Newcastle upon Tyne is seeking a Data Architect to define architectural standards and manage the data solution frameworks. You'll be responsible for the design and governance of the Medallion ...

optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance … from raw ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets ...

DevOps Engineer – BPSS/SC role 5-6 Month Contract Newcastle upon Tyne – Hybrid Working OUTSIDE IR35 - £400 Per Day One of my national consultancy clients’ is looking for 2 DevOps Engineers, to support one ...