11 of 11 Permanent Databricks Jobs in Newcastle upon Tyne

explain and implement the concepts and principles of data modelling. Ability to create and run simple unit tests. Desired Qualifications Certifications in AWS, Azure, Databricks, or related technologies. Example Experience with public sector data initiatives and compliance requirements. Knowledge of machine learning and artificial intelligence concepts. Where you’ll work ...

understanding of the concepts of real time data streaming, batch data processing and data transformation processes Experience with core tools such as Data Factory, Databricks, Synapse, Kafka and Python Any exposure to data migration/ETL would be highly beneficial, with SQL/T-SQL, SSIS, SSRS and SSAS ...

integration, prompt engineering, RAG architectures, fine-tuning, Agentic AI frameworks (LangChain, LlamaIndex, AutoGen). Data Engineering : ETL/ELT patterns, data modeling, Snowflake, Databricks, dbt, Airflow, Kafka/streaming architectures. Cloud Platforms : AWS (SageMaker, Bedrock, Glue), Azure (OpenAI Service, Synapse), GCP (Vertex AI, BigQuery). MLOps : Model deployment, monitoring, versioning ...

Data Engineering Testing tools like Great Expectations, dbt, pytest and data validation frameworks Experience of testing data platforms such as Snowflake, BigQuery, Redshift, Databricks, Azure Synapse Experience of testing pipelines in things like Airflow, Azure Data Factory, AWS Glue, Kafka Data Concepts ETL/ELT, data modelling, data warehousing, data ...

Gold layers of our Medallion Architecture, and all the way to analytics, AI and self‐service reporting. Working within the Microsoft Azure and Databricks ecosystem, you’ll help build a data platform that’s scalable, flexible and built to last. Your work will directly support high‐impact use cases, including … zone into the Bronze layer, working in partnership with the Development and Application Management team. Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. Ensure Silver and Gold layer data products are fit for purpose for analytics, pricing, AI and ML model consumption. ...

A leading insurance company in Newcastle upon Tyne is seeking a Data Architect to define architectural standards and manage the data solution frameworks. You'll be responsible for the design and governance of the Medallion ...

deliver solutions that support business objectives What We're Looking For Strong experience designing and building scalable data pipelines Proficiency in SQL, Python, Databricks or AWS Experience with tools such as Git and Airflow Proven ability to test, troubleshoot and improve data platforms Experience delivering high-quality, reliable data solutions ...

Lead Data Engineer - Python/SQL/Spark/Databricks is required by highly successful and fast growing technology consultancy. As a Lead Data Engineer, you'll be part of a team building innovative products. You should be skilled in problem-solving and experienced in designing scalable, resilient, and fault ...

optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance … from raw ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets ...

Data Engineer - Python/SQL/Spark/Databricks is required by highly successful and fast growing technology consultancy. As a Data Engineer, youll be part of a client team building innovative products. You should be skilled in problem-solving and experienced in designing scalable, resilient, and fault-tolerant architectures. … pipelines One or more programming languages, typically Python and SQL Some or all of the following: Distributed computing frameworks such as Spark or Databricks Cloud platforms such as AWS, Azure or GCP Orchestration tools such as Airflow Test, troubleshoot and improve data products, ensuring reliability and quality You will have ...

Lead Data Engineer - Python/SQL/Spark/Databricks is required by highly successful and fast growing technology consultancy. As a Lead Data Engineer, you'll be part of a team building innovative products. You should be skilled in problem-solving and experienced in designing scalable, resilient, and fault … pipelines One or more programming languages, typically Python and SQL Some or all of the following: Distributed computing frameworks such as Spark or Databricks Cloud platforms such as AWS, Azure or GCP Orchestration tools such as Airflow Test, troubleshoot and improve data products, ensuring reliability and quality You will have ...