23 of 23 Data Ingestion Jobs in Scotland

We support Asia’s growing energy needs while advancing the shift to a lower-carbon future. Purpose and Scope We are seeking a highly skilled and self-driven Azure Data Engineer with expertise in PySpark, Python, and modern Azure data services including Synapse Analytics and Azure Data Explorer. The ideal candidate will design, develop, and maintain scalable … data pipelines and architectures, enabling effective data management, analytics, and governance. Key Roles and Responsibilities Design, develop, and maintain scalable and efficient data pipelines (both batch and real-time streaming) using modern data engineering tools Build and manage data lakes, data warehouses, and data marts using Azure Data Services Integrate data from various sources including APIs, structured/unstructured files, IoT devices, and real-time streams Develop and optimize ETL/ELT workflows using tools such as Azure Data Factory, Databricks, and Apache Spark Implement real-time data ingestion and processing using Azure Stream Analytics, Event Hubs, or Kafka Ensure data quality, availability, and security across More ❯

Description Continuing Defleet's data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions with a focused mentality … on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science/ML platform using Oracle, Snowflake and … AWS tech stack. Qualifications We are looking for a skilled Lead Data Engineer who: Personal Attributes: Self-starter with initiative and enthusiasm Thrives at solving problems with minimal or no direction and developing their own solution Looks at the existing tech landscape and identifies improvements without prompting Desire and ability to continuously learn new technologies and practices and learn More ❯

Join to apply for the Lead Data Engineer role at Motability Operations Ltd 4 days ago Be among the first 25 applicants Join to apply for the Lead Data Engineer role at Motability Operations Ltd Continuing Defleet’s data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking … for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions with a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem … of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science/ML platform using Oracle, Snowflake and AWS tech stack. About You We are looking for a skilled Lead Data Engineer who: Personal Attributes Self-starter with initiative and enthusiasm More ❯

We are looking for an experienced Senior Data Engineer to join one of our client's team. Role and Responsibilities: Design, develop, and maintain scalable data pipelines using Python, PySpark and Databricks. Implement data orchestration workflows using Airflow or similar tools. Manage and optimize data storage solutions on Azure Cloud. Develop and maintain containerized microservices using … Docker. Collaborate with cross-functional teams to ensure data quality and integrity. Support the setup of blockchains. Build tools to acquire data from blockchains and perform analysis on the data using Databricks. Support production infrastructure, including monitoring and automation using tools like Datadog. Participate in agile development practices, contributing to planning exercises and working in cross-functional … groups. Skills and Qualifications: Bachelor's or Master's degree in Computer Science, Data Science, or a related field. Advanced Python knowledge for data processing and scripting. Proven experience with Spark and SQL for data engineering and analysis. Experience with data orchestration tools like Airflow. Hands-on experience with one or more cloud services (Azure, AWS More ❯

Social network you want to login/join with: Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial Intelligence/Machine Learning Reports To: Head of Engineering/CTO Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial … and images, with a roadmap that includes agentic superModels, API integrations, and video capabilities. Role Overview We are looking for a Senior DevOps Specialist with a strong python and data engineering background to support our R&D and tech teams by designing, building, and maintaining robust infrastructure and data pipelines across AWS and GCP. You will be instrumental … and proactively resolve infrastructure issues. Manage CI/CD pipelines for rapid, reliable deployment of services and models. Ensure high availability, disaster recovery, and robust security practices across environments. Data Engineering Build and maintain data processing pipelines for model training, experimentation, and analytics. Work closely with machine learning engineers and researchers to understand data requirements and workflows. More ❯

Social network you want to login/join with: Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial Intelligence/Machine Learning Reports To: Head of Engineering/CTO Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial … and images, with a roadmap that includes agentic superModels, API integrations, and video capabilities. Role Overview We are looking for a Senior DevOps Specialist with a strong python and data engineering background to support our R&D and tech teams by designing, building, and maintaining robust infrastructure and data pipelines across AWS and GCP. You will be instrumental … and proactively resolve infrastructure issues. Manage CI/CD pipelines for rapid, reliable deployment of services and models. Ensure high availability, disaster recovery, and robust security practices across environments. Data Engineering Build and maintain data processing pipelines for model training, experimentation, and analytics. Work closely with machine learning engineers and researchers to understand data requirements and workflows. More ❯

Social network you want to login/join with: Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial Intelligence/Machine Learning Reports To: Head of Engineering/CTO Job Title: DevOps Specialist & Data Engineer Location: Remote Type: Full-time Experience Level: Senior Industry: Generative AI/Artificial … and images, with a roadmap that includes agentic superModels, API integrations, and video capabilities. Role Overview We are looking for a Senior DevOps Specialist with a strong python and data engineering background to support our R&D and tech teams by designing, building, and maintaining robust infrastructure and data pipelines across AWS and GCP. You will be instrumental … and proactively resolve infrastructure issues. Manage CI/CD pipelines for rapid, reliable deployment of services and models. Ensure high availability, disaster recovery, and robust security practices across environments. Data Engineering Build and maintain data processing pipelines for model training, experimentation, and analytics. Work closely with machine learning engineers and researchers to understand data requirements and workflows. More ❯

Job Description: Your Impact As a Software Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and processing pipelines. … To succeed in this data engineering position, you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages. Key areas of responsibility. Orchestration ingestion and storage of raw data into structured or unstructured solutions. Design, Develop, Deploy and Support data infrastructure, pipelines and architecture. Implement reliable, scalable, and tested solutions to automate data ingestion. Development of systems to manage batch processing and real-time streaming of data Evaluate business needs and objectives. Facilitate pipelines, which prepare data for prescriptive and predictive modelling. Working with domain teams to scale the processing of data. Combine raw information from More ❯

Social network you want to login/join with: Software Data Engineer (All Levels), Edinburgh col-narrow-left Client: Leonardo Location: Edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Reference: a2d6f97aa72d Job Views: 4 Posted: 29.06.2025 Expiry Date: 13.08.2025 col-wide Job Description: Job Description: Your Impact As a Software Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and processing pipelines. To succeed in this data engineering position … you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages. Key areas of responsibility. Orchestration ingestion and storage of raw data into structured or unstructured solutions. Design, Develop, Deploy and Support data infrastructure, pipelines and architecture. Implement reliable, scalable More ❯

you will own flagship projects, lead enterprise transformation, and execute innovative solutions to address the complex challenges of our clients. You will be responsible for: Shaping and executing enterprise data transformation: Driving enterprise-level transformation projects, including designing data platforms, establishing robust governance frameworks, and embedding scalable AI and analytics solutions. Leading innovation in AI and analytics: Spearheading … the development of generative AI solutions, statistical models, and data visualisations that deliver actionable insights across various industries. Delivering strategic advisory: Acting as a trusted advisor to C-level stakeholders, providing insights that bridge the gap between technical capabilities and business goals. Portfolio and account management: Overseeing diverse project portfolios, managing commercial outcomes, and fostering strong client relationships to … Leading multidisciplinary teams in agile and lean environments, ensuring timely delivery of impactful, high-quality solutions. Fostering capability building: Empowering teams through coaching, training, and mentoring to enhance organisational data and AI competencies. This role also provides opportunities to contribute to the broader business through: Business Development: Leading or contributing to proposals, bids, and RFP responses, as well as More ❯

Social network you want to login/join with: Search 5.0 is delighted to partner with an innovative AI tech start-up in the financial data sector, seeking a Rust Developer to join their exciting journey! Our client equips traders with tools that are faster, smarter, and easier to use. This is a fantastic opportunity to join a leading … AI company whose Machine Learning product suite creates new trading opportunities through real-time analytics, supported by a world-class, scalable, ultra-low latency architecture. The Data Platform is crucial to all capabilities and services, enabling data ingestion, analysis, querying, and storage at near-real-time speeds, setting our client apart from competitors. While most companies process … data in days, hours, or minutes, our client operates at nano to millisecond scales, a speed typically reserved for stock market trading systems. Job: Rust Developer Joining an incredible, small tribe you will play a pivotal role in driving the technical direction ensuring requirements are met and building a scalable product. Develop and maintain the platform services that underpin More ❯

capabilities, and skills Formal training or certification on software engineering concepts and advanced applied experience Solid understanding of software applications and technical processes within a related technical discipline (. data ingestion, data storage, data serving, APIs, . Hands-on experience in data lake or data warehouse and related technologies (. Spark, ETL, Databricks). More ❯

Flexibility: Part-time, condensed hours, job-shares, and flexible arrangements considered Diversity & Inclusion: Extremely important—we encourage applicants from all backgrounds Who We Are Hypercube Consulting is an innovative data and AI consultancy dedicated to revolutionising the energy sector. We specialise in advanced Agentic AI workflows and Large Language Models (LLMs), empowering our clients to realise significant value from … Role Purpose We are looking for a Senior AI Engineer with experience in Agentic AI and LLM technologies to design and develop impactful AI solutions. You'll work alongside data engineers, data scientists, and cloud specialists, directly contributing to client success. In this role, you will: Engage with client challenges, developing practical and innovative Agentic AI and LLM … growing AI expertise. Key Responsibilities Technical Contribution Develop and deploy reliable AI and LLM solutions, collaborating with senior team members. Contribute to the implementation and maintenance of AI pipelines (data ingestion, model training, deployment, and monitoring). Work within multidisciplinary teams to deliver client-focused solutions. Effectively communicate AI concepts and methodologies to peers and clients. Stay up More ❯

s challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role As a Data & AI Operating Model Consultant, you will play a key role in setting organisations up for the fourth industrial revolution, helping them to effectively leverage data and AI to … drive business outcomes: You will play a central role in the designing of effective data, analytics & AI operating models, covering all components required to deliver an effective data & AI capability. You will be work with our clients to define their strategic objectives, and then build a picture of the requisite data & analytical capabilities required to realise those … strategic objectives. You will provide guidance and advisory services to clients on a wide range of data & analytics capabilities at all points of the data & analytics value chain. You will work with stakeholders at all levels of the organisation to assess the maturity of existing data capabilities, identifying opportunities for re-use and corresponding gaps in capability. More ❯

The Role Design, build and run the software and data that power our platform—from user interfaces to APIs and data ingestion—while keeping security front of mind. Key Responsibilities Build and maintain our Vue 3 platform plus supporting APIs, microservices, database and semantic layers. Write clean, well‐tested code and review peers’ work. Day to day … operation of systems, including availability, security and data integrity Collaborate daily with product, data and DevOps teams in short agile sprints. Skills & Experience Production experience with Vue.js, JavaScript/TypeScript, HTML and CSS. Proficiency in at least one back‐end language (Python, Java, C#, Node). Strong SQL and familiarity with relational & NoSQL databases. Experience designing and consuming … relationships with peers. Communicative: speak up in discussions, share ideas and listen actively. Leadership mindset: take initiative, mentor others and drive best practices. The Opportunity Shape an industry‐leading data platform used across the UK and US. Small, supportive team where every engineer has real impact. Rapid growth means space to develop your skills and career. Why Broker Insights More ❯

Dundee, 1 - 2 days a week in the office Salary £45 - £55k Head Resourcing are recruiting for a software engineer for a fintech company based in Dundee. We use data to transform the commercial insurance market. The role focuses on designing, building and running the software and data that power our platform-from user interfaces to APIs and … data ingestion-while keeping security front of mind. Key Responsibilities Build and maintain our Vue?3 platform plus supporting APIs, microservices, database and semantic layers. Write clean, well?tested code and review peers' work. Day to day operation of systems, including availability, security and data integrity Collaborate daily with product, data and DevOps teams in short … with peers. Communicative: speak up in discussions, share ideas and listen actively. Leadership mindset: take initiative, mentor others and drive best practices. Why Join Us? Shape an industry?leading data platform used across the UK and US. Small, supportive team where every engineer has real impact. Rapid growth means space to develop your skills and career. More ❯

Dundee, 1 - 2 days a week in the office Salary £45 - £55k Head Resourcing are recruiting for a software engineer for a fintech company based in Dundee. We use data to transform the commercial insurance market. The role focuses on designing, building and running the software and data that power our platform-from user interfaces to APIs and … data ingestion-while keeping security front of mind. Key Responsibilities Build and maintain our Vue 3 platform plus supporting APIs, microservices, database and semantic layers. Write clean, well tested code and review peers' work. Day to day operation of systems, including availability, security and data integrity Collaborate daily with product, data and DevOps teams in short … with peers. Communicative: speak up in discussions, share ideas and listen actively. Leadership mindset: take initiative, mentor others and drive best practices. Why Join Us? Shape an industry leading data platform used across the UK and US. Small, supportive team where every engineer has real impact. Rapid growth means space to develop your skills and career. More ❯

Other EU work permit required: Yes Job Reference: b81240d71528 Job Views: 5 Posted: 25.06.2025 Expiry Date: 09.08.2025 Job Description: Job Purpose To contribute to the growing research in Geospatial Data Science at the College of Science and Engineering, working with Professor Ana Basiri and the Geospatial Data Science Group at the University of Glasgow. The successful candidate will … geospatial AI tools, deliver high-impact research, and contribute to team development through research outputs, proposals, supervision, and presentations. Main Duties and Responsibilities Develop innovative GeoAI solutions combining geospatial data with APIs like GPT, Codex, etc., for data tasks and AI-driven applications. Study and develop applications using Large Language Models through fine-tuning and integration. Apply techniques … such as prompt engineering, data ingestion, and transformation. Collaborate with the Geospatial Data Science team and external partners across industry, government, and third sectors. Enhance research profile through publications, societal impact, and research indicators. Present work at conferences, seminars, and workshops. Support FAIR principles by making data and code repositories accessible and reproducible. Pursue personal and More ❯

energy systems), its current lease terms and use, its current value and future "green premium," and the value of energy flexibility in a future grid. We gather all this data and build machine learning models to find the optimum retrofits across a portfolio of buildings. To make this possible, we're building a diverse team of people passionate about … also be proficient in standard front-end technologies such as JavaScript, HTML5, and CSS3. Some experience in Python, including writing code that has gone into production. Demonstrated experience with data ingestion from various sources, including APIs and data downloads. Familiarity with database technologies (preferably PostgreSQL experience). Experience with cloud-based deployment and management, preferably with GCP. More ❯

experiences using the product by enabling swift, precise, and privacy-compliant observability and issue resolution across all devices and applications. Our mission is to transform vast amounts of device data into actionable insights that drive product stability and customer satisfaction. To achieve this mission, we provide comprehensive technology solutions spanning the entire observability lifecycle. This includes on-device crash … and log management, high-scale data ingestion services, advanced analytics dashboards, and intuitive troubleshooting interfaces. As a SDE on the team, you will be responsible for leading the architecture, design, development and launch of some of the core product features. You will have significant influence on our overall strategy by helping define these product features, drive the system More ❯

with applications. • Hands-on experience with UI modern technologies like ReactJS and/or AngularJS • Hands-on experience in database development with technologies such as Oracle, Cassandra. • Code efficient data ingestion orchestration using Spring Integration, Spring Batch to facilitate integration with upstream and downstream applications. • Solid understanding in Agile Methodology, CI/CD Pipelines and OKRs. • Expertise in More ❯

Social network you want to login/join with: Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centres, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in … inference of generative AI models. Design and implement advanced methodologies like LoRA, prefix-tuning, and adapter-based approaches for fine-tuning AI models. Develop robust, fault-tolerant systems for data ingestion, processing, and model customisation. Optimise GPU utilisation and system performance using frameworks like DeepSpeed, Triton Inference Server, TensorRT, and custom CUDA/Rocm kernels. Conduct performance testing … production environments. Proficiency in Python and PyTorch, with a strong understanding of transformer architectures, LLMs, and multimodal generative models. Expertise in distributed training frameworks like DeepSpeed or Fully Sharded Data Parallel (FSDP). Experience with GPU programming and optimisation e.g CUDA, TensorRT, or ROCm. Knowledge of fine-tuning methods, such as LoRA, prefix-tuning, and adapter-based techniques, and More ❯

Social network you want to login/join with: Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centres, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in … inference of generative AI models. Design and implement advanced methodologies like LoRA, prefix-tuning, and adapter-based approaches for fine-tuning AI models. Develop robust, fault-tolerant systems for data ingestion, processing, and model customisation. Optimise GPU utilisation and system performance using frameworks like DeepSpeed, Triton Inference Server, TensorRT, and custom CUDA/Rocm kernels. Conduct performance testing … production environments. Proficiency in Python and PyTorch, with a strong understanding of transformer architectures, LLMs, and multimodal generative models. Expertise in distributed training frameworks like DeepSpeed or Fully Sharded Data Parallel (FSDP). Experience with GPU programming and optimisation e.g CUDA, TensorRT, or ROCm. Knowledge of fine-tuning methods, such as LoRA, prefix-tuning, and adapter-based techniques, and More ❯