26 to 50 of 127 Data Pipeline Jobs in Scotland

motivated AI Full Stack Developer with a strong foundation in cybersecurity operations and artificial intelligence technologies. The ideal candidate will have deep technical expertise in AI/ML, data science, and programming, alongside a solid understanding of cyber threat detection and defensive operations frameworks. You will be instrumental in designing and implementing full-stack AI-driven solutions to … full-stack development tools and frameworks. · Collaborate with cybersecurity teams to build and enhance detection and mitigation systems based on frameworks like MITRE ATT&CK. · Develop and maintain data pipelines, APIs, dashboards, and visualization tools to support AI model insights. · Write clean, well-documented, and secure code using Python, C C#, or R. · Leverage libraries such as TensorFlow … AI services in cloud-native environments. · Familiarity with containerization (Docker, Kubernetes) and DevOps pipelines. · Exposure to security operations center (SOC) tools and SIEM platforms. · Experience working with big data platforms such as Spark, Hadoop, or Elastic Stack. #J-18808-Ljbffr More ❯

Job Description Lead AI and Data Engineer (Edinburgh) Our client is looking for a technical and strategic AI and Data Engineer to take ownership of AI and data engineering practices. You’ll play a central role in shaping modern data platforms, building production-grade AI solutions, and embedding best practices in data and AI engineering. Key Responsibilities: Lead the design and delivery of scalable data architectures and pipelines to meet analytics and AI needs. Develop and maintain AI and data products aligned to business outcomes. Champion automation-first engineering practices across ingestion, transformation, and deployment. Design and deploy generative and predictive AI solutions using Azure and … open-source frameworks. Implement modern DataOps and MLOps practices, including CI/CD pipelines and monitoring. Establish and maintain reusable enterprise data models and semantic layers for self-service analytics. Mentor and develop a high-performing engineering team. Collaborate with stakeholders to embed insight and AI in product delivery and business decision-making. About You: Deep expertise in More ❯

officially designated Posts. Our UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving theirdeliveryin a secure manner in compliance with Data regulations. The team is primarily Azure focused but other Cloud technologies such as Oracle andAmazon Web Services (AWS) are usedin FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load ETL and Extract Load and Transform ELT ) in the Azure cloud platforms. You will work with structured and unstructured data, data lakes, data warehouses to service operational and analytical business needs. The successful candidates will: Work collaboratively with Enterprise Data Colleagues to develop pipelines and data product that aligns with data architecture and engineering best practice. Develop data assets and products that assist Power Business Intelligence (BI) colleagues to More ❯

Job Description: Your Impact As a Software Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion … and processing pipelines. To succeed in this data engineering position, you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages. Key areas of responsibility. Orchestration ingestion and storage of raw data into structured or unstructured solutions. … Design, Develop, Deploy and Support data infrastructure, pipelines and architecture. Implement reliable, scalable, and tested solutions to automate data ingestion. Development of systems to manage batch processing and real-time streaming of data Evaluate business needs and objectives. Facilitate pipelines, which prepare data for prescriptive and predictive modelling. Working with domain teams More ❯

officially designated Posts. Our UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving their delivery in a secure manner in compliance with Data … The team is primarily Azure focused but other Cloud technologies such as Oracle and Amazon Web Services (AWS) are used in FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load {ETL} and Extract Load and Transform {ELT}) in the Azure cloud platforms. You will work with structured and unstructured data, data lakes, data warehouses to service operational and analytical business needs. Job description The successful candidates will: Work collaboratively with Enterprise Data Colleagues to develop pipelines and data product that aligns with data architecture and engineering best practice. Develop data assets and products that assist Power Business Intelligence (BI More ❯

the Scottish market Full-time · Permanent · Remote/Hybrid (Glasgow-based office) A leading SaaS business at the forefront of digital transformation is seeking an experienced Senior Engineer - Data to join its growing data team. As the company scales its next-generation platform, data is central to delivering fast, accurate, and intelligent user experiences. … This is a fantastic opportunity to play a key role in shaping a modern data and search infrastructure using cutting-edge technologies. About the Role You’ll be part of an agile, cross-functional team building a powerful data platform and intelligent search engine. Working with technologies like Apache Lucene, Solr, and Elasticsearch, you'll contribute … and development of scalable systems, with opportunities to explore machine learning, AI-driven categorisation models, and vector search. What You’ll Be Doing Design and build high-performance data pipelines and search capabilities. Develop solutions using Apache Lucene, Solr, or Elasticsearch. Implement scalable, test-driven code in Java and Python. Work collaboratively with Business Analysts, Data More ❯

for an experienced Senior AI/ML Engineer to join one of our client's team. Skills and Qualifications: Bachelor's or Master's degree in Computer Science, Data Science, or a related field. Strong expertise in Python, with a focus on FastAPI, Pandas, and NumPy. Experience with Retrieval Augmented Generation (RAG). Proficiency in Prompt Engineering. Knowledge … on experience with Azure cloud services, including Azure Functions. Familiarity with other cloud platforms and their AI services is a plus, but Azure experience is mandatory. Understanding of data engineering concepts, including data pipelines, data integration, and data warehousing. Ability to work with large datasets and perform data cleaning, transformation More ❯

its expanding technology team. You’ll contribute to the ongoing development of a next-generation platform, with a strong focus on high-performance search capabilities and large-scale data processing. This is a fantastic opportunity to work on impactful technology in a forward-thinking, data-driven environment. Your Role As a Senior Java Developer, you’ll … be instrumental in building and evolving a resilient data platform that underpins key SaaS offerings. You’ll work closely with engineers, designers, and product teams to deliver scalable, performant solutions that help end users derive real value from complex data. Key Responsibilities Write clean, maintainable, and well-tested code following Agile and TDD practices Contribute to both backend … services (Java) and modern front-end frameworks (React/TypeScript) Develop and fine-tune search functionality using technologies like Lucene, Solr, or OpenSearch Collaborate with Data Engineers to design and deliver data pipelines Leverage cloud platforms (AWS, Azure, or GCP) for scalable and resilient processing Participate in code reviews, pair programming, and mentoring team members Support More ❯

Lead Data Engineer Salary: £85,000–£90,000 + Up to 10% Bonus + 15% Non - Contributory Pension Location: Hybrid working (3 days P/W On-Site) | Edinburgh Level: Lead We’re looking for a Lead Data Engineer to play a key role in expanding our client's data-driven capabilities and driving … the evolution of our enterprise data ecosystem. This is a strategic, hands-on position where you’ll lead the design and delivery of large-scale data solutions that power critical business decisions. You'll work with a high-performing Data & Analytics team on mission-critical systems across Data Warehousing, Data … create real business impact. You’re proactive, hands-on, and comfortable working across both technical and business domains. ✅ Key technical skills: Strong SQL and ELT/data pipeline development experience Expertise in Data Warehouse & Data Lake design (including Star Schema, Snowflake Schema, Data Vault) Hands-on experience with enterprise databases: Oracle More ❯

employee exit rate is astonishingly low. Join us on our journey... Let’s create something awesome, together, today. About the role and team: We’re looking for a Data Engineer to join our growing Data Practice. You’ll work as part of a multidisciplinary team … delivering data solutions for some of the UK’s best-known brands. You’ll have a hand in everything from data ingestion and transformation to pipeline automation and data platform optimisation – always with a strong focus on quality, scalability, and value. What you'll be doing: Designing, building and maintaining robust, cloud-native … deliver value-focused data solutions We'd love to talk to you if: You’ve got solid experience working with Python, SQL, Spar and data pipeline tools such as dbt or Airflow You’re comfortable working across cloud platforms – especially AWS (Glue, Lambda, Athena), Azure (Data Factory, Synapse), or GCP (Dataflow, BigQuery, Cloud More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing … etc. They are looking for a senior Data Engineer to join their FinTech team! You will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing … Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements More ❯

Data Engineering Manager/Edinburgh/Microsoft Gold Partner Data Engineering Manager/Edinburgh/Microsoft Gold Partner Direct message the job poster from Opus Recruitment Solutions Recruitment Consultant | Microsoft Dynamics - CRM/CE at Opus Recruitment Are you a passionate Data Engineering Manager ready to unlock the power of data? Our … client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For: Proven Data Engineering experience (5+ years). Consultancy experience is a must. Leadership and multi-project More ❯

Senior Engineer - Data – Leading SaaS Business Full-time · Permanent · Remote/Hybrid (Glasgow-based office) A leading SaaS business at the forefront of digital transformation is seeking an experienced Senior Engineer - Data to join its growing data team. As the company scales its next-generation platform, data is central to delivering fast … accurate, and intelligent user experiences. This is a fantastic opportunity to play a key role in shaping a modern data and search infrastructure using cutting-edge technologies. About the Role You’ll be part of an agile, cross-functional team building a powerful data platform and intelligent search engine. Working with technologies like Apache Lucene, Solr … and development of scalable systems, with opportunities to explore machine learning, AI-driven categorisation models, and vector search. What You’ll Be Doing Design and build high-performance data pipelines and search capabilities. Develop solutions using Apache Lucene, Solr, or Elasticsearch. Implement scalable, test-driven code in Java and Python. Work collaboratively with Business Analysts, Data More ❯

Social network you want to login/join with: Python Data Software Engineer II, Glasgow col-narrow-left Client: Location: Glasgow, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Reference: 884ff738a8ca Job Views: 5 Posted: 17.06.2025 Expiry Date: 01.08.2025 col-wide Job Description: We have an exciting and rewarding opportunity within the … Client Onboarding & Know Your Customer technology group, to help implement our next generation data strategy and take your software engineering career to the next level. As a member of our Software Engineering Group, you will dive head-first into creating innovative solutions that advance businesses and careers. You’ll join an inspiring and curious team of technologists dedicated … documentation for complex applications while being accountable for ensuring design constraints are met by software code development. Gathers, analyses, synthesizes, and develops visualizations and reporting from large, diverse data sets in service of continuous improvement of software applications and systems. Proactively identifies hidden problems and patterns in data and uses these insights to drive improvements to More ❯

Social network you want to login/join with: Head of Data Engineering & Governance, glasgow col-narrow-left Client: Data Science Talent Location: glasgow, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 04.06.2025 Expiry Date: 19.07.2025 col-wide Job Description: Head of Data Engineering & Governance … client began to focus solely on wealth management. But now? This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance with data. It’s a … debate about the direction, fighting for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly established Group Data More ❯

Social network you want to login/join with: Head of Data Engineering & Governance, aberdeen col-narrow-left Client: Data Science Talent Location: Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 04.06.2025 Expiry Date: 19.07.2025 col-wide Job Description: Head of Data Engineering & Governance Investment & Wealth Management … client began to focus solely on wealth management. But now? This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance with data. It’s a … debate about the direction, fighting for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly established Group Data More ❯

Social network you want to login/join with: Head of Data Engineering & Governance, edinburgh col-narrow-left Client: Data Science Talent Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 04.06.2025 Expiry Date: 19.07.2025 col-wide Job Description: Head of Data Engineering & Governance … client began to focus solely on wealth management. But now? This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance with data. It’s a … debate about the direction, fighting for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly established Group Data More ❯

technology for insight and efficiency to make our business better and to help clients realize the objectives they desire. At BlackRock, there is a rich problem space for data scientists and engineers across all areas of the business including investments, sales, marketing, operations, product, UX, etc. and the potential to have large scale impact. In 2018, BlackRock accelerated … innovation and technology again with additional investment in artificial intelligence and data science and all the potential they represent. AI Labs was formed to act as a central hub with a firm-wide remit to solve strategic business challenges for the firm by bringing to bear our expertise in data science, machine learning, generative AI, optimization … a history of demonstrating machine learning at BlackRock to create alpha, the goal is to apply these same techniques throughout the business. The team has grown to 30+ data scientists and data engineers. Working collaboratively, the team is multi-disciplinary with the following skills and capabilities: machine learning, statistical modeling, signal detection, natural language processing, data More ❯

With a focus on applied machine learning, generative AI, and cloud-native tools, this is an opportunity to shape the future of the product and how they use data to solve real-world challenges. What You'll Be Doing: AI/ML Project Ownership Lead the delivery of machine learning initiatives from exploration through to deployment and monitoring. … Own the full development cycle for ML systems — including architecture, data pipelines, model training, and production deployment. Cross-functional Collaboration Work closely with Product, Engineering, and DevOps teams to embed intelligent features into our platform. Ensure smooth end-to-end integration of AI/ML functionality within broader product development efforts. Technical Leadership Define best practices for MLOps … of others through coaching, peer programming, and technical reviews. Share learnings and help foster a culture of innovation and experimentation. Problem Solving at Scale Tackle challenges related to data quality, interpretability, bias, and reliability. Build processes and infrastructure that make AI/ML workflows efficient, robust, and repeatable. Contribute to internal documentation and knowledge sharing resources. Refine and More ❯

With a focus on applied machine learning, generative AI, and cloud-native tools, this is an opportunity to shape the future of the product and how they use data to solve real-world challenges. What You'll Be Doing: AI/ML Project Ownership Lead the delivery of machine learning initiatives from exploration through to deployment and monitoring. … Own the full development cycle for ML systems — including architecture, data pipelines, model training, and production deployment. Cross-functional Collaboration Work closely with Product, Engineering, and DevOps teams to embed intelligent features into our platform. Ensure smooth end-to-end integration of AI/ML functionality within broader product development efforts. Technical Leadership Define best practices for MLOps … of others through coaching, peer programming, and technical reviews. Share learnings and help foster a culture of innovation and experimentation. Problem Solving at Scale Tackle challenges related to data quality, interpretability, bias, and reliability. Build processes and infrastructure that make AI/ML workflows efficient, robust, and repeatable. Contribute to internal documentation and knowledge sharing resources. Refine and More ❯

Octopus Investments and Octopus Money. Check out the Seccl website for the latest on our products and our mission to shape the future of investments. The role The Data Squad engineering manager's purpose is to lead and empower a high-performing team of data engineers to deliver scalable, reliable, and secure data capabilities. … This role is central to enabling internal and external data products, supporting business insights, and developing a modern data platform. The role will work closely with Product, Engineering, and Data Operations to align delivery with business outcomes, support the team's growth, and help shape our overall data strategy. On a typical … day you will Guide and mentor, a team of data engineers, fostering their professional growth and ensuring high performance.Setting clear, measurable goals with regular performance reviews, development plans, and recognition. Collaborate with product managers and engineering leaders to define priorities and align data initiatives with business goals. Provide technical guidance and architectural oversight across the data More ❯

its expanding technology team. You’ll contribute to the ongoing development of a next-generation platform, with a strong focus on high-performance search capabilities and large-scale data processing. This is a fantastic opportunity to work on impactful technology in a forward-thinking, data-driven environment. Your Role As a Senior Java Developer, you’ll … be instrumental in building and evolving a resilient data platform that underpins key SaaS offerings. You’ll work closely with engineers, designers, and product teams to deliver scalable, performant solutions that help end users derive real value from complex data. Key Responsibilities Write clean, maintainable, and well-tested code following Agile and TDD practices Contribute to both backend … services (Java) and modern front-end frameworks (React/TypeScript) Develop and fine-tune search functionality using technologies like Lucene, Solr, or OpenSearch Collaborate with Data Engineers to design and deliver data pipelines Leverage cloud platforms (AWS, Azure, or GCP) for scalable and resilient processing Participate in code reviews, pair programming, and mentoring team members Support More ❯

Data Engineering Team Manager Salary: National: £60,000 – £75,000/London: £80,000 – £100,000 At Aviva, data is more than a function—it’s our foundation. We’re on a bold journey to transform how we harness data to drive innovation, power AI, and deliver exceptional outcomes for our customers. As we … accelerate this transformation, we’re looking for a passionate and experienced Data Engineering Team Manager to lead one of our high-performing teams. This is a unique opportunity for someone who thrives on cutting-edge technology and is eager to shape the future of insurance through intelligent data solutions. You’ll lead a talented group of …/London Market is a plus. Agile & Quality Focus: Skilled in Agile delivery, DataOps, and CI/CD, with a commitment to data quality, governance, and scalable pipeline development. What you’ll get for this role: Our purpose - with you today, for a better tomorrow – is a promise we make to our colleagues too. And one of More ❯

We’re Hiring: Lead AI and Data Engineer 📍 Edinburgh (Hybrid) | 💰 £60,000–£68,000 Are you passionate about AI, data engineering , and building systems that actually make an impact ? We’re looking for a strategic and hands-on leader to shape the way our organisation leverages data and AI—from predictive models that drive … your chance to take the lead in a high-impact role where innovation, automation , and engineering excellence come together. 🚀 What You’ll Be Doing ✅ Design and deliver scalable data architecture , pipelines, and semantic models ✅ Build and deploy production-grade AI solutions (predictive & generative) ✅ Champion automation-first engineering and modern DataOps/MLOps practices ✅ Lead the development of AI … data products tied to measurable business outcomes ✅ Manage, mentor, and grow a high-performing data engineering team ✅ Work cross-functionally with architects, analysts, data scientists, and IT ✅ Use tools like Azure AI, Power BI , and open-source frameworks ✅ Stay ahead of the curve with GenAI and cloud advancements 🧠 What You Bring Strong background in More ❯

Job Title: Data Engineer Location: Edinburgh, UK About Us: Join one of the leading utilities companies in Edinburgh, driving innovation through data! We are looking for a skilled Data Engineer to support our data infrastructure and empower our teams with reliable and efficient data solutions. If you’re passionate about … building robust data pipelines and transforming raw data into valuable insights, we want you on our team! Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes. Collaborate with data analysts and stakeholders to understand data requirements. Ensure data integrity, security, and compliance with industry standards. … Optimize data architectures for performance and scalability. Monitor and maintain data systems to ensure high availability. Key Requirements: Proven experience as a Data Engineer or similar role. Proficiency in SQL, Python, or other data processing languages. Experience with cloud platforms (e.g., Azure, AWS, or Google Cloud). Knowledge of data More ❯