1 to 25 of 151 Data Pipeline Jobs in Scotland

Data Engineer - (Python and Power BI) at N Consulting Ltd Job Summary : We are seeking a skilled Data Engineer with expertise in Python development and Power BI to join our growing data team. The ideal candidate will be responsible for designing, building, and maintaining robust data pipelines, transforming raw data … and insights to drive business decisions. Design, develop, and optimize ETL/ELT pipelines using Python and SQL. Develop and maintain Power BI dashboards and reports to visualize data and track KPIs. Work with stakeholders to gather business requirements and translate them into technical solutions. Perform data wrangling, cleaning, and transformation to prepare datasets for analysis. … storage). Ensure data quality, integrity, and security across systems. Collaborate with data analysts, scientists, and engineering teams to support advanced analytics. Monitor and improve pipeline performance, scalability, and reliability. Advanced SQL skills (joins, CTEs, indexing, optimization) Experience with relational databases (e.g., SQL Server, PostgreSQL, MySQL) Understanding of ETL/ELT principles , data More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We’re technology agnostic, and we’re open minded when it More ❯

Join to apply for the Core Data Engineer role at Optima Partners Join to apply for the Core Data Engineer role at Optima Partners The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client base. … You’ll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering. The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client … to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. Key Responsibilities Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data Architecture: Develop and More ❯

constituent of the FTSE 100 index of the London Stock Exchange. The Position Our roster has an opening with your name on it! We are seeking a Staff Data Engineer to lead the technical design and implementation of our most critical data infrastructure and products. In this senior-level individual contributor role, you'll be responsible … for designing scalable systems, setting data architecture standards, and solving complex technical challenges that power analytics, data science, and business function use cases across the company. You will collaborate closely with engineers, product managers, and business stakeholders to architect data solutions that are performant, reliable, and built with a long-term, customer-centric mindset. … batch & streaming data pipelines and architectures that support enterprise-wide data needs Define standards for data modeling, data product design, and pipeline orchestration using modern tools and cloud-native technologies Collaborate with cross-functional stakeholders to translate business and analytical requirements into end-to-end data solutions Establish and More ❯

in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation … Backed by Goldman Sachs Asset Management and Everstone Capital, Apexon now has a global presence of 15 offices (and 10 delivery centers) across four continents. Job Title: Lead Data Engineer with Azure Experience Location: Sunderland, UK About the Role: We are currently looking for a Lead Data Engineer to manage teams on client projects. Your Profile … Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and data transformations including the creation of supplementary documentation. Proficiency in working with APIs and integrating them into More ❯

We are Dufrain, a pure-play data consultancy specialising in helping businesses unlock the true value of their data by providing market-leading data solutions and services which includes developing strategies for AI readiness, improving data literacy and culture, enhancing real-time reporting, and managing data from mergers and acquisitions. … At Dufrain, we pride ourselves on a creative and innovative approach, focusing on delivering exceptional outcomes for clients by leveraging data to drive growth and efficiency. Our mission is to inspire, shape and deliver the data capabilities of tomorrow. We have great opportunities for Data Engineers to play a pivotal role in supporting clients … as they navigate the complexities of data management, analytics, and strategy. Our Data Engineers Possess a broad range of data engineering skills, with a focus on Microsoft Azure, although experience with other cloud platforms is also desirable. Develop good working relationships with clients on a project including interpersonal skills with both business and technical More ❯

The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client base. You'll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering. This is an exciting and valuable opportunity for an enthusiastic, earlier-stage data engineer to advance their career across a variety of commercial sectors. At Optima, you'll work within cross-functional teams, collaborating with experts in different fields and gaining exposure to varied business challenges. Optima provides an inclusive space for professional development, with … to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. Key Responsibilities Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data Architecture: Develop and More ❯

Data Engineering Analyst at N Consulting Ltd Duration: Contract About Us: We're seeking a skilled Data Engineering Analyst to join our team. You'll play a key role in designing, building, and maintaining our data infrastructure, ensuring seamless data flow and insightful analytics. Key Responsibilities: Design, develop, and maintain data pipelines using Google Cloud Platform (GCP) services like BigQuery, Cloud Storage, and Cloud Dataflow. Develop and maintain reports, dashboards, and visualizations using Power BI, ensuring data-driven insights for business stakeholders. Write efficient and scalable Python code for data processing, transformation, and analysis. Collaborate with cross-functional teams to identify data requirements … for data visualization and reporting. Strong programming skills in Python, with experience in data processing libraries like Pandas and NumPy. Experience with data pipeline development, data warehousing, and ETL processes. Strong analytical and problem-solving skills, with attention to detail and ability to work in a fast-paced environment. Excellent communication More ❯

Social network you want to login/join with: Lead Data Scientist (Equity Only) - 1%, edinburgh col-narrow-left Client: Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 4 Posted: 16.06.2025 Expiry Date: 31.07.2025 col-wide Job Description: Lead Data scientist - Equity-Only role 1% About Luupli … marginalized communities. Luupli will highlight the work of diverse creators and provide them access to all the opportunities in the social media space. Job Description: As a Lead Data Scientist at Luupli, you will play a pivotal role in leveraging AWS analytics services to analyse and extract valuable insights from our data sources. You will collaborate … with cross-functional teams, including data engineers, product managers, and business stakeholders, to develop data-driven solutions and deliver actionable recommendations. Your expertise in AWS analytics tools and techniques will be crucial in shaping our data strategy and driving business growth. Responsibilities: Collaborate with cross-functional teams to understand business objectives, identify data More ❯

We are looking for a hands-on, technically driven Data Engineer to join our expanding data team. This role is ideal for someone with a strong foundation in SQL and data management, who has recently graduated from a Data Science or Data Engineering programme and is eager to apply their … skills in a modern, forward-thinking environment. You will be responsible for developing and maintaining robust data solutions that support reporting, automation, and strategic decision-making across the business. The successful candidate will have up-to-date knowledge of data tooling and methodologies, particularly in SQL, Power BI, and data automation, with an interest … in modern data architecture such as cloud-based warehousing and lakehouse solutions. As the role sits within a relatively new department, where methodologies and processes are still evolving, there is a significant opportunity to help shape the future of OEG’s data management practices. Key Responsibilities: Database and Data Engineering Design, develop, and maintain More ❯

We are looking for a hands-on, technically driven Data Engineer to join our expanding data team. This role is ideal for someone with a strong foundation in SQL and data management, who has recently graduated from a Data Science or Data Engineering programme and is eager to apply their … skills in a modern, forward-thinking environment. You will be responsible for developing and maintaining robust data solutions that support reporting, automation, and strategic decision-making across the business. The successful candidate will have up-to-date knowledge of data tooling and methodologies, particularly in SQL, Power BI, and data automation, with an interest … in modern data architecture such as cloud-based warehousing and lakehouse solutions. As the role sits within a relatively new department, where methodologies and processes are still evolving, there is a significant opportunity to help shape the future of OEG’s data management practices. Key Responsibilities: Database and Data Engineering Design, develop, and maintain More ❯

Join to apply for the Lead Data Engineer role at Motability Operations Ltd 4 days ago Be among the first 25 applicants Join to apply for the Lead Data Engineer role at Motability Operations Ltd Continuing Defleet’s data led journey and recognising the importance the Data & Analytics team performs in this … regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions with a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on … large change projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently You’ll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

to help clients realize the objectives they desire. BlackRock manages ~$10+ trillion in assets on behalf of investors worldwide. As such, there is a rich problem space for data scientists and engineers across all areas of the business including investments, sales, marketing, operations, product, UX, etc. and the potential to have large scale impact. In 2018 BlackRock accelerated … innovation and technology again with additional investment in artificial intelligence and data science and all the potential they represent. The AI Labs was formed to act as a central hub with a firm-wide remit to solve strategic business challenges for the firm by bringing to bear our expertise in machine learning, artificial intelligence, data science … a history of demonstrating machine learning at BlackRock to create alpha, the goal was to apply these same techniques throughout the business. The team has grown to 75+ data scientists and engineers. Working collaboratively, the team is multi-disciplinary with the following skills and capabilities: machine learning, optimization, statistical modeling, signal detection, natural language processing, data More ❯

Data Engineering Consultant (all levels) We’re working with a globally recognised consultancy that delivers cutting-edge digital, data, and analytics solutions to some of the most complex infrastructure projects across the UK and beyond. With a strong commitment to technical excellence, inclusion, and client impact, they partner with high-profile organisations to help them unlock … the full potential of their data assets. Due to continued growth and demand for their data services, they are expanding their specialist engineering team and seeking talented Data Engineering Consultants at various seniority levels — from experienced engineers to those ready to step into leadership roles. The Role As a Data Engineering Consultant … solutions, delivery plans, and architectural designs Build and maintain highly automated, scalable data pipelines using modern cloud-based tools Identify and resolve data quality and pipeline performance issues to ensure stability and integrity Contribute to the design and deployment of data lakes, warehouses, and streaming architectures Participate in mentoring and upskilling junior team More ❯

Data Engineering Consultant (all levels) We’re working with a globally recognised consultancy that delivers cutting-edge digital, data, and analytics solutions to some of the most complex infrastructure projects across the UK and beyond. With a strong commitment to technical excellence, inclusion, and client impact, they partner with high-profile organisations to help them unlock … the full potential of their data assets. Due to continued growth and demand for their data services, they are expanding their specialist engineering team and seeking talented Data Engineering Consultants at various seniority levels — from experienced engineers to those ready to step into leadership roles. The Role As a Data Engineering Consultant … solutions, delivery plans, and architectural designs Build and maintain highly automated, scalable data pipelines using modern cloud-based tools Identify and resolve data quality and pipeline performance issues to ensure stability and integrity Contribute to the design and deployment of data lakes, warehouses, and streaming architectures Participate in mentoring and upskilling junior team More ❯

Data Engineering Consultant (all levels) We’re working with a globally recognised consultancy that delivers cutting-edge digital, data, and analytics solutions to some of the most complex infrastructure projects across the UK and beyond. With a strong commitment to technical excellence, inclusion, and client impact, they partner with high-profile organisations to help them unlock … the full potential of their data assets. Due to continued growth and demand for their data services, they are expanding their specialist engineering team and seeking talented Data Engineering Consultants at various seniority levels — from experienced engineers to those ready to step into leadership roles. The Role As a Data Engineering Consultant … solutions, delivery plans, and architectural designs Build and maintain highly automated, scalable data pipelines using modern cloud-based tools Identify and resolve data quality and pipeline performance issues to ensure stability and integrity Contribute to the design and deployment of data lakes, warehouses, and streaming architectures Participate in mentoring and upskilling junior team More ❯

Lead Data Scientist (Equity Only) - 1% Job Description As a Lead Data Scientist at Luupli, you will leverage AWS analytics services to analyze data, develop data-driven solutions, and provide actionable insights. You will work with cross-functional teams, including data engineers, product managers, and stakeholders, to shape our data strategy and support business growth. Responsibilities Collaborate with teams to understand business objectives and define analytics goals. Develop data analysis strategies using AWS services like Amazon Redshift, Athena, EMR, and QuickSight. Design and build data pipelines and ETL processes from diverse sources into AWS. Apply statistical and machine learning techniques for predictive and segmentation … analyses. Create dashboards and KPIs with Amazon QuickSight for decision-making. Perform exploratory data analysis to identify trends and insights. Work with data engineers to optimize data architecture, quality, and governance in AWS. Requirements Bachelor's or master's degree in Computer Science, Statistics, Mathematics, or related field. Experience as a Data More ❯

Lead Data Scientist (Equity Only) - 1% Location: Glasgow, United Kingdom Job Category: Other EU work permit required: Yes Job Views: 4 Posted: 16.06.2025 Expiry Date: 31.07.2025 Job Description: About Luupli Luupli is a social media app 'in-development', which aims to provide opportunities for diverse creators from marginalized communities. It will highlight their work and offer access to … social media opportunities. Job Description: As a Lead Data Scientist, you will leverage AWS analytics services to analyze data, collaborate with cross-functional teams, and develop data-driven solutions to support business growth. Responsibilities: Work with teams to understand objectives and define analytics goals. Develop data analysis strategies using AWS services like … Amazon Redshift, Athena, EMR, and QuickSight. Design data pipelines and ETL processes for data extraction and transformation. Apply statistical and machine learning techniques for predictive analysis, clustering, and pattern recognition. Develop KPIs and dashboards using Amazon QuickSight for decision-making. Perform exploratory data analysis to identify trends and insights. Collaborate on data More ❯

Join to apply for the Principal Data Scientist role at Wood Mackenzie Join to apply for the Principal Data Scientist role at Wood Mackenzie Get AI-powered advice on this job and more exclusive features. Wood Mackenzie is the global data and analytics business for the renewables, energy, and natural resources industries. Enhanced by … Customer committed – we put customers at the heart of our decisions Future Focused – we accelerate change Curious – we turn knowledge into action Wood Mackenzie is an industry-leading data analytics company that provides analysis and insight on the world’s natural resources with a presence all around the globe. We’re working on a next generation data … visualisation platform that enables our customers to drive billion-dollar decisions and accelerate the world's transition to a more sustainable tomorrow. We're looking for a Senior Data Scientist to join our team. This role will be working within our AI group and focus on the research, development and delivery of Generative AI solutions within our products. More ❯

Description Continuing Defleet's data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions … a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science … projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently Minimum criteria You'll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

technologies to improve safety and the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models, perform data cleansing, transformation, and validation. Maintain accurate documentation of data … resources and troubleshooting data-related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency and reliability. Project & Improvement: Assist More ❯

the first 25 applicants Get AI-powered advice on this job and more exclusive features. Direct message the job poster from SR2 | Socially Responsible Recruitment | Certified B Corporation Data Engineering Recruitment Consultant @ SR2 | Certified B Corporation Lead Data Engineer Lead Data Engineer Salary: £85,000–£90,000 +Up to 10% Bonus + 15% Non … Contributory Pension Location: Hybrid working (3 days P/W On-Site) |Edinburgh Level: Lead We’re looking for a Lead Data Engineer to play a key role in expanding our client's data-driven capabilities and driving the evolution of our enterprise data ecosystem. This is a strategic, hands-on position where you … create real business impact. You’re proactive, hands-on, and comfortable working across both technical and business domains. Key technical skills: Strong SQL and ELT/data pipeline development experience Expertise in Data Warehouse & Data Lake design (including Star Schema, Snowflake Schema, Data Vault) Hands-on experience with enterprise databases: Oracle More ❯

EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 10.06.2025 Expiry Date: 25.07.2025 col-wide Job Description: Key Responsibilities Lead the technical delivery of complex data engineering projects, ensuring solutions are scalable, secure, and aligned with our delivery framework, and client goals. Design and build high-quality data pipelines and integration workflows, setting … throughout the development lifecycle. Collaborate with multidisciplinary teams, including a wide range of other roles, to shape solutions that meet both technical and business requirements. Mentor and support data engineering teams, fostering a culture of continuous improvement, knowledge sharing, and technical excellence. Support testing activities by ensuring pipelines are testable, observable, and reliable; work with QA and analysts … to define test strategies, implement automated tests, and validate data quality and integrity. Contribute to technical planning, including estimation, risk assessment, and defining delivery approaches for client engagements and new opportunities. Engage with clients and stakeholders, translating data requirements into technical solutions and communicating complex ideas clearly and effectively. Champion engineering standards, contributing to the development More ❯

Join to apply for the Python Data Engineer III- Machine Learning role at Latinx in AI (LXAI) 2 days ago Be among the first 25 applicants Join to apply for the Python Data Engineer III- Machine Learning role at Latinx in AI (LXAI) Get AI-powered advice on this job and more exclusive features. J Python … Data Engineer III- Machine Learning JPMorganChase Full-time On-site Glasgow, United Kingdom Technology Description Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team. As a Data Engineer III at JPMorgan Chase within the Developer Platforms and Insights team, you serve as a seasoned member of an … agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. You are responsible for developing, testing, and maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives. Job Responsibilities Design, develop, and deploy More ❯

GB Macclesfield, GB Glasgow, GB Company: Royal London Group Contract Type: Permanent Location: Wilmslow or Edinburgh or Glasgow Working style: Hybrid 50% home/office based The Group Data Office (GDO) is responsible for enhancing the effectiveness of how data is used across Royal London (RL). The function provides Data Governance, Data Management, Data Strategy, Analytics/AI and Insight capabilities. It sets and oversees data related policies and standards, so data is collected, processed, and exploited in accordance with the Group’s risk appetite. Role Overview Royal London is seeking experienced data modellers with extensive data warehouse design skills … in building and maintaining data models using data modelling tools (Idera ER/Studio or similar). Data engineering/data pipeline experience, with hands-on experience on integration tools such as Azure Databricks Notebooks, Azure Data Factory or PySpark. Python extremely beneficial. About Royal London We’re the More ❯