76 to 100 of 127 Data Pipeline Jobs in Scotland

Are you a passionate Data Engineering Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end … to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For … Proven Data Engineering experience (5+ years). Consultancy experience is a must. Leadership and multi-project environments experience. Expertise in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, data lakes, and data warehouses. Hands-on experience with Apache Spark and bonus points More ❯

Social network you want to login/join with: Data Engineering Manager/Edinburgh/Microsoft Gold Partner, edinburgh col-narrow-left Client: Opus Recruitment Solutions Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 2 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Are you a passionate Data Engineering Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation … and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For: Proven Data Engineering More ❯

Social network you want to login/join with: Data Engineering Manager/Edinburgh/Microsoft Gold Partner, dunfermline col-narrow-left Client: Opus Recruitment Solutions Location: Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 2 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Are you a passionate Data Engineering … Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with … clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For: Proven Data Engineering experience (5+ years More ❯

Social network you want to login/join with: Data Engineering Manager/Edinburgh/Microsoft Gold Partner, livingston col-narrow-left Client: Opus Recruitment Solutions Location: livingston, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 2 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Are you a passionate Data Engineering Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation … and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For: Proven Data Engineering More ❯

Social network you want to login/join with: Data & AI Solutions Deployment Architect, glasgow col-narrow-left Client: Bytes Software Services Location: glasgow, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 31.05.2025 Expiry Date: 15.07.2025 col-wide Job Description: Bytes is a top provider of premium IT solutions … foster a culture of innovation, collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just the theory) grasp of Azure … Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and C#. This role would suit an experienced Data platform engineer or More ❯

Social network you want to login/join with: Data & AI Solutions Deployment Architect, edinburgh col-narrow-left Client: Bytes Software Services Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 8 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Bytes is a top provider of premium IT solutions … foster a culture of innovation, collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just the theory) grasp of Azure … Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and C#. This role would suit an experienced Data platform engineer or More ❯

foster a culture of innovation, collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just the theory) grasp of Azure … Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and C#. This role would suit an experienced Data platform engineer or … Data Solutions Architect who thrives in working across multiple projects with a diverse range of clients, and who places the importance of bridging the gap between complex data problems and real life product requirements to non-technical stakeholders. KEY RESPONSIBILITIES: Have 5+ years of technical consulting or a similar chronology of demonstrable success in coding and More ❯

Social network you want to login/join with: Senior Backend Engineer (Platform Experience & Data Warehousing teams) ??? Location: UK, Ireland, Poland, Lithuania, Spain, Portugal, Finland ? numi are proud to have partnered with a front runner in data integration technology providing fantastic experience for their clients (who are some of the biggest names in the space)! We … are looking for experienced Senior Backend Engineers (PHP) to join either our Platform Experience or Data Warehousing teams. The Platform Experience team focuses on optimizing end-to-end platform journeys and building impactful customer experiences. They are currently developing a data visualization tool (dashboards) and build features primarily from scratch. The Data Warehousing team … is responsible for data storage and transfer solutions, helping users streamline data transfers to data warehouses/datalakes and improving internal storage. This involves scaling, refactoring, and enhancing a complex data transfer system handling large data volumes. As a Senior Backend Engineer, you will play a key role in the More ❯

Social network you want to login/join with: Senior Backend Engineer (Platform Experience & Data Warehousing teams) ??? Location: UK, Ireland, Poland, Lithuania, Spain, Portugal, Finland ? numi are proud to have partnered with a front runner in data integration technology providing fantastic experience for their clients (who are some of the biggest names in the space)! We … are looking for experienced Senior Backend Engineers (PHP) to join either our Platform Experience or Data Warehousing teams. The Platform Experience team focuses on optimizing end-to-end platform journeys and building impactful customer experiences. They are currently developing a data visualization tool (dashboards) and build features primarily from scratch. The Data Warehousing team … is responsible for data storage and transfer solutions, helping users streamline data transfers to data warehouses/datalakes and improving internal storage. This involves scaling, refactoring, and enhancing a complex data transfer system handling large data volumes. As a Senior Backend Engineer, you will play a key role in the More ❯

many ways. Here are some examples to give you a flavour of what you will do: Identify and define technical tasks from project requirements Implement machine learning models, data pipelines, and integrations with Storm's product suite Investigate new AI approaches, frameworks, and best practices for application in public sector contexts Work closely with AI Consultants to refine More ❯

Data Engineering Associate Salary: National £40,000 - £50,000/London £50,000 - £60,000 Are you energised by solving complex data challenges and excited to shape the future of insurance through innovation? At Aviva, we’re looking for a Data Engineer Associate to join our Quantum Data Science Practice—a collaborative … forward-thinking team transforming how data powers decisions across the organisation. You’ll be part of a vibrant Data Engineering community, working in a multi-cloud environment with the latest technologies. This is your chance to grow your skills, contribute to cutting-edge analytics and data science, and help deliver smarter, faster, and more … re Looking For: Proficiency in SQL, with the ability to write clean, efficient, and maintainable code; familiarity with Python is a plus. Hands-on experience in data pipeline development and continuous delivery, ideally using Snowflake and/or DBT. Strong grasp of cloud-based data engineering conceptsand modern best practices. Expertise in data More ❯

Social network you want to login/join with: A leading Financial Services client in the City of London is now seeking an experienced Data Engineering Manager to join on a permanent basis. This role is offering a base of £85,000 + a strong benefits package and flexible working. The ideal Data Engineering Manager will … come from a data engineering background and have strong knowledge in SQL, Snowflake, Microsoft Azure, Azure Data Factory and Azure DevOps. The Engineering Manager will design, improve and maintain robust data pipelines within data architecture. To be considered for this role you will need the following: Experience designing, improving and maintaining robust … data pipelines Strong SQL programming skills. Knowledge of other programming languages such as Python, C++ and Java beneficial Possesses a strong understanding of Snowflake - beneficial Experience managing small teams of Data Engineers Strong experience working in a cloud environment and knowledge in the following very beneficial: Microsoft Azure, Azure Data Factory and Azure DevOps More ❯

Data Engineer| 6 Month Contract | (Inside IR35) | Hybrid - Edinburgh | Starting ASAP Day Rate: DOE Role Overview Harvey Nash are seeking a highly skilled and experienced Data Engineer to join our team. This role is ideal for someone with a strong background in automated data processing, relational databases, and real-time data feeds … using Python. You will be responsible for designing and implementing data pipelines, developing interactive dashboards, and delivering insightful analytics to support decision-making across the University. You will collaborate closely with internal team members and stakeholders from across the University, gathering requirements, documenting specifications, and presenting your work to both technical and non-technical audiences. Key Responsibilities Design … and implement automated data processing pipelines using Python. Develop and maintain relational database schemas and data models. Integrate real-time data feeds and API-based data sources. Create and manage web-based data visualisation dashboards. Collaborate with stakeholders to understand data requirements and translate them into actionable insights. More ❯

We are seeking a talented Data Scientist to join our team, dedicated to driving commercial value from data science across the Corporate Bank. In this role, you will have the opportunity to leverage machine learning techniques and advanced analytics to extract valuable insights and create tangible business impact. We embrace a design-led thinking approach that … with business stakeholders and a fail-fast mindset. This position offers the chance to work with extensive datasets that span billions of rows, applying state-of-the-art data science techniques to unlock millions in business value. To be successful as a Data Scientist, you should have experience with: Commercial or academic experience in developing machine … learning models (e.g., clustering, classification, regression, NLP). Comfortable working in Python and using key data science packages. Working knowledge of SQL. Ability to articulate technical findings in business terms and demonstrate value to stakeholders. Some other highly valued skills include: Understanding of code best practice including Git. Familiarity with AWS data science services, including SageMaker. More ❯

CI/CD pipelines to automate deployment processes. Monitor system performance and troubleshoot issues using CloudWatch and other observability tools. Manage and optimise Kafka clusters for real-time data streaming. Oversee and maintain containerized workloads using EKS (Kubernetes on AWS) . Support data infrastructure, including Amazon Redshift for analytics and storage. Implement security best practices and … fixes and provide Level 2 technical support. Build tools to reduce the occurrence of errors and improve customer experience. Develop software to integrate with internal back-end systems (data). Perform root cause analysis of production errors and resolve technical issues. Develop scripts to automate visualisation (management tools such as Sentry and Grafana). Design procedures for system … best practices. Experience with version control systems such as Git . Ability to troubleshoot complex system issues in a production environment. Nice to Have Knowledge of data pipeline and event-driven architectures. Experience with serverless computing (AWS Lambda, Fargate). Familiarity with compliance and security frameworks (ISO 27001, SOC2, etc.). Experience working in an Agile/ More ❯

Social network you want to login/join with: AI Connect are partnering with a boutique, data enabled organisation to deliver high-impact machine learning projects. We're looking for Senior Data Science contractors who can hit the ground running and build production-ready solutions using Python. Role responsibilities/experience required: • Lead the development of … data science solutions • Scope technical tasks and shape project deliverables with stakeholders • Build and optimise traditional machine learning models • Perform complex data wrangling, cleansing, and enrichment • Design and evaluate models with a focus on robustness and interpretability • Develop scalable data pipelines and work with large datasets • Collaborate closely with engineering and product to embed … solutions into workflows • Strong experience in applied Data Science and machine learning • Track record of delivering commercial ML solutions • Ability to scope work independently and drive delivery • Bonus: cloud experience (Azure) If you have the required skills and experience, then please apply for more details #J-18808-Ljbffr More ❯

Join us as a Data and Software Engineer at Barclays where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionise our digital offerings, ensuring unapparelled customer experiences. This role will leverage technologies such as Kafka and Java and others listed below, to improve our legacy technology … stack. You will play a key role in the input of the design and creation of these initial systems. To be successful as a Data and Software Engineer , you should have experience with: Kafka developing Java developing Databases (e.g. Snowflake, MongoDB) Other highly valued skills include: Cloud Platform (e.g. AWS) Flink/Kstreams Linux/Shell You may … technical skills This role will be based out of our Glasgow campus. Purpose of the role To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure. Accountabilities Build More ❯

Location: Birmingham, Bristol, Cambridge, Cardiff, Edinburgh, Gatwick, Glasgow, Leeds, Liverpool, Manchester, Milton Keynes, Newcastle, Reading, Southampton Business Line: Enabling Functions Date published: 28-May-2025 Job Title: Data Analyst/Technical Business Analyst Connect to your Industry: Do you enjoy being the “voice of the business" ensuring that our technology teams deliver solutions that meet business needs? If … so, this opportunity is for you! Deloitte is looking for a mid-level Data Analyst/Technical Business Analyst to join its talented Business Change & Analysis team. Connect to your career at Deloitte: Deloitte drives progress. Using our expertise, we help clients become leaders. We invest in outstanding people, build diverse teams, and empower them to achieve more. … firm, using traditional and innovative tools to enable positive transformation. Industry experience is welcomed but not required. Your role involves ensuring solutions meet business and technical needs, transforming data into insights for strategic decisions, understanding business processes, eliciting requirements, and improving user experience and technical quality. You will also support the Head of Business Analysis in practice development More ❯

Who We Are Addepar is a global technology and data company that helps investment professionals provide the most informed, precise guidance for their clients. Hundreds of thousands of users have entrusted Addepar to empower smarter investment decisions and better advice over the last decade. With client presence in more than 45 countries, Addepar's platform aggregates portfolio, market … and client data for over $7 trillion in assets. Addepar's open platform integrates with more than 100 software, data and services partners to deliver a complete solution for a wide range of firms and use cases. Addepar embraces a global flexible workforce model with offices in Silicon Valley, New York City, Salt Lake City, Chicago … London, Edinburgh and Pune. The Role We are seeking a Head of Portfolio Data Operations who excels in strategic problem-solving, innovative thinking, and strategic execution, with a proven track record of building impactful teams focused on operational excellence. This role is based in Addepar's Edinburgh, UK office! As part of the wider Addepar Data More ❯

Job Description Purpose of the role To implement data quality process and procedures, ensuring that data is reliable and trustworthy, then extract actionable insights from it to help the organisation improve its operation, and optimise resources. Accountabilities Investigation and analysis of data issues related to quality, lineage, controls, and authoritative source identification. Execution of … data cleansing and transformation tasks to prepare data for analysis. Designing and building data pipelines to automate data movement and processing. Development and application of advanced analytical techniques, including machine learning and AI, to solve complex business problems. Documentation of data quality findings and recommendations for improvement. Vice President Expectations … They will also be expected to demonstrate the Barclays Mindset - to Empower, Challenge and Drive - the operating manual for how we behave. Join us at Barclays as a Data Business Partner. In this role you will be acting within a team of data partners and analysts, you will be responsible for the delivery of an evolving More ❯

Join to apply for the Data Business Partner role at Barclays Business Banking 2 days ago Be among the first 25 applicants Join to apply for the Data Business Partner role at Barclays Business Banking Get AI-powered advice on this job and more exclusive features. Join us at Barclays as a Data Business … Partner. In this role you will be acting within a team of data partners and analysts, you will be responsible for the delivery of an evolving capability within the Chief Data Office (CDO) focused on meeting data quality and control standards across multiple areas of the business. This is a unique role for an … individual with strong project delivery experience, but also a willingness to investigate business challenges at a detailed level across data sets, operational processes, business controls and applications. You will need the ability to perform detailed impact assessments, development of business requirement and business cases; with the key goal to be providing best in class data advisory More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Social network you want to login/join with: Who We Are Solera is a global leader in data and software services that strives to transform every touchpoint of the vehicle lifecycle into a connected digital experience. We provide products and services to protect assets like homes and digital identities. Processing over 300 million digital transactions annually for … insights, algorithms, and automation. For more information, visit solera.com. The Role We are seeking a versatile, all-rounder with experience and interest in Embedded Systems, Machine Learning, and Data Engineering to help design and build Convolutional Neural Networks for Smart Sensors, develop and troubleshoot firmware for automotive IoT devices, and perform data engineering tasks such as … data workflow management, gathering, and analysis. Spireon develops computer vision-based smart sensors for environmental detection and reporting in the transportation sector. In this role, you will participate in all project phases from machine learning to data engineering and embedded systems development, collaborating across teams to meet project objectives. What You’ll Do Responsibilities: Embedded Systems More ❯

regulatory reporting solutions. While a large part of our work involves building and maintaining ETL adapters between Avaloq and external systems, were also responsible for native reporting logic, data model extensions, and ongoing support of core legal reporting features. If you're passionate about data pipelines, financial systems, and regulatory tech, this role offers both depth … and diversity. What Youll Do Design, develop, and maintain data integration components (ETL/adapters) connecting Avaloqs data model to third-party reporting platforms, and native reporting solutions built directly within Avaloq Collaborate with business stakeholders and internal tech teams to ensure regulatory requirements are translated into robust, scalable reporting logic Drive improvements in PL/… SQL logic, data structures, and mapping layers Contribute to ongoing efforts around data quality, performance tuning, and compliance-driven automation Act as a senior engineer within the team, offering technical guidance and support to colleagues Required profile: Strong proven track record of working in a similar role at a senior level Strong experience with relational databases More ❯

reporting solutions. While a large part of our work involves building and maintaining ETL adapters between Avaloq and external systems, we’re also responsible for native reporting logic, data model extensions, and ongoing support of core legal reporting features. If you're passionate about data pipelines, financial systems, and regulatory tech, this role offers both depth … and diversity. What You’ll Do Design, develop, and maintain data integration components (ETL/adapters) connecting Avaloq’s data model to third-party reporting platforms, and native reporting solutions built directly within Avaloq Collaborate with business stakeholders and internal tech teams to ensure regulatory requirements are translated into robust, scalable reporting logic Drive improvements in … PL/SQL logic, data structures, and mapping layers Contribute to ongoing efforts around data quality, performance tuning, and compliance-driven automation Act as a senior engineer within the team, offering technical guidance and support to colleagues Qualifications Strong proven track record of working in a similar role at a senior level Strong experience with relational More ❯