1 to 25 of 50 Data Pipeline Jobs in Edinburgh

Data Engineer - (Python and Power BI) at N Consulting Ltd Job Summary : We are seeking a skilled Data Engineer with expertise in Python development and Power BI to join our growing data team. The ideal candidate will be responsible for designing, building, and maintaining robust data pipelines, transforming raw data … and insights to drive business decisions. Design, develop, and optimize ETL/ELT pipelines using Python and SQL. Develop and maintain Power BI dashboards and reports to visualize data and track KPIs. Work with stakeholders to gather business requirements and translate them into technical solutions. Perform data wrangling, cleaning, and transformation to prepare datasets for analysis. … storage). Ensure data quality, integrity, and security across systems. Collaborate with data analysts, scientists, and engineering teams to support advanced analytics. Monitor and improve pipeline performance, scalability, and reliability. Advanced SQL skills (joins, CTEs, indexing, optimization) Experience with relational databases (e.g., SQL Server, PostgreSQL, MySQL) Understanding of ETL/ELT principles , data More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We’re technology agnostic, and we’re open minded when it More ❯

Join to apply for the Core Data Engineer role at Optima Partners Join to apply for the Core Data Engineer role at Optima Partners The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client base. … You’ll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering. The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client … to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. Key Responsibilities Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data Architecture: Develop and More ❯

constituent of the FTSE 100 index of the London Stock Exchange. The Position Our roster has an opening with your name on it! We are seeking a Staff Data Engineer to lead the technical design and implementation of our most critical data infrastructure and products. In this senior-level individual contributor role, you'll be responsible … for designing scalable systems, setting data architecture standards, and solving complex technical challenges that power analytics, data science, and business function use cases across the company. You will collaborate closely with engineers, product managers, and business stakeholders to architect data solutions that are performant, reliable, and built with a long-term, customer-centric mindset. … batch & streaming data pipelines and architectures that support enterprise-wide data needs Define standards for data modeling, data product design, and pipeline orchestration using modern tools and cloud-native technologies Collaborate with cross-functional stakeholders to translate business and analytical requirements into end-to-end data solutions Establish and More ❯

in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation … Backed by Goldman Sachs Asset Management and Everstone Capital, Apexon now has a global presence of 15 offices (and 10 delivery centers) across four continents. Job Title: Lead Data Engineer with Azure Experience Location: Sunderland, UK About the Role: We are currently looking for a Lead Data Engineer to manage teams on client projects. Your Profile … Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and data transformations including the creation of supplementary documentation. Proficiency in working with APIs and integrating them into More ❯

We are Dufrain, a pure-play data consultancy specialising in helping businesses unlock the true value of their data by providing market-leading data solutions and services which includes developing strategies for AI readiness, improving data literacy and culture, enhancing real-time reporting, and managing data from mergers and acquisitions. … At Dufrain, we pride ourselves on a creative and innovative approach, focusing on delivering exceptional outcomes for clients by leveraging data to drive growth and efficiency. Our mission is to inspire, shape and deliver the data capabilities of tomorrow. We have great opportunities for Data Engineers to play a pivotal role in supporting clients … as they navigate the complexities of data management, analytics, and strategy. Our Data Engineers Possess a broad range of data engineering skills, with a focus on Microsoft Azure, although experience with other cloud platforms is also desirable. Develop good working relationships with clients on a project including interpersonal skills with both business and technical More ❯

The Role As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client base. You'll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering. This is an exciting and valuable opportunity for an enthusiastic, earlier-stage data engineer to advance their career across a variety of commercial sectors. At Optima, you'll work within cross-functional teams, collaborating with experts in different fields and gaining exposure to varied business challenges. Optima provides an inclusive space for professional development, with … to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. Key Responsibilities Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data Architecture: Develop and More ❯

Data Engineering Analyst at N Consulting Ltd Duration: Contract About Us: We're seeking a skilled Data Engineering Analyst to join our team. You'll play a key role in designing, building, and maintaining our data infrastructure, ensuring seamless data flow and insightful analytics. Key Responsibilities: Design, develop, and maintain data pipelines using Google Cloud Platform (GCP) services like BigQuery, Cloud Storage, and Cloud Dataflow. Develop and maintain reports, dashboards, and visualizations using Power BI, ensuring data-driven insights for business stakeholders. Write efficient and scalable Python code for data processing, transformation, and analysis. Collaborate with cross-functional teams to identify data requirements … for data visualization and reporting. Strong programming skills in Python, with experience in data processing libraries like Pandas and NumPy. Experience with data pipeline development, data warehousing, and ETL processes. Strong analytical and problem-solving skills, with attention to detail and ability to work in a fast-paced environment. Excellent communication More ❯

Social network you want to login/join with: Lead Data Scientist (Equity Only) - 1%, edinburgh col-narrow-left Client: Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 4 Posted: 16.06.2025 Expiry Date: 31.07.2025 col-wide Job Description: Lead Data scientist - Equity-Only role 1% About Luupli … marginalized communities. Luupli will highlight the work of diverse creators and provide them access to all the opportunities in the social media space. Job Description: As a Lead Data Scientist at Luupli, you will play a pivotal role in leveraging AWS analytics services to analyse and extract valuable insights from our data sources. You will collaborate … with cross-functional teams, including data engineers, product managers, and business stakeholders, to develop data-driven solutions and deliver actionable recommendations. Your expertise in AWS analytics tools and techniques will be crucial in shaping our data strategy and driving business growth. Responsibilities: Collaborate with cross-functional teams to understand business objectives, identify data More ❯

Join to apply for the Lead Data Engineer role at Motability Operations Ltd 4 days ago Be among the first 25 applicants Join to apply for the Lead Data Engineer role at Motability Operations Ltd Continuing Defleet’s data led journey and recognising the importance the Data & Analytics team performs in this … regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions with a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on … large change projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently You’ll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

to help clients realize the objectives they desire. BlackRock manages ~$10+ trillion in assets on behalf of investors worldwide. As such, there is a rich problem space for data scientists and engineers across all areas of the business including investments, sales, marketing, operations, product, UX, etc. and the potential to have large scale impact. In 2018 BlackRock accelerated … innovation and technology again with additional investment in artificial intelligence and data science and all the potential they represent. The AI Labs was formed to act as a central hub with a firm-wide remit to solve strategic business challenges for the firm by bringing to bear our expertise in machine learning, artificial intelligence, data science … a history of demonstrating machine learning at BlackRock to create alpha, the goal was to apply these same techniques throughout the business. The team has grown to 75+ data scientists and engineers. Working collaboratively, the team is multi-disciplinary with the following skills and capabilities: machine learning, optimization, statistical modeling, signal detection, natural language processing, data More ❯

Join to apply for the Principal Data Scientist role at Wood Mackenzie Join to apply for the Principal Data Scientist role at Wood Mackenzie Get AI-powered advice on this job and more exclusive features. Wood Mackenzie is the global data and analytics business for the renewables, energy, and natural resources industries. Enhanced by … Customer committed – we put customers at the heart of our decisions Future Focused – we accelerate change Curious – we turn knowledge into action Wood Mackenzie is an industry-leading data analytics company that provides analysis and insight on the world’s natural resources with a presence all around the globe. We’re working on a next generation data … visualisation platform that enables our customers to drive billion-dollar decisions and accelerate the world's transition to a more sustainable tomorrow. We're looking for a Senior Data Scientist to join our team. This role will be working within our AI group and focus on the research, development and delivery of Generative AI solutions within our products. More ❯

This job is brought to you by Jobs/Redefined, the UK's leading over-50s age inclusive jobs board. Job Title: Senior Data Modeller Contract Type: Permanent Location: Wilmslow or Edinburgh or Glasgow Working style: Hybrid 50% home/office based The Group Data Office (GDO) is responsible for enhancing the effectiveness of how data is used across Royal London (RL). The function provides Data Governance, Data Management, Data Strategy, Analytics/AI and Insight capabilities. It sets and oversees data related policies and standards, so data is collected, processed, and exploited in accordance with the Group's risk appetite. Role Overview … models using data modelling tools (Idera ER/Studio or similar). Expert level SQL skills required. Data engineering/data pipeline experience, with hands on experience on integration tools such as Azure Databricks Notebooks, Azure Data Factory or PySpark. Python extremely beneficial. About Royal London We're the More ❯

Description Continuing Defleet's data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions … a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science … projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently Minimum criteria You'll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

technologies to improve safety and the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models, perform data cleansing, transformation, and validation. Maintain accurate documentation of data … resources and troubleshooting data-related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency and reliability. Project & Improvement: Assist More ❯

the first 25 applicants Get AI-powered advice on this job and more exclusive features. Direct message the job poster from SR2 | Socially Responsible Recruitment | Certified B Corporation Data Engineering Recruitment Consultant @ SR2 | Certified B Corporation Lead Data Engineer Lead Data Engineer Salary: £85,000–£90,000 +Up to 10% Bonus + 15% Non … Contributory Pension Location: Hybrid working (3 days P/W On-Site) |Edinburgh Level: Lead We’re looking for a Lead Data Engineer to play a key role in expanding our client's data-driven capabilities and driving the evolution of our enterprise data ecosystem. This is a strategic, hands-on position where you … create real business impact. You’re proactive, hands-on, and comfortable working across both technical and business domains. Key technical skills: Strong SQL and ELT/data pipeline development experience Expertise in Data Warehouse & Data Lake design (including Star Schema, Snowflake Schema, Data Vault) Hands-on experience with enterprise databases: Oracle More ❯

EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 10.06.2025 Expiry Date: 25.07.2025 col-wide Job Description: Key Responsibilities Lead the technical delivery of complex data engineering projects, ensuring solutions are scalable, secure, and aligned with our delivery framework, and client goals. Design and build high-quality data pipelines and integration workflows, setting … throughout the development lifecycle. Collaborate with multidisciplinary teams, including a wide range of other roles, to shape solutions that meet both technical and business requirements. Mentor and support data engineering teams, fostering a culture of continuous improvement, knowledge sharing, and technical excellence. Support testing activities by ensuring pipelines are testable, observable, and reliable; work with QA and analysts … to define test strategies, implement automated tests, and validate data quality and integrity. Contribute to technical planning, including estimation, risk assessment, and defining delivery approaches for client engagements and new opportunities. Engage with clients and stakeholders, translating data requirements into technical solutions and communicating complex ideas clearly and effectively. Champion engineering standards, contributing to the development More ❯

Description As a Data Solutions Architect you'll work closely with key stakeholders to map out their data/insights requirements and develop roadmaps and design architecture which you'll communicate effectively to the project teams, implementation teams and end-users - leading top quality solutions design and delivery. Requirements Strong experience as a Cloud data … more of the following architectural frameworks (TOGAF, ZACHMAN, FEAF) Cloud Experience: AWS or GCP preferred, particularly around migrations and cloud architecture Good technical knowledge and understanding of big data frameworks like Hadoop, Cloudera etc. Deep technical knowledge of database development, design and migration Experience of deployment in cloud using Terraform or CloudFormation Automation or Scripting experience using languages … such as Python & Bash etc. ETL and workflow management knowledge Experience of Agile methodologies Experience in the Financial Services Sector Data Engineering or Data Science experience Job responsibilities Interface with client project sponsors to gather, assess and interpret client needs and requirements. Define architecture vision, roadmaps and strategy while engaging relevant stakeholders to execute. Work across More ❯

join us at the heart of trust! As an Analytics Engineer at Trustpilot, you'll be a crucial bridge between our business and technical teams, directly influencing Analytics, Data Science, Data Engineering, and Product. You'll leverage our cutting-edge modern data stack to expand and enhance our data warehouse, empowering data-driven decisions across the organisation. This role offers the exciting opportunity to shape our data landscape and contribute to innovative analytics solutions. What you'll be doing: Maintain and evolve our data warehouse and business intelligence tool, providing reliable data access to users throughout Trustpilot Design, build, maintain, and rigorously monitor robust … data pipelines and transformative models using our modern data stack, including GCP, Airflow, dbt, and potentially emerging technologies like real-time streaming platforms Develop and manage reverse ETL processes to seamlessly integrate data with our commercial systems, ensuring operational efficiency Maintain and optimise our Customer Data Platform (CDP, ensuring effective data More ❯

Job Description Lead AI and Data Engineer (Edinburgh) Our client is looking for a technical and strategic AI and Data Engineer to take ownership of AI and data engineering practices. You’ll play a central role in shaping modern data platforms, building production-grade AI solutions, and embedding best practices in data and AI engineering. Key Responsibilities: Lead the design and delivery of scalable data architectures and pipelines to meet analytics and AI needs. Develop and maintain AI and data products aligned to business outcomes. Champion automation-first engineering practices across ingestion, transformation, and deployment. Design and deploy generative and predictive AI solutions using Azure and … open-source frameworks. Implement modern DataOps and MLOps practices, including CI/CD pipelines and monitoring. Establish and maintain reusable enterprise data models and semantic layers for self-service analytics. Mentor and develop a high-performing engineering team. Collaborate with stakeholders to embed insight and AI in product delivery and business decision-making. About You: Deep expertise in More ❯

Lead Data Engineer Salary: £85,000–£90,000 + Up to 10% Bonus + 15% Non - Contributory Pension Location: Hybrid working (3 days P/W On-Site) | Edinburgh Level: Lead We’re looking for a Lead Data Engineer to play a key role in expanding our client's data-driven capabilities and driving … the evolution of our enterprise data ecosystem. This is a strategic, hands-on position where you’ll lead the design and delivery of large-scale data solutions that power critical business decisions. You'll work with a high-performing Data & Analytics team on mission-critical systems across Data Warehousing, Data … create real business impact. You’re proactive, hands-on, and comfortable working across both technical and business domains. ✅ Key technical skills: Strong SQL and ELT/data pipeline development experience Expertise in Data Warehouse & Data Lake design (including Star Schema, Snowflake Schema, Data Vault) Hands-on experience with enterprise databases: Oracle More ❯

employee exit rate is astonishingly low. Join us on our journey... Let’s create something awesome, together, today. About the role and team: We’re looking for a Data Engineer to join our growing Data Practice. You’ll work as part of a multidisciplinary team … delivering data solutions for some of the UK’s best-known brands. You’ll have a hand in everything from data ingestion and transformation to pipeline automation and data platform optimisation – always with a strong focus on quality, scalability, and value. What you'll be doing: Designing, building and maintaining robust, cloud-native … deliver value-focused data solutions We'd love to talk to you if: You’ve got solid experience working with Python, SQL, Spar and data pipeline tools such as dbt or Airflow You’re comfortable working across cloud platforms – especially AWS (Glue, Lambda, Athena), Azure (Data Factory, Synapse), or GCP (Dataflow, BigQuery, Cloud More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing … etc. They are looking for a senior Data Engineer to join their FinTech team! You will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing … Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements More ❯

Data Engineering Manager/Edinburgh/Microsoft Gold Partner Data Engineering Manager/Edinburgh/Microsoft Gold Partner Direct message the job poster from Opus Recruitment Solutions Recruitment Consultant | Microsoft Dynamics - CRM/CE at Opus Recruitment Are you a passionate Data Engineering Manager ready to unlock the power of data? Our … client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For: Proven Data Engineering experience (5+ years). Consultancy experience is a must. Leadership and multi-project More ❯

Social network you want to login/join with: Head of Data Engineering & Governance, edinburgh col-narrow-left Client: Data Science Talent Location: edinburgh, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 04.06.2025 Expiry Date: 19.07.2025 col-wide Job Description: Head of Data Engineering & Governance … client began to focus solely on wealth management. But now? This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance with data. It’s a … debate about the direction, fighting for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly established Group Data More ❯