1 to 25 of 25 Data Pipeline Jobs in Scotland

Are you an experienced Data Engineer ? Would you like to work for a fast growing business that specialises in Business Intelligence & Data Analytics ? Its a Hybrid working role and you will be based from either their London, Leeds, Manchester or Edinburgh office, typically 3 days a … week from home and 2 day in office/on client site. Responsibilities: Developing and optimizing data pipelines Building and maintaining data architectures Leading projects and teams Ensuring data quality and integrity Preferred skills: Proficiency in AWS Glue, AWS Redshift, and Python Experience with … ETL processes, data integration, and data warehousing. Strong SQL skills Experience with Big Data technologies such as Hadoop, Spark, and Kafka Familiarity with cloud platforms (AWS, Azure, Google Cloud) Working knowledge of data visualisation tools (PowerBI, Tableau, Qlik Sense) Additional skills: Client more »

AI Labs team works collaboratively; and is a multi-disciplinary team with the following skills and capabilities: optimization, machine learning, statistical modeling, exploratory data analysis, natural language processing, data visualization, network/graph modeling, ETL, data pipelines, data architecture, communication, project/… product management and strategy. We work with data from a wide variety of sources including text, news feeds, financial reports, time series transactions, user behavior logs, imagery, and real-time data. AI Labs has offices in New York, Palo Alto, and Edinburgh. The team has several Stanford professors … enable the rapid delivery of machine learning products to production environments Continuously improve existing products, tools, and services, by maintaining close relationships with data scientists, researchers, and other BlackRock users Write quality maintainable code that is comprehensively tested and fit for the intended objective Provide ongoing maintenance and more »

Job Title :: Pyspark Data Engineer Job Type :: Permanent Work Model :: Edinburgh, Scotland, United Kingdom (HYBRID) Please find below the JD of Pyspark Data Engineer – Coding skills: Pyspark or Scala(level to troubleshoot issues) and SQL. Must have good pyspark knowledge to build data transformation. … Hands on Azure data platforms i.e; Azure Datafactory, Databricks, Synapse, SQL, Data lake storage. Experience building and optimizing ‘big data’ data pipelines, architectures and data sets. Must have experience on production environment specific to data processing and activities … like Monitoring, Troubleshoot, Deployments. Good understanding on Data Cleansing, Ingestion ,Transformation and Visualization(PowerBI) flow. Good Data analytics skill and able to communicate with customers/business users. more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … Azure or AWS cloud services, and a proven track record in delivering high quality work to tight deadlines. You will work closely with data scientists, software engineers, and domain experts, to gather the business needs and translate them into AI applications. Designing, testing, evaluating, and deploying cutting-edge … efficiency of various domains. There will also be the opportunity to research and implement best practices and standards for AI development, such as data quality, scalability, security, and ethics. SC Clearance/Eligibility required. Qualifications Experience in developing and deploying AI solutions using Python, TensorFlow, PyTorch, Keras, Snowflake more »

Senior Data Engineer £80k-£90k Hybrid working 2 days in Glasgow office a week Your New … Role: Platform Development: Build the foundations of the new data platform, leveraging extensive business data for both automation and analytics. Pipeline Design: Create and develop data processing pipelines to support the ingestion, enrichment, and quality control of client and partner data. Data Warehouse Design: Design and develop a data warehouse to facilitate reporting and analytics. Team Development: Coach and mentor team members, enhancing their technical skills and knowledge of the platform. Process Improvement: Identify and implement internal process improvements, automate manual processes, and optimize data delivery. more »

Are you an experienced Data Engineer ? Would you like to work for a fast growing business that specialises in Business Intelligence & Data Analytics ? Its a Hybrid working role and you will be based from either their London, Leeds, Manchester or Edinburgh office, typically 3 days a … site. Preferred skills: SAS Data Engineering Proficiency in SQL and/or similar data technologies Familiarity with data pipeline tools and ETL processes Knowledge of cloud platforms and data architecture Experience working in the Retail Banking sector or financial services would more »

Job Description for Data Engineer - Coding skills: Pyspark or Scala(level to troubleshoot issues) and SQL. Must have good pyspark knowledge to build data transformation. Hands on Azure data platforms i.e; Azure Datafactory, Databricks, Synapse, SQL, Data lake storage. Experience building and … optimizing ‘big data’ data pipelines, architectures and data sets. Must have experience on production environment specific to data processing and activities like Monitoring, Troubleshoot, Deployments. Good understanding on Data Cleansing, Ingestion ,Transformation and Visualization(PowerBI) flow. Good Data more »

Senior Data Engineer Glasgow Based/Hybrid working 1 day a week in office. About the Opportunity: MBN are partnered with a growing Glasgow based company to find them a Senior data Engineer. They are entering a new phase of development aimed at refining their product … platform for the future as part of a long-term business expansion strategy. They are looking for a Data Engineer to join and lead the development and strategy of their platform and functionality. You new role : Act as the Subject Matter Expert for Data Engineering, helping … to set the standards for data engineering and analytics best-practice for the business. Lay the foundations of the new data platform as a hands-on practitioner, building on the rich data sets we have in the business for both automation and analytics. Design more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. … About you We know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing … data streams into actionable insights for direct use or for onward processing. You will have a proven track record of delivery and a passion for adherence to best practices in the data engineering discipline. You will be passionate about data in all its forms more »

Job Title: Technical Lead Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £70,000 - £85,000 The Role: Lead the Data team to deliver ETL processes, build and maintain data pipelines, enhance scalability, performance, maintainability, configurability … and usability Be involved in the entire life-cycle of the Data Strategy, including idea creation, deployment, and longer term updates and enhancements Support the product and Software Development Managers in prioritisation, scoping and leading delivery of Data Strategy features in the exciting world of Future … State Architecture Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Able to zero-in on bugs without looking at the code Technical Lead Data Engineer Requirements: Good SQL and NoSQL experience Expert-level ability in TSQL more »

We are currently seeking a Data Engineer to join our client, a unique scale-up business who have received significant investment, that is entering a phase of large scale growth. This role will be a pivotal role in shaping the organisation’s data operations, as the … company grows and matures. As a Data Engineer you will design and build data pipelines that will fuel our client’s groundbreaking product lines. You'll collaborate across multiple disciplines to ensure seamless data flow, and go beyond pipelines to champion data … of what the successful candidate will require (don’t worry if you don’t tick all the boxes!): Python for building and automating data pipelines. A solid understanding of ETL processes for data extraction, transformation, and loading. Experience with relational databases ( SQL ) and database design principles more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability … Develop scalable and re-usable frameworks for ingestion and transformation of large data sets Integrate the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

JOB DESCRIPTION About The Role We are recruiting for a Senior Data Architect to join our team. Our Senior Data Architect will work in multi-disciplinary teams that build, support & maintain User-Centred digital solutions that offer real value and work for everyone. As a Senior … Data Architect , you will be working as part of teams to build solutions that make data accessible to enable solutions to be optimised by using an evidence-based approach. Engaging with our clients you will design and implement data solutions ensuring solutions are integrated … strategy Build and maintain appropriate Enterprise Architecture artefacts including Entity Relationship Models, Data Dictionary, ETL/ELT definitions and Data Pipeline/Lineage Models Be accountable to the client for the rationale of the architecture and detailed solution Define the methods of governance to ensure more »

Job Title: Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £50,000 - £70,000 The Role: Design and develop ETL processes Design, build and maintain end-to-end data pipelines Identify, root cause analysis, and fix incidents related … to data management Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Routinely provide status updates to colleagues and stakeholder groups Data Engineer Requirements: Hands-on experience with SQL database design Experience writing automated tests … for data pipelines Technical expertise with data models, data mining, and segmentation techniques Knowledge of programming languages (e.g. Python, R) Data Engineer Desirable's: Proven experience of Azure Services Knowledge of Data bricks and Jupyter notebooks Experience in data more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … implementing cloud solutions for different applications, automating cloud operations and cloud deployments Assume responsibility for the architecture of solutions that leverage advanced analytics, data pipelines, web portals, statistical models, Machine Learning and AI, through our technology platform Draw on the strength and experience of a wider team of more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

two days) at our Leeds, Edinburgh, or Bristol locations. About this opportunity Join IP&I as we embark on a transformative journey in data engineering. We are seeking a dynamic and experienced Senior Manager within Advanced Analytics to lead our team in architecting and implementing cutting-edge data solutions. If you are passionate about driving innovation and possess a proven track record in cloud and legacy data engineering leadership, we want to hear from you! You will be at the forefront of our data engineering initiatives. You'll lead a talented team … in designing, developing, and optimizing scalable data solutions. Your role will involve providing technical leadership, driving innovation, and ensuring the successful delivery of complex data projects. Here's where you'll make the difference: Lead and mentor a team of data engineers Architect and more »

also offer you a chance to work in a highly motivated and smart team of engineers? You'll work as a Mid-Level Data Engineer building a data and analytics platform that's used by the hedge fund's trading desks. You'll join at a … time where the fund are diversifying into new markets and therefore you'll play a crucial part in all aspects of data operations in this new area for them. You'll work extremely closely with Portfolio Managers/Quants - You'll work with them to understand exactly what … they need from the data and work fully end to end to create solutions that present data in a way that can be consumed effectively by them. Qualifications: Strong experience with Python for data engineering. Experience building custom Data Pipelines Good knowledge more »

Are you an experienced Senior Data Engineer ? Would you like to work for a fast growing business that specialises in Business Intelligence & Data Analytics ? Its a Hybrid working role and you will be based from either their London, Leeds, Manchester or Edinburgh office, typically 3 days … site. Preferred skills: Experience with Azure and Databricks Proficiency in SQL and/or similar data technologies Familiarity with data pipeline tools and ETL processes Knowledge of cloud platforms and data architecture Experience working in the Retail Banking sector or financial services would more »

is on an exciting journey and there is no better a time to join us. The investments we are making in our people, data, and technology are leading to innovative projects; fresh possibilities have provided new ways for our people to work, learn and thrive. About us… Like … the modern Britain we serve, we're evolving. Investing billions in our people, data and tech to transform the way we meet the ever-changing needs of our 26 million customers. We're growing with purpose. Join us on our journey and you will too… What you'll … across Product & Engineering. Proven experience in hands-on software engineering using a broad range of technologies including the following: Java or Python Microservices Data pipelines and database programming such as DBT, SQL, BigQuery, Cloud Composer etc CI/CD/DevOps tooling experience e.g., GIT, Jenkins etc What more »