1 to 25 of 54 Data Pipeline Jobs in the North of England

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

Data Engineer £50,000 - £60,000 - Hybrid My client, industry leading EdTech organisation are currently looking to expand their Data team! become a Data Engineer with vast progression opportunities, where your work is all for a great cause! Requirements Thrive in … a fast paced and dynamic environment Exceptional SQL skills including an appreciation for performance tuning on a variety of database engines Data pipeline development experience in Python (especially using pandas, parsing API responses, posting data to API’s etc.) Excellent analytical and problem-solving skills … An appreciation for the big picture while delivering short/mid-term solutions Experience of building data warehouses, enterprise data warehouses or data lakes with ETL pipelines Experience with AWS tools including Glue, Lambda, S3, RDS, is preferred An understanding of CICD concepts including more »

Title: Principle Data Engineer Location: Hybrid - Once per Month Travel to office in North West. Salary: £75,000 + Package Role Overview: The Principal Data Engineer holds a key position within the Data Engineering function, reporting directly to the Head of Data & Platform Engineering. This role requires a deep understanding of Google Cloud Platform to ensure that all production data processes are reliable, efficient, and compliant. Key responsibilities include overseeing data ingestion, quality, integrity, modeling, warehousing, transformation, security, encryption, monitoring tools, and batch management. The Principal … engineering team. This role involves close collaboration with the Principal Cloud Engineer and the Data Architect to ensure data pipeline designs are optimized and reliable within the Google Cloud Platform. Effective documentation and communication of solutions to both technical and non-technical stakeholders are more »

Senior Data Engineer (Remote) - Stealth SaaS Startup Location: Manchester (Remote Friendly) Salary: £65,000 - £75,000 per annum About Us: We are an innovative stealth start-up set to launch a cutting-edge SaaS product next year. We’re looking for a talented Senior Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with … to bring the AI/ML models into production Databricks for daily data processing Qualifications: Experience in data engineering and pipeline construction. GCP and familiarity with other cloud environments. Understanding of SDLC. Experience with Databricks. Programming skills (Python and SQL). What We Offer more »

Test Engineer Contract Type: Inside IR35 Rate: £500 per day Duration: 6 months This is a highly back-end/data-focused testing position, so experience in this area is essential. Location: Remote with travel to Leeds or London as required Brio is working with a leading provider … Requirements:** Proven experience as an Automation Test Analyst, preferably within the healthcare or NHS sector. Strong scripting skills in SQL and Python ETL, Data Migrations, Data Pipelines Experience with test automation frameworks. AWS cloud Excellent communication and collaboration skills. Ability to work effectively in a fast more »

Senior Data Engineer | Full Time | On-Site | Python | SQL | DataBricks| Azure| As a Senior Data Engineer, you will play a key technical role in design, implementation, and continuous refinement of data pipelines and architectures within Microsoft’s Azure ecosystem. This role involves leveraging Azure … s diverse data services, such as Data Factory, Databricks, Azure SQL and Synapse Analytics, among others, to build scalable and secure data solutions that support complex onward processing by both operational and analytical teams. What will this role look like? - Develop and maintain scalable … operational and analytical teams to solve complex business problems. - Monitor system performance, identify bottlenecks, and implement changes to improve data processing and pipeline/systems performance. - Work closely with Data Scientists within the Operational Analytics team to understand data needs and deliver solutions more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

Principal GCP Data Engineer Remote Up to £85,000 VIQU are seeking Principal GCP Data Engineer to join a leading social enterprise working to transform lives by developing and managing vibrant communities. They excel in placemaking, regeneration, development, investment, property management, and leisure services. With a … numerous specialist subsidiaries, they create thriving environments and reinvest profits to enhance community projects, ensuring sustainable and flourishing communities. The successful Principal GCP Data Engineer will be responsible for ensuring robust and compliant data processes on Google Cloud Platform. The key responsibilities of the role will … include data ingestion, quality, modelling, transformation, security, and monitoring. The Principal GCP Data Engineer will also be involved in automation, process improvement, mentoring engineers, setting coding standards, and optimising queries. The Principal Data Engineer will work in collaboration with the Principal Cloud Engineer and more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability … Develop scalable and re-usable frameworks for ingestion and transformation of large data sets Integrate the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability … Develop scalable and re-usable frameworks for ingestion and transformation of large data sets Integrate the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … our impact using your knowledge, experience, and passion for your chosen field. The Data Scientist role requires expertise in machine learning, statistical analysis, anomaly detection, and strong communication and collaboration skills. Key responsibilities include developing innovative models, designing, applying, and optimising models in a dynamic environment. The role more »

Data Engineer Leeds - on site 2x per week up to £55,000 Looking to join a dynamic team in the financial sector? Fruition IT are looking for an experienced Data Engineer to join a leading company specializing in financial solutions. Working closely with the Senior Data … whilst maintaining a core SAS user base. Requirements: Demonstratable proficiency with SAS platforms Strong commercial experience with SQL Hands on experience with Azure Data Factory and the build of data pipelines Proficiency in Python for data manipulation, transformation and analysis Experience with Azure Synapse more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … implementing cloud solutions for different applications, automating cloud operations and cloud deployments Assume responsibility for the architecture of solutions that leverage advanced analytics, data pipelines, web portals, statistical models, Machine Learning and AI, through our technology platform Draw on the strength and experience of a wider team of more »

Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise … Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages … Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS,... more »

Kubernetes) and Workflow orchestration (Temporal) .Strong skills in distributed architecture – local and cross site resiliency and failover .Achieving scalability on Apache Nifi based data pipelines to run several concurrent data movement feeds .Being able to visualize and integrate the platform and application architecture .Own and manage more »

Data Engineer - Manchester - £35,000 - Hybrid Working My client is hiring a full time Data Engineer to contribute to their expanding data science and analytics initiatives. Your role involves designing, building, and maintaining data pipelines, ensuring seamless data flow into … their central Microsoft Azure repository. Collaborate with Data Analysts and Scientists to optimise data quality, reliability, security, and automation. Skills & Responsibilities: Configure and troubleshoot Microsoft Azure, manage data ingestion Develop ETL scripts using Python, handle web scraping, APIs, and relational databases Familiarity with Power … will not have to use Power BI, but being aware of how to collaborate with people that do is important) Work closely with data analysts and scientists, contribute to data models and algorithms, and participate in data events Working knowledge of Git If you more »

Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise … Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages … Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data warehousing, data pipelines, data modeling more »

Data Engineer Consultant - Data & AI (DV Cleared/Eligible)65,000 – 75,000 + packageLocation: United Kingdom (Greater Manchester – Manchester, Greater Manchester) Type: PermanentData Engineer Consultant - Data & AI (DV Cleared/Eligible)Location: ManchesterSalary: 65k - 75kAbout the Role:Our client, a leading consultancy in … the Defence & Security sector, is seeking a skilled Data Engineer to join their AI & Data Engineering team. This pivotal role offers the opportunity to be at the forefront of advanced technology projects aimed at enhancing national security. The position is perfect for individuals who are passionate … data security and quality across projects.Qualifications and Skills:Proven experience in data engineering, particularly with ETL processes, data pipeline construction, and workflow orchestration.Familiarity with distributed computing techniques, such as parallel processing and batch data management.Strong foundation in data system more »

Data Engineer We are seeking a Data Engineer to join our friendly team. Our client is building a new data platform centred around a Microsoft SQL server-based data warehouse that runs on AWS RDS. This role is crucial in expanding our … client's data capabilities and supporting the launch of our new platform and mobile app. Key Responsibilities: Support and maintain the central Data Platform, ensuring its availability and performance to meet business needs. Build data pipelines to connect APIs, files, databases, and data streams to our client's Data Platforms. Shape data from various sources into a transactional ODS for detailed reporting and a Kimball-modelled Data Warehouse for analytics and dashboards. Define and embed data quality measures within data pipelines. more »

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on business requirements and priorities As part of … Coding experience with Python/Javascript/C# Familiarity with statistical/machine learning/AI concepts and techniques Understanding of data pipeline/orchestration tools e.g. dbt, dataform Appreciation of GCP’s serverless technologies e.g. Cloud Run/Workflows Understanding of Google’s marketing stack, Google more »

experienced AWS Cloud Architect to design AWS reference architecture for our services-led client. Responsibilities: Design and implement scalable, secure, and high-performance data architectures on AWS. Develop data models and database designs that align with business requirements and ensure data integrity. Proficiency in … or AWS CodePipeline. Hands-on experience with Infrastructure as Code (IaC) tools like AWS CloudFormation, Terraform, or similar. Create and maintain ETL processes, data pipelines, and data integration workflows using AWS services. Implement data storage solutions, including data lakes, data warehouses, and operational databases. Provide technical leadership and guidance on AWS data services and architecture to the team. Requirements: Proven experience as an AWS Data Architect or similar role, with a strong focus on data architecture and management. In-depth knowledge of more »