1 to 25 of 28 Data Engineering Jobs in Yorkshire

Group Head of Data Engineering & Governance Are you an accomplished Head of Data Engineering & Governance looking for the next move in your career? InX are currently working on a retained Search basis with a Financial Services organisation to place a Group Head of Data in … the CIO's senior management team. The Role As the Group Head of Data Engineering & Governance, you will be responsible for leading the data engineering team in designing, implementing, and maintaining scalable data pipelines and infrastructure. You will collaborate closely with cross-functional teams to … ensure the integrity, availability, and security of the organisation's data, while also driving innovation in data engineering practices. You will work closely with the Data Science and Analytics team (which sits outside of the remit of this position) but will be responsible for the growth more »

Digital with Purpose. DWP Digital is seeking a Senior Data Engineer to apply data engineering on a massive scale, influencing issues which affect millions of citizens and making a real difference to how the UK works. DWP is the UK's largest department, every year we pay … with huge datasets (quite possibly the largest on offer in the UK), you'll be delivering benefits that positively impact thousands of UK citizens. Data modelling is key in this role - you'll be running platforms and transforming raw data into model raw datasets. Our production environment is … cloud engineering. You'll be executing code in different places across the following tech stack: Azure, Databricks, PySpark and Pandas. You will steer the data engineering function within a wider product team. There'll be lots of connecting and interaction with stakeholders - you'll work closely with product more »

Digital with Purpose. DWP Digital is seeking a Senior Data Engineer to apply data engineering on a massive scale, influencing issues which affect millions of citizens and making a real difference to how the UK works. DWP is the UK's largest department, every year we pay … with huge datasets (quite possibly the largest on offer in the UK), you'll be delivering benefits that positively impact thousands of UK citizens. Data modelling is key in this role - you'll be running platforms and transforming raw data into model raw datasets. Our production environment is … cloud engineering. You'll be executing code in different places across the following tech stack: Azure, Databricks, PySpark and Pandas. You will steer the data engineering function within a wider product team. There'll be lots of connecting and interaction with stakeholders - you'll work closely with product more »

Job description: hands-on experience in business intelligence development or data engineering experience,Solid experience with data modelling, data warehouse design and data lake concepts and practices; Exposure working in a Microsoft Azure Data Platform environment. Exposer inAzure Data Factory Azure Storage Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics • Building new data pipelines and optimizing data flows using the Azure cloud stack. • Building data products from scratch. • Support … Business Analysts and Data Architects with discovery and best practices. • Self-directed and comfortable supporting the data needs of multiple teams, systems and products. What’s required ? • Advanced Azure knowledge and experience working and migrating data products from on-prem to Azure. • Experience building and optimizing ‘big more »

firm working with one of the biggest financial services organisations in the world. The successful candidate will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like: Extracting data from various sources (databases, APIs, files) Transforming and cleaning the data Loading the data into data warehouses or data lakes (e.g., BigQuery, Amazon Redshift) Automating the data pipeline execution using scheduling tools (e.g., Airflow) Work with Big Data technologies: You'll likely work with various Big Data technologies alongside Spark, including: Hadoop Distributed File … System (HDFS) for storing large datasets Apache Kafka for real-time data streaming Apache Hive for data warehousing on top of HDFS Cloud platforms like AWS, Azure, or GCP for deploying and managing your data pipelines Key skills: At least 8+ Years of IT Experience with designing more »

strategy and supporting the mission to deliver exceptional Management (MI) products and services? Do you want to collect and manage a vast array of data which is used by the Secretary of State? DWP Digital is seeking a Senior Data Engineer to join our MI team who provide … me more about the position: Working with modern cloud environments and a mix of legacy on-premises technologies, you'll help develop the DWP Data Engineering Strategy for MI products and services that enables the department to gain insight and make more informed choices. The work is fast … paced with a large and growing portfolio of MI products built upon hundreds of data sets. You'll focus on back-end data processes to support MI reporting, rather than visualisation work. This role has an emphasis on data engineering technical leadership - you'll lead other more »

strategy and supporting the mission to deliver exceptional Management (MI) products and services? Do you want to collect and manage a vast array of data which is used by the Secretary of State? DWP Digital is seeking a Senior Data Engineer to join our MI team who provide … me more about the position: Working with modern cloud environments and a mix of legacy on-premises technologies, you'll help develop the DWP Data Engineering Strategy for MI products and services that enables the department to gain insight and make more informed choices. The work is fast … paced with a large and growing portfolio of MI products built upon hundreds of data sets. You'll focus on back-end data processes to support MI reporting, rather than visualisation work. This role has an emphasis on data engineering technical leadership - you'll lead other more »

DWP. Digital with purpose. Are you a leader in data engineer with the experience and drive to lead a team of talented Data Engineers to success? Are you passionate about building and supporting data capabilities that are essential to DWP services? Do you want to be involved … in agile, multidisciplinary teams to deliver these changes, making over services easier to find, access and use. We're looking for a talented Lead Data Engineer with the experience and passion to lead a high-performing team of data engineers working to build and support data capabilities … maintain an effective professional community. Defining, updating and implementing Career Pathways to enhance capability development, talent management and reduce attrition. Defining and implementing the Data Engineering standards and Best Practices for DWP. Understanding the demands and development of the profession in the wider industry and feed this into more »

DWP. Digital with purpose. Are you a leader in data engineer with the experience and drive to lead a team of talented Data Engineers to success? Are you passionate about building and supporting data capabilities that are essential to DWP services? Do you want to be involved … in agile, multidisciplinary teams to deliver these changes, making over services easier to find, access and use. We're looking for a talented Lead Data Engineer with the experience and passion to lead a high-performing team of data engineers working to build and support data capabilities … maintain an effective professional community. Defining, updating and implementing Career Pathways to enhance capability development, talent management and reduce attrition. Defining and implementing the Data Engineering standards and Best Practices for DWP. Understanding the demands and development of the profession in the wider industry and feed this into more »

Director of Data & Systems - SaaS - Up to £85,000 DOE + Benefits Hybrid Working - Sheffield Our Client's mission is dedicated to empowering educational institutions around the globe to achieve compliance and elevate their standards. They offer a comprehensive SaaS solution that merges top-tier professional development with cutting … edge management tools, all accessible through a single, unified platform. Are you passionate about technology and data? Do you thrive in a fast-paced, collaborative environment? If so, we have the perfect opportunity for you. We are seeking a Director of Data and Systems to lead our IT … and data initiatives, ensuring high-quality, reliable solutions that delight our users and stakeholders. Key Responsibilities: Strategic Planning : Build and implement a roadmap for IT that supports our overall technology and business strategy, driving continuous improvement. Internal Infrastructure Management : Oversee all aspects of internal IT with the support of more »

Data Engineer - Hybrid (1 day Leeds City Centre) - up to £55k + bonus & benefits Head Resourcing are recruiting on behalf of this leading financial services client who are a dynamic new company in the financial advice market who have a brilliant range of roles available across the IT space. … An exciting opportunity has arisen to join the growing Data Engineering team, based in Leeds city centre (hybrid). The role will be critical in delivering the Technology strategy, ensuring data quality, consistency, and reliability. Delivering this strategy will also provide ample development opportunities, in particular progressing … our new Data Lakehouse and implementing an iPaaS solution throughout 2024. Working with Python, SQL, AWS and PowerBI, you will build, rebuild and maintain data pipelines as well as have the opportunity to work with Terraform and shape the role and your progression Location | Leeds, West Yorkshire Salary more »

people who collaborate to get things done and have lots of fun along the way. We are committed to building scalable and re-usable data solutions to enable our clients to leverage their data for mission-critical business insights, driving decisions, and delivering value. To do this we … employ a wide range of tools and technologies, recognising that the fields of Data Engineering, Data Visualisation and Data Science are swiftly evolving. We currently have offices in Sheffield (our HQ), Leeds, Manchester & Bristol, however, a lot of us work remotely. We prefer for everyone to … in a variety of projects in the cloud (AWS & Azure). If you are on the Azure side, you will learn about and use data services such as Azure SQL, Synapse Analytics, Azure Data Factory, Azure Data Warehouse, Databricks, Bicep & PowerBI. About You: You're an engineer more »

DWP. Digital with Purpose. We have an exciting opportunity to join our community of experts at DWP Digital as a Data Lead. DWP is the UK's largest government department. We help people into work and make payments worth over £195bn a year, to support millions of daily users. … user-centric digital solutions, positively changing how government work. We design, build, and run digital public services, making them simpler and faster for everyone. Data with Potential. As a Data Lead in DWP Digital, you'll be able to influence huge change and make a lasting difference as … support the millions of people who interact with DWP every day. You'll need deep technical expertise in one or more areas such as data engineering, data governance, data science or performance analysis. You'll be the expert we come to for advice, guidance and leadership more »

DWP. Digital with Purpose. We have an exciting opportunity to join our community of experts at DWP Digital as a Data Lead. DWP is the UK's largest government department. We help people into work and make payments worth over £195bn a year, to support millions of daily users. … user-centric digital solutions, positively changing how government work. We design, build, and run digital public services, making them simpler and faster for everyone. Data with Potential. As a Data Lead in DWP Digital, you'll be able to influence huge change and make a lasting difference as … support the millions of people who interact with DWP every day. You'll need deep technical expertise in one or more areas such as data engineering, data governance, data science or performance analysis. You'll be the expert we come to for advice, guidance and leadership more »

Foundry Specialist/Architect to join our team. The ideal candidate will possess extensive knowledge and hands-on experience with Palantir Foundry, including its data integration, analytics, and operational decision-making capabilities. As a Palantir Foundry Specialist/Architect, you will be responsible for designing, developing, and maintaining data-driven solutions that leverage Palantir Foundry's platform to address complex business challenges. Key Responsibilities: - Design and architect end-to-end data solutions using Palantir Foundry, ensuring alignment with business goals and technical requirements. - Develop scalable and efficient data models, pipelines, and workflows within the Foundry environment. … Collaborate with stakeholders to understand business needs and translate them into technical requirements and solutions. - **Data Integration:** - Implement data integration solutions to ingest, transform, and manage large datasets from various sources into Palantir Foundry. - Ensure data quality, consistency, and security throughout the data lifecycle. **Data more »

converters. Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise scale … and Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages: Proficiency … in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data warehousing, data pipelines, data modeling, and data cleansing techniques more »

Financial Services firm looking for a heavy hitting Data engineer where some of your Key Responsibilities will include: Develop new pipelines based on business needs or to improve the current system. Maintain existing data pipelines, anticipating and addressing issues promptly and professionally. Monitor critical processes effectively. Collaborate with … pipelines and ETL. Essential Skills and Qualifications Technical Proficiency : Expertise in SQL (Postgres, Athena Presto SQL) and proficiency in Python for building and managing data pipelines. AWS Experience : Skilled in using serverless architecture, including Lambda, Glue, Step Functions, SNS, SQS, and S3. Data Engineering Expertise : Over … years of experience in data engineering, creating data pipelines in a cloud environment, with a solid understanding of best practices. Effective Communication : Ability to clearly convey and gather requirements from non-technical business stakeholders. Deployment and Documentation : Proficient in controlled deployment and documentation practices using GitHub, ensuring more »

The role will be critical in ensuring data quality, consistency, and reliability for SPW. The role will also provide ample development opportunities, in particular progressing our new Data Lakehouse and implementing an iPaaS solution. Location | Leeds, West Yorkshire Salary | £50,000 - £55,000 Key Responsibilities: Maintenance of existing … data pipelines; pre-empting issues and responding to any incidents in a timely, professional manner. Development of new pipelines as per business requests or to stabilise the current operating environment. Liaising with other areas across the business to help define requirements for new projects, including both pipeline and ETL … key processes. Core Skills/Competencies Languages - Expert in SQL (Postgres, Athena Presto SQL) and proficient in using Python to build, maintain and orchestrate data pipelines across the business. Amazon AWS - Experience using a serverless approach primarily utilising Lambda Functions, Glue (Jobs, Crawlers, Catalogues), Step Functions, SNS (notifications), SQS more »

Digital with Purpose. DWP Digital is hiring Data Engineers who are passionate about utilising cutting-edge technologies to deliver impactful data products. You'll help us design and develop of modern cloud-based data solutions, and develop your skills and career in a unique environment. The scale … Strategy and is radically transforming how we detect and prevent fraud and error in the welfare system. You will deliver proactive, innovative and insightful data products in a hybrid cloud environment using agile approaches. The data products you work on will tangibly improve the outcomes for millions of … citizens across the country. Working closely with stakeholders to develop and maintain robust data products and services, you'll support the continuous improvement of data engineering standards and engage with stakeholders at all stages of the data engineering lifecycle. DWP Digital is a great place more »

Digital with Purpose. DWP Digital is hiring Data Engineers who are passionate about utilising cutting-edge technologies to deliver impactful data products. You'll help us design and develop of modern cloud-based data solutions, and develop your skills and career in a unique environment. The scale … Strategy and is radically transforming how we detect and prevent fraud and error in the welfare system. You will deliver proactive, innovative and insightful data products in a hybrid cloud environment using agile approaches. The data products you work on will tangibly improve the outcomes for millions of … citizens across the country. Working closely with stakeholders to develop and maintain robust data products and services, you'll support the continuous improvement of data engineering standards and engage with stakeholders at all stages of the data engineering lifecycle. DWP Digital is a great place more »

DevOps Engineer to join a growing tech team in Leeds. The organisation has significant financial backing and is looking to invest this in the engineering and IT teams. It's one of the best projects we've seen this year! Key details: Salary: £55,000-90,000 Benefits … skills desired/what you will learn: Terraform AWS: S3, Lambda/Serverless TypeScript, NodeJS, and ReactJS Python CI/CD: CircleCI, GitHub Actions Data Engineering technologies can be taught longer-term Role overview: Above all, this firm offers one of the best benefits packages outside of London … the most modern tech stack available. In the long term, there will be opportunities for vertical and horizontal moves, such as management, lead-level, data engineering and more. Interested? Please apply with your CV and/or message Billy Hall for further details. Xpertise acts as an employment more »

Data Engineer | Yorkshire | up to £45,000 basic | Flexible Working A regular feature in the "UK's Top 10 places to work", my client is an award-winning business with a reputable brand and is widely respected for investing in their employees, placing importance on personal development and career … progression. The Environment you'll be joining: You'll join a friendly 10+ Data Engineering team as the business undergoes a complete greenfield re-architecture from the ground up in Microsoft Azure. The Tech you'll be playing with: Azure Data Factory Azure Databricks PySpark SQL DBT … What you need to bring: 1-3 year's experience in building Data Pipelines SQL experience in data warehousing Python experience would be desirable Experience in Azure/GCP/AWS is a bonus Why join? Joining an award-winning business comes with its perks. Outside of the more »

Recruitment are currently recruiting on behalf of a prestigious client in the Professional Services sector for a BI Developer to join the Transformation and Data Team which delivers digital, data and technology-based services and solutions across the business. Reporting to the Business Analysis Manager, the successful candidate … will work closely with business analysis and business intelligence colleagues, supporting IT Leadership and stakeholders on business improvement, data insights, including intelligent automation, and data driven decision making activities. BI Developer responsibilities Provide timely, accurate and actionable information and insights across the business by using a range of … tools to produce high quality business intelligence & analytics outputs. Evaluate reporting and analytics requirements and produce high quality deliverables Apply data modelling techniques to identify trends in business and customer data. Support IT Leadership and business stakeholders in data insight related projects, including enhancing business intelligence capabilities, and more »

Data Engineer Leeds, onsite 2x per week £45,000 - £55,000 This is an exciting opportunity to join a rapidly growing financial organisation, grown from a joint venture with a major bank. They are seeking an experienced Data Engineer to play a crucial role in the progression of … a new Data Lakehouse, and implementation of an iPaaS solution. As the Data Engineer, you will bring extensive commercial experience in working with sizeable ETL processes/pipelines, AWS serverless technology and Python. In addition to strong technical skills, effective communication and ability to engage with non technical … culture while having extensive financial backing to ensure their technology is the best the market has to offer! Requirements: 3+ years commercial experience within Data Engineering SQL Python AWS Excellent communication skills Package: 35 hour working week L&D budget 15% pension Hybrid working 30 days annual leave more »

Company: Amber Labs Location: Hybrid - Sheffield - 2 days a week About Us: At Amber Labs, we specialise in Data Engineering, Cloud, and Analytics Consultancy, dedicated to empowering our clients' digital transformation journeys. Our mission is to unlock the potential of our clients' data through innovative strategies that … enhance their data capabilities, analytics, and data governance. Position Overview: Need to display evidence of past work in problem management/proactive problem management, which would be beneficial. Should have prior experience in Service Management or Problem Management. Should also demonstrate proficiency in utilizing ServiceNow as a data … Sense.• Responsibilities: o Design, develop, and maintain BI solutions using ServiceNow, Power BI, and QlikSense. o Develop and deploy dashboards and reports. o Integrate data from various sources. o Collaborate with cross-functional teams. o Maintain and support BI platforms. o Develop and execute queries. o Ensure data more »