1 to 20 of 20 Data Engineering Jobs in West Yorkshire

Group Head of Data Engineering & Governance Are you an accomplished Head of Data Engineering & Governance looking for the next move in your career? InX are currently working on a retained Search basis with a Financial Services organisation to place a Group Head of Data in … the CIO's senior management team. The Role As the Group Head of Data Engineering & Governance, you will be responsible for leading the data engineering team in designing, implementing, and maintaining scalable data pipelines and infrastructure. You will collaborate closely with cross-functional teams to … ensure the integrity, availability, and security of the organisation's data, while also driving innovation in data engineering practices. You will work closely with the Data Science and Analytics team (which sits outside of the remit of this position) but will be responsible for the growth more »

Digital with Purpose. DWP Digital is seeking a Senior Data Engineer to apply data engineering on a massive scale, influencing issues which affect millions of citizens and making a real difference to how the UK works. DWP is the UK's largest department, every year we pay … with huge datasets (quite possibly the largest on offer in the UK), you'll be delivering benefits that positively impact thousands of UK citizens. Data modelling is key in this role - you'll be running platforms and transforming raw data into model raw datasets. Our production environment is … cloud engineering. You'll be executing code in different places across the following tech stack: Azure, Databricks, PySpark and Pandas. You will steer the data engineering function within a wider product team. There'll be lots of connecting and interaction with stakeholders - you'll work closely with product more »

Job description: hands-on experience in business intelligence development or data engineering experience,Solid experience with data modelling, data warehouse design and data lake concepts and practices; Exposure working in a Microsoft Azure Data Platform environment. Exposer inAzure Data Factory Azure Storage Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics • Building new data pipelines and optimizing data flows using the Azure cloud stack. • Building data products from scratch. • Support … Business Analysts and Data Architects with discovery and best practices. • Self-directed and comfortable supporting the data needs of multiple teams, systems and products. What’s required ? • Advanced Azure knowledge and experience working and migrating data products from on-prem to Azure. • Experience building and optimizing ‘big more »

firm working with one of the biggest financial services organisations in the world. The successful candidate will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like: Extracting data from various sources (databases, APIs, files) Transforming and cleaning the data Loading the data into data warehouses or data lakes (e.g., BigQuery, Amazon Redshift) Automating the data pipeline execution using scheduling tools (e.g., Airflow) Work with Big Data technologies: You'll likely work with various Big Data technologies alongside Spark, including: Hadoop Distributed File … System (HDFS) for storing large datasets Apache Kafka for real-time data streaming Apache Hive for data warehousing on top of HDFS Cloud platforms like AWS, Azure, or GCP for deploying and managing your data pipelines Key skills: At least 8+ Years of IT Experience with designing more »

strategy and supporting the mission to deliver exceptional Management (MI) products and services? Do you want to collect and manage a vast array of data which is used by the Secretary of State? DWP Digital is seeking a Senior Data Engineer to join our MI team who provide … me more about the position: Working with modern cloud environments and a mix of legacy on-premises technologies, you'll help develop the DWP Data Engineering Strategy for MI products and services that enables the department to gain insight and make more informed choices. The work is fast … paced with a large and growing portfolio of MI products built upon hundreds of data sets. You'll focus on back-end data processes to support MI reporting, rather than visualisation work. This role has an emphasis on data engineering technical leadership - you'll lead other more »

DWP. Digital with purpose. Are you a leader in data engineer with the experience and drive to lead a team of talented Data Engineers to success? Are you passionate about building and supporting data capabilities that are essential to DWP services? Do you want to be involved … in agile, multidisciplinary teams to deliver these changes, making over services easier to find, access and use. We're looking for a talented Lead Data Engineer with the experience and passion to lead a high-performing team of data engineers working to build and support data capabilities … maintain an effective professional community. Defining, updating and implementing Career Pathways to enhance capability development, talent management and reduce attrition. Defining and implementing the Data Engineering standards and Best Practices for DWP. Understanding the demands and development of the profession in the wider industry and feed this into more »

Data Engineer - Hybrid (1 day Leeds City Centre) - up to £55k + bonus & benefits Head Resourcing are recruiting on behalf of this leading financial services client who are a dynamic new company in the financial advice market who have a brilliant range of roles available across the IT space. … An exciting opportunity has arisen to join the growing Data Engineering team, based in Leeds city centre (hybrid). The role will be critical in delivering the Technology strategy, ensuring data quality, consistency, and reliability. Delivering this strategy will also provide ample development opportunities, in particular progressing … our new Data Lakehouse and implementing an iPaaS solution throughout 2024. Working with Python, SQL, AWS and PowerBI, you will build, rebuild and maintain data pipelines as well as have the opportunity to work with Terraform and shape the role and your progression Location | Leeds, West Yorkshire Salary more »

people who collaborate to get things done and have lots of fun along the way. We are committed to building scalable and re-usable data solutions to enable our clients to leverage their data for mission-critical business insights, driving decisions, and delivering value. To do this we … employ a wide range of tools and technologies, recognising that the fields of Data Engineering, Data Visualisation and Data Science are swiftly evolving. We currently have offices in Sheffield (our HQ), Leeds, Manchester & Bristol, however, a lot of us work remotely. We prefer for everyone to … in a variety of projects in the cloud (AWS & Azure). If you are on the Azure side, you will learn about and use data services such as Azure SQL, Synapse Analytics, Azure Data Factory, Azure Data Warehouse, Databricks, Bicep & PowerBI. About You: You're an engineer more »

DWP. Digital with Purpose. We have an exciting opportunity to join our community of experts at DWP Digital as a Data Lead. DWP is the UK's largest government department. We help people into work and make payments worth over £195bn a year, to support millions of daily users. … user-centric digital solutions, positively changing how government work. We design, build, and run digital public services, making them simpler and faster for everyone. Data with Potential. As a Data Lead in DWP Digital, you'll be able to influence huge change and make a lasting difference as … support the millions of people who interact with DWP every day. You'll need deep technical expertise in one or more areas such as data engineering, data governance, data science or performance analysis. You'll be the expert we come to for advice, guidance and leadership more »

Foundry Specialist/Architect to join our team. The ideal candidate will possess extensive knowledge and hands-on experience with Palantir Foundry, including its data integration, analytics, and operational decision-making capabilities. As a Palantir Foundry Specialist/Architect, you will be responsible for designing, developing, and maintaining data-driven solutions that leverage Palantir Foundry's platform to address complex business challenges. Key Responsibilities: - Design and architect end-to-end data solutions using Palantir Foundry, ensuring alignment with business goals and technical requirements. - Develop scalable and efficient data models, pipelines, and workflows within the Foundry environment. … Collaborate with stakeholders to understand business needs and translate them into technical requirements and solutions. - **Data Integration:** - Implement data integration solutions to ingest, transform, and manage large datasets from various sources into Palantir Foundry. - Ensure data quality, consistency, and security throughout the data lifecycle. **Data more »

converters. Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise scale … and Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages: Proficiency … in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data warehousing, data pipelines, data modeling, and data cleansing techniques more »

Financial Services firm looking for a heavy hitting Data engineer where some of your Key Responsibilities will include: Develop new pipelines based on business needs or to improve the current system. Maintain existing data pipelines, anticipating and addressing issues promptly and professionally. Monitor critical processes effectively. Collaborate with … pipelines and ETL. Essential Skills and Qualifications Technical Proficiency : Expertise in SQL (Postgres, Athena Presto SQL) and proficiency in Python for building and managing data pipelines. AWS Experience : Skilled in using serverless architecture, including Lambda, Glue, Step Functions, SNS, SQS, and S3. Data Engineering Expertise : Over … years of experience in data engineering, creating data pipelines in a cloud environment, with a solid understanding of best practices. Effective Communication : Ability to clearly convey and gather requirements from non-technical business stakeholders. Deployment and Documentation : Proficient in controlled deployment and documentation practices using GitHub, ensuring more »

The role will be critical in ensuring data quality, consistency, and reliability for SPW. The role will also provide ample development opportunities, in particular progressing our new Data Lakehouse and implementing an iPaaS solution. Location | Leeds, West Yorkshire Salary | £50,000 - £55,000 Key Responsibilities: Maintenance of existing … data pipelines; pre-empting issues and responding to any incidents in a timely, professional manner. Development of new pipelines as per business requests or to stabilise the current operating environment. Liaising with other areas across the business to help define requirements for new projects, including both pipeline and ETL … key processes. Core Skills/Competencies Languages - Expert in SQL (Postgres, Athena Presto SQL) and proficient in using Python to build, maintain and orchestrate data pipelines across the business. Amazon AWS - Experience using a serverless approach primarily utilising Lambda Functions, Glue (Jobs, Crawlers, Catalogues), Step Functions, SNS (notifications), SQS more »

Digital with Purpose. DWP Digital is hiring Data Engineers who are passionate about utilising cutting-edge technologies to deliver impactful data products. You'll help us design and develop of modern cloud-based data solutions, and develop your skills and career in a unique environment. The scale … Strategy and is radically transforming how we detect and prevent fraud and error in the welfare system. You will deliver proactive, innovative and insightful data products in a hybrid cloud environment using agile approaches. The data products you work on will tangibly improve the outcomes for millions of … citizens across the country. Working closely with stakeholders to develop and maintain robust data products and services, you'll support the continuous improvement of data engineering standards and engage with stakeholders at all stages of the data engineering lifecycle. DWP Digital is a great place more »

DevOps Engineer to join a growing tech team in Leeds. The organisation has significant financial backing and is looking to invest this in the engineering and IT teams. It's one of the best projects we've seen this year! Key details: Salary: £55,000-90,000 Benefits … skills desired/what you will learn: Terraform AWS: S3, Lambda/Serverless TypeScript, NodeJS, and ReactJS Python CI/CD: CircleCI, GitHub Actions Data Engineering technologies can be taught longer-term Role overview: Above all, this firm offers one of the best benefits packages outside of London … the most modern tech stack available. In the long term, there will be opportunities for vertical and horizontal moves, such as management, lead-level, data engineering and more. Interested? Please apply with your CV and/or message Billy Hall for further details. Xpertise acts as an employment more »

Recruitment are currently recruiting on behalf of a prestigious client in the Professional Services sector for a BI Developer to join the Transformation and Data Team which delivers digital, data and technology-based services and solutions across the business. Reporting to the Business Analysis Manager, the successful candidate … will work closely with business analysis and business intelligence colleagues, supporting IT Leadership and stakeholders on business improvement, data insights, including intelligent automation, and data driven decision making activities. BI Developer responsibilities Provide timely, accurate and actionable information and insights across the business by using a range of … tools to produce high quality business intelligence & analytics outputs. Evaluate reporting and analytics requirements and produce high quality deliverables Apply data modelling techniques to identify trends in business and customer data. Support IT Leadership and business stakeholders in data insight related projects, including enhancing business intelligence capabilities, and more »

Data Engineer Leeds, onsite 2x per week £45,000 - £55,000 This is an exciting opportunity to join a rapidly growing financial organisation, grown from a joint venture with a major bank. They are seeking an experienced Data Engineer to play a crucial role in the progression of … a new Data Lakehouse, and implementation of an iPaaS solution. As the Data Engineer, you will bring extensive commercial experience in working with sizeable ETL processes/pipelines, AWS serverless technology and Python. In addition to strong technical skills, effective communication and ability to engage with non technical … culture while having extensive financial backing to ensure their technology is the best the market has to offer! Requirements: 3+ years commercial experience within Data Engineering SQL Python AWS Excellent communication skills Package: 35 hour working week L&D budget 15% pension Hybrid working 30 days annual leave more »

Digital with Purpose. Are you ready to join a dynamic team at the forefront of digital transformation? DWP Digital is seeking Data Engineers who are passionate about utilising cutting-edge technologies to deliver impactful data products. DWP is the UK's largest department, every year we pay over … combination of microservices that make up a single customer journey. Our production environment is currently on-prem, but the direction of travel is cloud engineering, and you'll be executing code across the following tech stack: Azure, Databricks, PySpark and Pandas. DWP Digital is a great place to work … where you can grow your career and make a real difference. Essential criteria: Commercial experience of Databricks, PySpark and Pandas Commercial experience of Azure data engineering tools such as Azure Data Factory, dedicated SQL pools and ADSL Gen 2 Experience of working with data lakes An more »

Stride Recruitment is working with a leading data consultancy focused on delivering Data & Analytics Solutions for clients in markets such as; Financial Services, Retail, CPG, and Telco. They specialise in the delivery of Modern Data Solutions for firms in their chosen target markets. Using innovative technologies, they … deliver data pipelines, cloud data architectures, advanced analytical and AI/ML solutions. We are looking for ambitious and self-motivated sales professional with a passion for helping customers achieve more with Data, Analytics and AI. You'll have experience in new business sales, ideally in a … Skills: Demonstrated ability in negotiating high-value contracts with a track record of successful deal closures. Analytical Mindset: Strong analytical skills to understand complex data environments and articulate the value proposition of our solutions effectively. Technological Proficiency: Familiarity with the latest trends in data analytics, machine learning, and more »

SQL Server SSIS 2019 .. latest versions, ADF , Azure Data Engineering Excellent Written & verbal communication Stakeholder Management (Preparing and publishing status report , Team Management) Azure Cloud Experience Design Experience and review of technical artefacts. more »