51 to 75 of 336 Data Pipeline Jobs in England

Senior Data Engineer Location: London, UK Duration: 9 months Role/Position Overview As a Senior Data Engineer, you will lead the team’s thinking when it comes to building, deploying, and hosting data engineering pipelines and solutions in production. You will be responsible … for designing and implementing scalable data pipelines, managing data infrastructure, and tooling, and working closely with stakeholders to deliver quality solutions. This role will be a key contributor to the ongoing development of our data architecture and data governance capabilities. We want … to minimize manual processes and strong software engineering proficiency in this role is key. Key Responsibilities: - Design and implement scalable data pipelines that extract, transform and load data from various sources into the data lakehouse. - Help teams push the boundaries of analytical insights, creating more »

promises exciting, engaging and rewarding projects for those that are keen to develop and build a successful career. The Role The Data Pipeline Engineers ensure that our customer work with updated and correct data in our data analytics tool. You are the first … schedules so that pipelines run effectively. up and maintaining health checks on different data pipelines Triage, and debug the data pipeline when broken. code and writing code changes and/or modifying the monitoring set-up and understanding how to navigate the data … providers when data is incorrect or not received on time. outages with the end users of a data pipeline What We Value reading and writing code in Python, Pyspark and Java. understanding of Spark and interested in learning the basics of tuning Spark more »

Job Title: Junior Data Engineer Location : London (Anywhere in UK) Pay Rate : £150/Day We are looking to hire a Junior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures on AWS. You will work closely … with senior data engineers … data scientists, and other stakeholders to ensure data is accessible, reliable, and ready for analysis. Key Responsibilities: Data Pipeline Development : Design, build, and maintain data pipelines to ingest, transform, and load data from various sources into AWS data more »

Introduction Axtria is a global provider of cloud software and data analytics to the Life Sciences industry. We help Life Sciences companies transform the product commercialization journey to drive sales growth and improve healthcare outcomes for patients. We are acutely aware that our work impacts millions of patients … Artificial Intelligence and Machine Learning. Our cloud-based platforms - Axtria DataMax™, Axtria InsightsMax™, Axtria SalesIQ™, and Axtria MarketingIQ™ - enable customers to efficiently manage data, leverage data science to deliver insights for sales and marketing planning and manage end-to-end commercial operations. With customers in over … to be the leaders we produce for the industry. Will you be next? Job Description: We are seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have 5-7 years of recent experience in data engineering and hold a more »

Position: Data Engineer Reporting to: Lead Data Engineer Contract type: Full Time, Permanent, Remote (with Occasional Travel) This is an exciting opportunity to join one of Europe's fastest growing healthtech startups About Cera At Cera Care, our mission is to empower people to live longer … the global leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. . About Data at Cera Data has a critical role to … operational workforce, from caregiver recruitment to care delivery to our client base, and provides insights for performance improvement and cost-efficiency. Health-related data offers a comprehensive financial overview that could lead to cost savings and crucially lead to health deterioration prevention. The primary mission of our Data more »

Data Engineer - £95,000 - DV Clearance Required We are currently looking for a passionate and experienced Data Engineer to come and join one of our biggest clients - A consultancy that specialises in FS, Public Sector, Media, Software and Data Warehousing. They are looking for … available across the UK so no matter where you are from, we welcome your applications!!! The Role: We are looking for a savvy Data Engineer to … join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The Data Engineer will support our software more »

Title: Principle Data Engineer Location: Hybrid - Once per Month Travel to office in North West. Salary: £75,000 + Package Role Overview: The Principal Data Engineer holds a key position within the Data Engineering function, reporting directly to the Head of Data & Platform Engineering. This role requires a deep understanding of Google Cloud Platform to ensure that all production data processes are reliable, efficient, and compliant. Key responsibilities include overseeing data ingestion, quality, integrity, modeling, warehousing, transformation, security, encryption, monitoring tools, and batch management. The Principal … engineering team. This role involves close collaboration with the Principal Cloud Engineer and the Data Architect to ensure data pipeline designs are optimized and reliable within the Google Cloud Platform. Effective documentation and communication of solutions to both technical and non-technical stakeholders are more »

Job Title: Azure Data Engineer (Contract) Duration: 3 months Rate: £400 per day Location: Remote Skills: Databricks/Azure Data Lake/Kubernetes IR35 Determination: Outside IR35 Company Overview: We are seeking a skilled and experienced Azure Data Engineer to join our team on … a contract basis. This is an exciting opportunity to contribute to our data infrastructure and play a key role in ensuring the reliability, efficiency, and performance of our data solutions on the Azure platform. Responsibilities: Design, develop, and implement data solutions using Azure services … including Azure Databricks, Azure Data Lakes, and other cloud-native technologies. Work closely with cross-functional teams to understand data requirements and translate them into technical specifications. Build and maintain data pipelines, ETL processes, and data warehouses on Azure, leveraging technologies such more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data more »

inside IR35 Our client is currently working on several end-client programmes, for several Government technical Consultancies to support various public body Government Data Engineering departments as part of their respective cloud migration strategies. The Data Platform Engineer will be assigned to one (perhaps more depending … on project timelines) of our client's data engineering projects based on data engineering and technical skills. The successful candidate will fulfil the following responsibilities: Requirements analysis and design of solutions for the data platform; Integration and analysis of solutions; Understanding of toolsets including … AWS hosted databases (Postgres, S3 based data lake, AWS glue MySQL, Athena); Integration services (AWS API gateway, lambda functions), nice to have MuleSoft or Axway; ETL services (AWS Glue, Step functions); Infrastructure a management tool such as Terraform and extensive use of SQLs, Python, and Node JS. The more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … databases, as well as NoSQL databases Familiarity with cloud services (preferably GCP ) and understanding of its ETL frameworks. Desired : experience with data pipeline and workflow management (Airflow, Composer). Desired : Familiarity with machine learning algorithms and data science principles is a plus. Qualifications Degree in more »

Data Engineer £50,000 - £60,000 - Hybrid My client, industry leading EdTech organisation are currently looking to expand their Data team! become a Data Engineer with vast progression opportunities, where your work is all for a great cause! Requirements Thrive in … a fast paced and dynamic environment Exceptional SQL skills including an appreciation for performance tuning on a variety of database engines Data pipeline development experience in Python (especially using pandas, parsing API responses, posting data to API’s etc.) Excellent analytical and problem-solving skills … An appreciation for the big picture while delivering short/mid-term solutions Experience of building data warehouses, enterprise data warehouses or data lakes with ETL pipelines Experience with AWS tools including Glue, Lambda, S3, RDS, is preferred An understanding of CICD concepts including more »

record in the development and management of an Azure cloud-based data & analytics platform. Comprehensive experience in the complete data pipeline development process, encompassing data warehousing, data analytics, and ETL processes. Profound understanding of best practices in data governance … in communication, leadership, and team management. Capability to handle multiple tasks, meet stringent deadlines, and deliver top-notch solutions. Solid grasp of contemporary data architecture concepts and experience in data modelling. Familiarity with the costs associated with data management, including CI/CD driven … of business rule modifications on data. Robust problem-solving, decision-making, and analytical abilities. Self-motivated with a commitment to continual enhancement of data platforms and processes. Previous experience in the Banking and Financial Services sector is advantageous. Deep knowledge or experience with using as much of the more »

Principal Data Engineer £80,000 - £103,000 + 15% Bonus London based - Hybrid working A consultancy at the heart of ground-breaking innovation and transformative solutions where expertise meets ambition. With a focus on collaborative brilliance shaping industries and propelling client towards their full potential whilst driving meaningful … change is seeking to add a Principal Data Engineer to the team based in London. Requirements of a Principal Data Engineer; Adeptness in utilising AWS and Azure data and analytics services (or their open-source equivalents), including but not limited to EMR, Glue, RedShift … Kinesis, Lambda, Data Factory, and Databricks. Strong experience in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Proficiency in the design and deployment of production data pipelines, involving languages like Java, Python, Scala, Spark, and more »

Data Engineer Grade: Senior Consultant – Senior Manager Location: London Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based data solutions - transforming raw data … verticals: Consumer & Manufacturing, Defence & Security, Energy & Utilities, Financial Services, Government & Public Services, Healthcare, Life Sciences, and Transport. Essential Skills & Experience: • Design and deploy data pipelines using Java, Python, Scala, Spark, and SQL in big data architecture. • Execute tasks involving scripting, API data extraction, and … SQL queries. • Proficient in data cleaning, wrangling, visualization, and reporting. • Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and more »

knowledge on which to develop their careers and progress in the direction they choose. The Role: We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a … fantastic team of technology experts. You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. … Self motivated, enthusiastic and committed The ideal candidates will share our values and be aligned with our ways of working Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design and build data models, schemas to support business requirements Develop and maintain data more »

Data Engineer Paddington, London (Hybrid) Overview Data and Analytics is foundational to our Petcare OGSM and will drive our transformation to a business that is powered by data. To deliver on this ambition set by this OGSM we require the very highest level of technical/… engineering expertise within Global Petcare Data & Analytics. There is a need to expand this high performing team of creative, skilled individuals – to help build out new capabilities and bring fresh ideas to the table. Key Responsibilities Engineer and orchestrate data flows & pipelines in a cloud environment … in a cloud environment (Azure, AWS, GCP) Experience in at least one of: Python (or similar), SQL, PySpark Experience in building data pipeline/ETL/ELT solutions Ability and strong desire to research and learn new technologies and languages Interest in the contemporary data more »

Job title: Lead Data Engineer Client: Elite Fintech Salary: Up to £160,000 + Bonus Location: London/Hybrid My client are seeking a Lead Data Engineer to own delivery of scalable data pipelines that are critical to the entire firms infrastructure. Being a … Lead Data Engineer at a truly elite hedge fund, you will join an environment that prides themselves on working with the most cutting edge technologies and boast a world class technical team where you will work with and learn from the best. Responsibilities: As a Lead Data … based on very specifically tailored data requirements. Architect and implement data ingestion, transformation, and storage processes. Optimise data pipeline performance. Keep up with market developments in the data engineering and AI space and implement these best practices. Requirements and Skills: Bachelor more »

and Oaktree Capital. Find out more about us by visiting our website at: http://www.hartreepartners.com/JOB OVERVIEW: As a Senior Data Scientist at Hartree, you will play a pivotal role in shaping our data strategy and driving the development of advanced machine learning … time series forecasting solutions, specifically focused on the European Power Markets. You will collaborate with cross-functional teams to design, implement, and optimize data pipelines and predictive models that inform trading decisions and enhance operational efficiency. JOB RESPONSIBILITIES: Market Data Analysis: Conduct comprehensive analysis of data … Python and its data science libraries (e.g., Pandas, NumPy, Scikit-Learn, TensorFlow, PyTorch). Solid understanding of ML and data pipeline architectures and best practices. Experience with big data technologies and distributed computing (e.g., Spark, Hadoop) is a plus. Proficient in SQL and more »

Runcorn, WA7 3DJ Salary: £46,000 - £49,000 and extensive benefits package Hours: 37.5 hours per week Are you a seasoned expert in data engineering looking to make a significant impact in the healthcare industry? Phoenix Medical Supplies is seeking a Senior DataOps Engineer to lead our data … services across the region. As a Senior DataOps Engineer, you will be instrumental in architecting and implementing solutions across the entire data pipeline, from ingestion to visualisation. The role will require you to showcase your skills in data ingestion, storage, modelling, transfer, hosting, visualisation, analysis … and consumption, all while mentoring and training team colleagues. Key Responsibilities: Design and develop robust data pipelines for ingestion, storage, processing, and interface creation. Produce both machine consumable interfaces and human consumable data products, including standard and ad hoc reports. Maintain and optimise data more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal … teams around the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand more »

Senior Data Engineer (GCP) My client a financial services business are looking for a Senior Data Engineer (GCP), to spearhead global initiatives to migrate complex data systems to GCP. You will design, implement, and optimise data pipelines, ensuring data integrity … during migration and utilising GCP services for scalable and efficient data transformation. Salary: Up to £65K Hybrid: 2 days a week Key Responsibilities: Lead data engineering projects in Europe, working with partners for GCP transition. Design and deploy data pipelines with automated data … GCP. Essential Qualifications: 5+ years of SQL. 3+ years of GCP and its different services. Experience with microservice architecture and data processing pipeline design. Desired Qualifications: Certification in GCP. Financial Services experience. Experience coaching and mentoring data engineers. If this role is of interest, please more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

Data Engineer – Azure A fast growing and innovative company are looking for a Data Engineer to join their team. They are a people focused company, who since 2021 have won 3 best place to work awards, representing a fantastic working environment. Based in either Derby or … the Chief Operating Officer, the candidate will play a crucial role in driving the company's digital initiatives forward. Key Responsibilities As a Data Engineer, you will play a pivotal role in supporting the delivery and evolution of the company's digital portfolio. Collaborating with a talented team … the candidate will oversee the management and maintenance of existing data pipelines, monitor workloads, conduct root cause analysis, ensure data quality, and escalate technical issues as necessary. What you’ll be getting up to: Monitoring and recovery of Azure Data Factory and Databricks jobs more »

Senior Data Engineer (Remote) - Stealth SaaS Startup Location: Manchester (Remote Friendly) Salary: £65,000 - £75,000 per annum About Us: We are an innovative stealth start-up set to launch a cutting-edge SaaS product next year. We’re looking for a talented Senior Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with … to bring the AI/ML models into production Databricks for daily data processing Qualifications: Experience in data engineering and pipeline construction. GCP and familiarity with other cloud environments. Understanding of SDLC. Experience with Databricks. Programming skills (Python and SQL). What We Offer more »