76 to 100 of 200 Permanent Data Pipeline Jobs

expertise, applies company policies and procedures to resolve a variety of issues. Works on problems of moderate scope where analysis of situations or data requires a review of a variety of factors. Exercises judgment within defined procedures and practices to determine appropriate action. Builds productive internal/external … Proven experience creating automation frameworks and tools to support test automation Experience working in a Microservices based architecture in a cloud native environment Data & Analytics 2 years of experience in integrating, transforming and consolidating data from various structured and unstructured data systems for building … with relational and NoSQL databases such as Cosmos, Cassandra 2 years of Experience in SQL, Java or Python programming Experience in Data Pipeline and integration workflow management tools: Talend, Store Proc, Change Data Capture (CDC), Spark & Azure API Experience identifying operational issues and recommending fixes more »

Data Engineer - London/Hybrid - £70,000 FTC I am currently working with a well-established organisation within the education sector who are looking for a Data Engineer to join their established IT team. Their current project is based around the creation and implementation of a … brand new Azure Data Platform. You will take responsibility for the design, development and maintenance of the data platform utilising your experience with the Azure tech stack. This is an opportunity to join a team who are serious about the way that they use data and work within an organisation who are providing significant investment into all of their data teams. The teams itself are growing in size but the company are also investing in the technology that they use with an emphasis on Microsoft tech such as Azure Synapse, Databricks more »

financial backbone of the nation. Inspire People is working with the Bank of England to seek an experienced Solution Architect with expertise in data analytics and cloud-based solutions (Azure) to play a key role in the development and enhancement of the architecture strategy and roadmaps for data … financial system, to collaborate with industry experts and thought leaders and contribute to the shaping of national financial policies and practices through effective data analytics. Key Responsibilities: * Lead pivotal programmes and projects, crafting data solutions that align with business and technology strategies. * Translate business goals into … enterprise-wide technology and data strategies. * Design and produce architectural artefacts for data platforms, migration strategies, and analytics solutions. * Drive the development and enhancement of data and analytics architecture strategy. * Guide the implementation of data governance standards and best practices. * Represent and more »

Lead Data Engineer (Java/Python/AWS/Glue/Dremio). Investment Management. £160,000 - £170,000 25% Bonus Target and Benefits. Hybrid 2 Days a week in St Pauls Office. CANNOT SPONSOR CANNOT SPONSOR My client is a top tier multi asset investment management house … based in Central London. The Data Office team is playing a key role in helping build the future of financial services, working together with business partners to create client experiences that are changing the way people invest. You will work with smart, talented people across our business. We … platform s architectural design Development and testing of the integration, modelling, data persistence and querying tools and analytical systems Data pipeline implementation, maintenance and testing Metadata management processes and tools Monitoring the overall performance and stability of the pipelines Implementing data curation, metadata more »

We are seeking a passionate and driven Junior SQL Data Engineer to join our team. As a Junior SQL Data Engineer, you will work closely with our data engineering and analytics teams to develop, optimize, and maintain SQL databases and data pipelines. … paced, dynamic environment . REQUIRED - 5 years in the UK as a minimum Role and responsibilities: Design, develop and optimize SQL databases and data models to support business requirements. Build and maintain data pipelines for efficient data extraction, transformation, and loading (ETL) processes. Collaborate … with cross-functional teams to understand data requirements and implement solutions that meet business needs. Perform data analysis and troubleshoot issues to ensure data accuracy and integrity. Support Legacy Data Warehouse: Bug fixes, batch ETL processing and data related queries. more »

Group Head of Data Engineering & Governance Are you an accomplished Head of Data Engineering & Governance looking for the next move in your career? InX are currently working on a retained Search basis with a Financial Services organisation to place a Group Head of Data in the CIO's senior management team. The Role As the Group Head of Data Engineering & Governance, you will be responsible for leading the data engineering team in designing, implementing, and maintaining scalable data pipelines and infrastructure. You will collaborate closely with … cross-functional teams to ensure the integrity, availability, and security of the organisation's data, while also driving innovation in data engineering practices. You will work closely with the Data Science and Analytics team (which sits outside of the remit of this position) but more »

Data Engineer - Manchester - £35,000 - Hybrid Working My client is hiring a full time Data Engineer to contribute to their expanding data science and analytics initiatives. Your role involves designing, building, and maintaining data pipelines, ensuring seamless data flow into … their central Microsoft Azure repository. Collaborate with Data Analysts and Scientists to optimise data quality, reliability, security, and automation. Skills & Responsibilities: Configure and troubleshoot Microsoft Azure, manage data ingestion Develop ETL scripts using Python, handle web scraping, APIs, and relational databases Familiarity with Power … will not have to use Power BI, but being aware of how to collaborate with people that do is important) Work closely with data analysts and scientists, contribute to data models and algorithms, and participate in data events Working knowledge of Git If you more »

promotion that seen the Head of Engineering promoted into the CTO role there s an opportunity to join the business as the Lead Data Engineer. Working hand in hand with the CTO you will play an instrumental part in shaping and leading the Data Engineering function … used on those days. This is a rare opportunity to work with a well liked and respected CTO and a great business. Lead Data Engineer £80k-£90k Hybrid working 2 days in Glasgow office a week Your New … Role: Platform Development: Build the foundations of the new data platform, leveraging extensive business data for both automation and analytics. Pipeline Design: Create and develop data processing pipelines to support the ingestion, enrichment, and quality control of client and partner data. Data more »

firm and throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Do As a Data Architect at BCG, you will be pivotal in designing and developing cloud-based data systems for our clients. Your responsibilities include consulting … logical architecture solution roadmaps, integrating these solutions with client business processes and technology frameworks. Your role involves developing innovative solution sets and enhancing data platform capabilities to address emerging data challenges, leveraging the best of approved Cloud platforms. You'll also guide clients in optimizing their … as Spark/Databricks, Snowflake, BigQuery, Redshift, Azure Synapse, ETL and Data Orchestration tools Good understanding of DevOps and Data pipeline automation platforms Familiarity with Container Technologies such as Kubernetes is desirable but not mandatory Familiarity with legal and regulatory requirements relevant to cloud data more »

firm and throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Do As a Data Architect at BCG, you will be pivotal in designing and developing cloud-based data systems for our clients. Your responsibilities include consulting … logical architecture solution roadmaps, integrating these solutions with client business processes and technology frameworks. Your role involves developing innovative solution sets and enhancing data platform capabilities to address emerging data challenges, leveraging the best of approved Cloud platforms. You'll also guide clients in optimizing their … as Spark/Databricks, Snowflake, BigQuery, Redshift, Azure Synapse, ETL and Data Orchestration tools Good understanding of DevOps and Data pipeline automation platforms Familiarity with Container Technologies such as Kubernetes is desirable but not mandatory Familiarity with legal and regulatory requirements relevant to cloud data more »

Senior Data Engineer Python, SQL, Big Data Frameworks London - 4x a week £80-90,000 bonuses Seeking a skilled Data Engineer to join a company specialising in quantitative trading and investment management. This role is pivotal in building and maintaining the infrastructure that supports … their cutting-edge trading strategies. Key Responsibilities: Design and manage scalable data pipelines. Integrate data from various sources and develop ETL processes. Optimise database systems for high performance. Work with data scientists and analysts to meet data needs. Automate data processing tasks to enhance efficiency. Implement data security measures and ensure compliance with industry standards. Requirements: Master s degree in Computer Science, Engineering, Mathematics, or related field from a top uni. Proven experience as a Data Engineer, preferably in the financial services or hedge more »

facing web app : presenting the analytics to the customer in a way that surfaces the key insights into their portfolio Data processing pipeline : that ingestion of banking data from our customers and generation of proprietary analytics Tech Stack Frontend: Our web app is React + … re deploying changes to our app multiple times a day with Github and CircleCI Data: Databricks workflows power our data pipeline Your time at ONci As soon as you join you will be contributing to the development of our platform - there s plenty to do more »

Data Architect/Cloud Data Architect (GCP) Google Cloud/GCP, Machine Learning/Generative AI Remote-based or Hybrid should you prefer (London) £100k-£140k basic salary bonus Benefits: 25 days holiday bank holidays Private Healthcare Discounted Gym Membership (Nuffield Hussle) Life Insurance, Income Protection … Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect will work with an array of technologies with clients from several industries & sectors meaning you will find … devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to more »

Stride Recruitment are looking for skilled and motivated SAS Data Engineers to join a consulting firm offering end-to-end data solutions, delivering best-in-class experiences for their external clients. Technical Background: SAS SAS Base … Azure or AWS or GCP Python/PySpark Proficiency in SQL and/or similar data technologies Familiarity with data pipeline tools and ETL processes Knowledge of cloud platforms and data architecture Additional Skills: Excellent communication and stakeholder management skills Ability to translate … complex data into actionable insights. Strong analytical and problem-solving abilities This is a great opportunity to work with an award-winning Data Management and Analytics consultancy, working closely with leading software vendors and top industry experts across a range of sectors, unlocking value and insight more »

9.30am - 5.30pm) Reports to: Head of IT Systems Location: Central London Salary: £65k Bonus Structure! In this role, you'll work within the Data & BI team to manage, develop, and optimize the firm’s data infrastructure. This involves designing, developing, and implementing databases, data warehouses, data pipelines, and business intelligence systems to maximize the value of the firm’s data. Responsibilities Maintain SQL Server databases for reliable access and performance. Ensure data system integrity, security, backup, disaster recovery, and availability. Develop and maintain BI solutions using SSRS, Power … BI, Power Apps, and Power Automate. Collaborate with all business areas to develop and deploy data products. Oversee data integration across systems ensuring data quality and consistency. Document the data architecture to maintain an accurate view of the data environment. more »

contributions are valued. We offer world class learning and career development opportunities, while we celebrate an inclusive culture. The role As a Lead Data Scientist (NLP & GenAI) at Allianz Commercial, you will work closely with our team of data scientists, data engineers, ML engineers … their strategic objectives. Research, design, and develop solutions using NLP models and algorithms to extract insights from unstructured text data. Collaborate closely with data engineers to pre-process and clean text data, ensuring data quality and compatibility with NLP models. Apply machine learning and … utilize LLMs, such as GPT, BERT or open source foundational LLM's (Llama2), for natural language understanding and generation tasks. Conduct thorough exploratory data analysis to gain insights into text data characteristics and develop effective pre-processing strategies. Collaborate with data engineers and MLops more »

Job Title: Data Engineer (Contract) Recruitment Company: Open-Source Recruitment Location: Fully Remote Contract Duration: 6 Months Day Rate: £450 per day IR35 Status: Inside IR35 About the Client: Our client is a tech-focused organisation dedicated to making a positive impact through innovative solutions. They are seeking … a skilled Data Engineer for a 6-month contract. Key Responsibilities: Develop and maintain data pipelines and ETL processes. Handle large datasets using PySpark. Manage data workflows with Azure Synapse Analytics and Azure Data Factory. Collaborate with data scientists to … ensure data quality. Optimise data storage and retrieval processes. Implement data governance and security measures. Requirements: Experience: 5 years in data engineering. Programming: Proficiency in Python, SQL. Microsoft Azure: Experience with Azure Synapse Analytics and Azure Data Factory. ETL: Experience more »

time (Monday - Friday, 9.30am - 5.30pm) Reports to: Head of IT Systems Salary: £65k Bonus Structure! In this role, you'll work within the Data & BI team to manage, develop, and optimize a leading Law firm's data infrastructure. This involves designing, developing, and implementing databases, data warehouses, data pipelines, and Business Intelligence systems to maximize the value of the firm's data. Responsibilities Maintain SQL Server databases for reliable access and performance. Ensure data system integrity, security, backup, disaster recovery, and availability. Develop and maintain BI solutions using SSRS, Power … BI, Power Apps, and Power Automate. Collaborate with all business areas to develop and deploy data products. Oversee data integration across systems ensuring data quality and consistency. Document the data architecture to maintain an accurate view of the data environment. more »

Data Engineer (6-Month Contract) - £450 per day inside IR35 Birmingham/Hybrid We are recruiting on behalf of a leading company dedicated to innovative solutions in waste management. As part of our ongoing commitment to excellence, we are currently seeking a skilled Data Engineer to … join our Commercial Waste programme on a 6-month contract basis. Role Overview We are looking for a talented Data Engineer to support enterprise data platforms and warehouses in Azure. You will play a crucial role in gathering detailed requirements from business sponsors/owners and … executing data ingestion and modelling tasks. Experience in producing Power BI reports is desirable but not essential. Key Responsibilities Collaborate with business sponsors/owners to understand and document detailed requirements. Design, develop, and maintain data pipelines using Azure Data Factory and Azure Databricks. more »

Senior Data Engineer Full Time On-Site Python SQL DataBricks Azure As a Senior Data Engineer, you will play a key technical role in design, implementation, and continuous refinement of data pipelines and architectures within Microsoft s Azure ecosystem. This role involves leveraging Azure … s diverse data services, such as Data Factory, Databricks, Azure SQL and Synapse Analytics, among others, to build scalable and secure data solutions that support complex onward processing by both operational and analytical teams. What will this role look like? - Develop and maintain scalable … operational and analytical teams to solve complex business problems. - Monitor system performance, identify bottlenecks, and implement changes to improve data processing and pipeline/systems performance. - Work closely with Data Scientists within the Operational Analytics team to understand data needs and deliver solutions more »

Lead Data Engineer85,000 – 103,000Location: United Kingdom (London – London) Type: PermanentPrincipal Data EngineerLocation: London (2 days a week in the office)Salary: 90k - 105k (DOE)About the Company:Our client, a leading technology-driven organization, is seeking a talented and experienced Lead Data … edge technology to drive business success. This is a fantastic opportunity to work with a team of experts who are passionate about harnessing data to create impactful solutions.Key Responsibilities:Design, develop, and maintain robust ETL (Extract, Transform, Load) processes to ensure efficient data flow and integration … across various systems.Utilize Python for data manipulation, transformation, and automation tasks.Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality solutions.Implement data pipelines and workflows to support real-time and batch data processing.Optimize database more »

Leading ecommerce client are now searching for a Senior Data Engineer to contribute towards the delivery of their data strategy. This engineer will shape our client s data function, delivering end to end solutions for an array of customer data projects. The … role: Designing data pipelines, managing data warehouses, and implementing complex ETL processes. Work closely with data scientists, analysts, and stakeholders to optimise our client s infrastructure. Optimise data driven products, personalisation, reporting and overall business success. Manage the whole data lifecycle (End to End). Ensure the reliability and quality of data. Key Requirements: Strong experience designing data pipelines/warehouses using AWS and Snowflake. Exposure to big data technologies such as Kafka, Spark, or Hadoop. Solid experience with Snowflake, including performance optimisation and more »

for full-stack Haskell engineer to develop and maintain the in-house crypto trading platform. To build upon a real-time data pipeline and web application, tackling tasks like API integration, trade execution algorithms, and more. Ownership is key, with projects ranging from design and architecture to … s success through increased trader productivity. Key Skills: - Proven ability to build user interfaces with React and TypeScript. - Strong SQL proficiency for efficient data management. - Experience with heterogeneous time series data (financial data a plus). - Familiarity with cloud platforms (AWS/GCP/ more »

unit we ensure a high level of customer orientation and we design our services to be cooperative, efficient and future-oriented. As a Data Engineer in our unit Global Overseas Delivery, you ensure data provision for various tools and applications e.g. dashboards and reports and set … up sustainable end-to-end data pipelines due to needs using different platforms and tools. This includes designing the necessary system architecture, programming specific applications as well as configuring and designing interfaces and databases. Furthermore, you support the Data Analytics Team in KPI configuration based on … relevant business data. Working with the relevant stakeholders, you help with data-related technical questions, support their data infrastructure requirements and create your own queries to gain business-relevant insights. In addition, you perform root cause analysis on internal and external data and processes more »

Our client, an exciting start-up customer obsessed behavioural agency founded by two huge names in the data space (that have grown teams previously to 400), is looking for a personable and hard-working Junior Data Engineer to design, build, and maintain a variety of solutions … real business results, a stark contrast to the generic approaches often found at larger agencies. Leading with Relationships and Results As a Junior Data Engineer, you'll leverage your knowledge of data warehousing and Google Cloud Platform (GCP) to design and build data pipelines. … You'll combine your technical skills in data modelling and transformation with your expertise in cloud-based data services like BigQuery and Dataflow. What will you be doing? Data Engineering in Google BigQuery: Design, build, test, and deploy data models and transformations more »