101 to 125 of 422 Permanent Data Pipeline Jobs

Role: Data Engineer - Tech Lead Job Type: Permanent Location: London, UK Do you strive for engineering excellence and are ready to lead a modern tech transformation ? Do you have experience as a Azure, Pyspark, Cloud? We have the perfect role for you- Data Engineer - Tech Lead … engineers. Work with some of the brightest global minds in the industry. Be exposed to the future of data. The Role Being a Data Engineering Tech Lead, you will be at the forefront of leveraging Azure cloud services to design, develop, and implement cutting-edge data engineering solutions. You will be leading a team of data engineers in harnessing the power of Databricks and Synapse to build scalable and efficient data pipelines, data lakes, and analytical solutions. Lead Data Engineering Solutions Development: Lead the design, development more »

Data Engineer – Pricing/Market Data A leading financial services firm are improving their data product offering and have an opportunity for a Data Engineer paly a key role in the development of key data platforms. You will have expertise … in Microsoft technologies for data engineering including significant ETL development experience, including MS SQL, Azure, Fabric, and .NET technologies. This includes creating data pipelines, ingesting data and data modelling. You will be working in a fast-paced execution environment, with a focus … on providing data solutions for market data and pricing within a financial markets context. You should apply for this role if you are/have: 5+ years data engineering experience within financial services Financial markets (buy or sell side) with pricing and/or more »

Data Engineer Remote working, West Sussex, Manchester Up to £54,000 per annum + Bonus An exciting opportunity has arisen to join a leading company in the Insurance and technology sector, renown for its outstanding working culture, in a Data Engineering role. The role will require … who has strong Pyspark and SQL experience alongside an Azure background. Key responsibilities: Develop low to high complexity, secure, governed, high quality, efficient data pipelinesfrom a variety of on and off premise, internal and external data sources Ensure that data is cleansed, mapped, transformed … and optimised for storage to meet requirements for business and technical use cases. Build data observability and data quality by design into all Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies. Build solutions that pipe transform data more »

Our client within the Education sector are looking for a Senior Data Engineer to join them on a contract basis. Inside IR35 6 month contract Hybrid 1 – 2 days per week onsite Technical Skills: In depth knowledge of relevant Azure services such as Azure Data Lake … Storage, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, Azure Stream Analytics, etc. Strong Python or Scala with Spark, PySpark experience Experience with relational databases and NoSQL databases Significant experience and in-depth knowledge of creating data pipelines and associated design principles, standards … Data modelling concepts, ETL procedures, and all steps of data production process Experience with unit and integration testing, and data quality frameworks such as Deequ, Great Expectations or Delta Live Experience working with continuous integration Good understanding of data modelling (medallion architectures more »

Data Engineer Salary: £35-70k (plus very attractive bonus on top) Location: London/Leeds (hybrid working of 1 day a week in either the London or Leeds office, if prefered) We are looking to fill 2 or 3 roles at various seniority levels. PLEASE only apply … recruiting for a Sports Analytics company - a clear leader in their market with considerable expansion plans. Purpose of role: The purpose of the Data Engineer is to implement and … maintain data infrastructure that facilitates data-driven decision making, innovation and operational efficiency while ensuring that the data pipeline is secure, reliable, and scalable. The role holder will build and maintain high-performance data systems that are foundational to driving business more »

using your expertise to uncover valuable insights that drive strategic decision-making and fuel business growth? Do you thrive in a high volume data-oriented environment, where every record holds the potential to uncover radical insights and opportunities? Do you have exceptional skills in big data technologies, real-time data processing, and advanced analytics, and would be capable of handling and extracting insights from a highly dynamic dataset growing by 8 billion records each day? If youre a Big Data Analyst used to working with big data and … high-volume records, and have what it takes to transform complex data into actionable recommendations, informed decisions, and impactful outcomes, then this could be the perfect position for you! Salary & Benefits: Competitive salary between £60,000 - £80,000 per annum depending on relevant technical skills and experience Fully more »

Data Engineer - Advanced Analytics Remote Essential Skills: Computer Science or equivalent education and experience BPSS (SC desired) 2+ years of data experience with data analysis and preparation, including experience with very large data sets Data manipulation and extraction using Python … SQL, AWS Redshift Skills required: Python, SQL, AWS Glue, AWS Redshift, AWS Lambda, AWS Step Functions, AWS App Config Understanding of data migration, migration of existing data platforms to new environment Strong understanding of Python (other languages an advantage), AWS technologies (as above) Strong understanding Data Governance tooling Strong experience in SQL, database development & data modeling Solid business and collaboration skills, and responsive to service needs and operational demands Day to day activities: Act as a liaison between consumer teams and source system/data owners Assess and understand data more »

nineDots are excited to be helping Cloudsmith ramp up their data engineering function. They are on a truly amazing upward trajectory with fantastic seasoned pros at the helm. It's gonna be a fun journey. Are you ready to lead the design and implementation of a best-in … class data platform on AWS?Doyou have a deep understanding of data engineering principles and a passion for creating scalable, efficient, and secure data solutions? If you're seeking a challenging and rewarding role, join Cloudsmith as our Principal Data Engineer! We … engineering (e.g., Redshift, S3, Glue, Lambda, Kinesis). ● Strong Programming Skills: Proficiency in languages like Python, SQL, and experience with data pipeline tools. ● DataModeling and Warehousing: Expertise in data modeling techniques and experience with data warehousing solutions. ● Excellent Problem-Solving Skills: Ability more »

Data Engineer Location: London, 3 days per week in the office Salary range: £50,000-£70,000 + benefits Purpose: Design, build, and maintain scalable data architectures, including pipelines and cloud-based data warehouses. Tech: Python (NumPy, Pandas), SQL, ETL, Cloud (AWS, Azure or … consultancy who solely specialise in the Maritime trading industry. You will work onsite with 1 specific client to design, build and maintain scalable data architectures working with key stakeholders in their business to shape their data-driven strategies. The Company: They have a long history providing … ETL Processes: Deep expertise in developing and optimising ETL pipelines. Version Control: Experience with Git for code collaboration and change tracking. Data Pipeline Tools: Proficiency with tools such as Apache Airflow. Cloud Platforms: Familiarity with AWS, Azure, Snowflake, and GCP. Visualisation: Tableau or PowerBI Delivery Tools: Familiarity more »

We Want We are looking for a Staff Backend Engineer to join our Insights Back-End team to help build out data pipeline, data store and API layer of Bazaarvoice data that will power our various client-facing dashboards, raw data reporting, product performance reporting. The ideal candidate has built and operated systems that aggregate and share data sets to support new product offerings and is well experienced in evaluating the right technologies to build those data sets. Success in the role is underpinned by … a strong understanding of the business cases of the product portfolio and building relevant data sets, API layer and access methodologies that support the portfolio goals. What You’ll Be Doing Own, design, implement and support of major components of the unified back end data. Define how data more »

Data Engineer – London – 6 months contract - £400 per day Umbrella You'll work with the one of the solution team in the biggest functional practice in the firm, focusing on client operations improvement. Our clint fosters innovation driven by analytics, design thinking, by developing new products/services … concepts, to building production ready solutions aligned with our Firm IT ecosystem and its standards. Core of your responsibilities will be in building data pipelines to fuel our advanced analytics models. In these efforts you’ll leverage cloud native services (e.g., Azure, AWS), our Firm frameworks (e.g., Kedro … and variety of industry standard tools. Responsibilities: Collaborate with stakeholders and product managers to build and recommend appropriate data requirements. Data transformation: Identify data sources and build advanced code to extract, transform and load different sources of data using query languages, programming more »

SENIOR DATA SCIENTIST (NLP & LLM) Up to £85,000 LONDON Ready to elevate your career to new heights? Join a team of experienced Data Scientists and Data Engineers to create impactful LLM's models and work with cutting-edge NLP technologies. The Role: Work … performance, and make informed solution selections. Conduct comprehensive work spanning the entire data lifecycle, including data exploration, data pipeline development, insights and feature extraction, machine learning model creation, and deployment. Collaborate on consulting-style pilot projects aimed at helping clients gain a deep … understanding of their data and identifying problems that can be addressed through the platform. Collaborate closely with data scientists and data engineers to construct and implement machine learning models. Take an active role in leading team meetings and client workshops. Assist in establishing software more »

Data Engineer Grade: Consultant Analyst/Consultant Location: Belfast – BT6 8DN In-Office: 2 days per week Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based … data solutions - transforming raw data into actionable insights that inspire ingenuity. The Company: The Organisation consists of 4000 high-performing Digital Strategists and Engineering Technologists - united in delivering human-centred innovation and making a meaningful impact on global society. Boasting a vast network of FTSE … Defence and security, Energy and utilities, Financial Services, Government and Public Services, Healthcare, Life Sciences, and Transport. Essential Skills & Experience: Design and deploy data pipelines in big data architecture using Java, Python, Scala, Spark, and SQL. Execute tasks involving scripting, API data extraction, and more »

developer 6 months contract – Competitive market rates – Inside IR35 - London – 3 days on-site Key Skills: Python, Power BI, SQL, Experience with Big Data pipelines, DAX, Excel, Financial services background The Role: A global trading organisation is looking a Python/Power BI to join their Data … be working with the business helping to build Apps using Python and Power BI for the output dashboards. Key Responsibilities: Develop and construct data pipelines to facilitate efficient extraction, transformation, and loading of data from diverse sources. Create and implement Business Information Access portals, comprising dashboards … continued deployment and enhancements. Innovate existing solutions and forecast future demands to meet evolving business needs. Develop analytics tools leveraging the data pipeline to offer actionable insights across various business functions, thereby enhancing key performance metrics. Emphasize automation and adopt an early feedback approach to streamline processes. more »

Role : Data Product Owner Location : Leeds UK- Hybrid Type of resource : Contract 6+ Months Must Have : Primary POC to Stake holders, Strong Understand of Azure Data Lake & Data Bricks Data product owner responsibilities includes the following: Act as the primary point of … contact for the data product, understanding and representing the needs of stakeholders. Assisting the development team in clarifying requirements and helping them to understand the business side- Mortgage and Savings. Collaborate with stakeholders to gather and prioritize requirements. Develop and maintain a product roadmap, aligning it with business … goals and technical capabilities. Play a crucial role in establishing robust data pipelines, defining data models, setting up data governance systems, and ensuring the secure and efficient processing of data in the lakehouse. Oversee the execution of data projects, ensuring more »

Principal Data Engineer – National Salary Circa £80,000. London Salary Circa £100,000. Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity for an Engineer Manager with significant experience of managing data pipelines. As … well as being an expert in cloud platform, you’ll have a strong background in Data Ingestion and Integration, designing and implementing ETL pipelines on various technologies, Data Quality monitoring and a rounded understanding of data operations. Aviva believes strongly in experimentation leading to … teams across multiple areas. Coding Mastery: Proficient in writing clean, robust code in SQL and Python, with a strong foundation in data pipeline software engineering, testing, version control, and debugging. Experience of mentoring and supporting development of junior engineers. Data Analysis and Processing: Skilled in more »

My Prop-Fintech client is looking for a Data Engineer to join the Head of Data and 2 data analysts in the building, extraction, and refining of new and innovative datasets. This Geospatial Dataset contains billions of datapoints and has infinite possibilities of growth. … bonus, private healthcare for you and the family, Gym Benefits, and home office equipment. KEY RESPONSIBILITY Design and uphold an efficient data pipeline architecture. Enhance data storage platforms for optimal speed, scalability, reliability, and performance. Oversee critical functions such as data acquisition, extraction … preparation, and processing of extensive datasets for predictive modelling purposes. Conduct web scraping and proficiently manipulate data as required. Take the lead in pioneering modelling techniques utilising emerging technologies. Collaborate closely with data subject specialists to enhance data functionality and accessibility. Provide technical support more »

My Prop-Fintech client is looking for a Data Engineer to join the Head of Data and 2 data analysts in the building, extraction, and refining of new and innovative datasets. This Geospatial Dataset contains billions of datapoints and has infinite possibilities of growth. … bonus, private healthcare for you and the family, Gym Benefits, and home office equipment. KEY RESPONSIBILITY Design and uphold an efficient data pipeline architecture. Enhance data storage platforms for optimal speed, scalability, reliability, and performance. Oversee critical functions such as data acquisition, extraction … preparation, and processing of extensive datasets for predictive modelling purposes. Conduct web scraping and proficiently manipulate data as required. Take the lead in pioneering modelling techniques utilising emerging technologies. Collaborate closely with data subject specialists to enhance data functionality and accessibility. Provide technical support more »

Our client, a well-known and market-leading Financial Services company, is currently recruiting for a Senior Data Engineer to join their Data engineering team. The Senior Data Engineer will be responsible for designing, developing, and maintaining data pipelines and workflows to … support the organisation's data processing needs and facilitate BI reporting and analytics delivery. Responsibilities for the Senior Data Engineer Design, develop, and maintain scalable data pipelines and workflows using Azure … Data Factory and Databricks Implement data quality checks and validation processes to ensure accuracy, completeness, and consistency of data. Monitor pipeline performance and troubleshoot issues promptly to ensure uninterrupted data processing. Key Skills & Experience for the Senior Data Engineer Essential Hands more »

Stott and May has partnered with a global healthcare organisation within the private sector, to source a senior data engineer to join their growing tech team. Hybrid (3/4 days per month in London or Manchester) Salary: Circa £80k p/annum Key Responsibilities: Lead the consolidation … and migration of legacy data to our new cloud-based data platform. Design and implement master and transactional data models to ensure data integrity and consistency. Develop ETL pipelines and cloud data storage solutions that meet business requirements and non … functional needs. Collaborate closely with data analysts, visualization engineers, and project managers to deliver data solutions. Surface data for analytical consumption and prepare data for ML/predictive and prescriptive models. What we are looking for: Proficiency with Databricks and building data more »

Vision, and Values guide the way we do business. Are you ready for your next career move? We are in search of a Data Systems Engineer to join us in support of the United States Central Command (USCENTCOM) Information Technology Services (CITS) contract at MacDill, AFB in Tampa … to maintain high readiness rates for an innovative enterprise IT infrastructure on a global scale. In this role, you will work with the Data Analytics and Predictive Analytics PWS task area for the J6 Knowledge Management Data (KM-D) customer set. The Mission The Digital Modernization … query performance. Assist the Government with the selection of backend database technologies, configuration and utilization, and the optimization of the full data pipeline infrastructure to support the actual content, volume, Extract, Transform, Load (ETL), and periodicity of data to support the intended kinds of queries more »

A Financial Service Company is currently seeking a Lead Data Engineer to join its EMEA Technology team. This person will lead the design, development, and maintenance of scalable and efficient data pipelines to support data-driven applications The Lead Data Engineer will … also collaborate with cross-functional teams, including data scientists, software engineers, an product managers, to understand data requirements and deliver high-quality solutions as well as architecting data ingestion, transformation, and storage … processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies. Other core duties include optimizing data pipeline performance, ensuring data accuracy, reliability, and timely delivery. Requirements; Significant experience working as a Lead Data Engineer within Financial Services more »

Data Engineer £35,000 To £45,000 (Dependent on Experience) Norwich Hybrid (2/3 Days per Week Onsite) An exciting opportunity for a data engineering professional to take full ownership of new projects. Join a longstanding business with a talented and dedicated internal IT team … as well as an environment that fosters personal progression. This job role is ideal for someone with a strong background in data engineering and an interest in data analysis. Duties: Lead the implementation and ongoing management of data strategy and analysing raw data … build robust data systems Conduct advanced data analysis Participate in Agile ceremonies and practices Required Skills: ETL Data Pipeline development experience Designing, developing and maintaining data lakes and warehouses SQL Database Design, Coding and Maintenance Programming languages such as SQL and more »

Data Engineer – Salary National up to £35,000/London up to £45,000 Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity to join our highly skilled Data Engineer community. Within one of … our Data Engineering teams, we are looking for an individual who is passionate about data to work within our multi cloud environment to h elp shape the future of insurance through cutting-edge predictive analytics and Data Science. A bit about the job: Data … maintainable, and robust code in SQL or Python. Scala, Java or similar languages would be advantageous but not required. Experience in data pipeline engineering and best practice with CI/CD setup. Working with Oracle and Snowflake are desirable. Knowledge and expertise of data engineering more »

Data Engineer - Rapidly growing highstreet brand - Hybrid - up to £60,000 Burns Sheehan is partnered with a rapidly growing high-street chain who have seen great success recently and are in the process of expanding further in the UK and abroad. This will be s a great opportunity … to have a real impact on the strategy and roadmap for data across the business. Working closely with the wider tech teams on a super greenfield project, this is the perfect opportunity to have a real impact on a scaling business. You will be working very closely with … Integration: Gather data from various departments and tools, integrating it into a central data lake or warehouse. Data Pipeline Development: Design and construct data pipelines to automate data flow, involving ETL processes as needed. Modern tech stack - Python, AWS more »