1 to 25 of 184 Permanent Data Pipeline Jobs in London

sold expertise of working SQL Server SSIS and SSRS. Responsible for the development and delivery of pipelines (collect, validate and prepare high quality data) to automate and streamline processes required by different departments. Support existing SSIS ETLs (overnight and intraday), proactively fine-tune and enhance existing codes to … system definition documents. Proficiency in advanced 4GL programming languages and able to apply to the fully development life-cycle. Database Support To build data pipelines within the SQL environment in SSIS packages, where required, introduce new field and data types for new information to be collated … different business units. To build reports based on business requirements using SQL SSRS. To integrate with all the Bank's sub-systems using data abstraction and connectivity layers, i.e. ODBC, ADO.net. Ensure that the environment is running optimally and with high standard house-keeping procedures. Sharepoint Integration: To more »

sold expertise of working SQL Server SSIS and SSRS. Responsible for the development and delivery of pipelines (collect, validate and prepare high quality data) to automate and streamline processes required by different departments. Support existing SSIS ETLs (overnight and intraday), proactively fine-tune and enhance existing codes to … system definition documents. Proficiency in advanced 4GL programming languages and able to apply to the fully development life-cycle. Database Support To build data pipelines within the SQL environment in SSIS packages, where required, introduce new field and data types for new information to be collated … different business units. To build reports based on business requirements using SQL SSRS. To integrate with all the Bank's sub-systems using data abstraction and connectivity layers, i.e. ODBC, ADO.net. Ensure that the environment is running optimally and with high standard house-keeping procedures. Sharepoint Integration: To more »

Fund at the forefront of Quantitative Finance, known for pioneering research and cutting-edge investment strategies is looking to hire an experienced Quantative Data Developer. Responsibilities: Collaborate closely with Quantitative Researchers to implement and optimize trading strategies. Develop and maintain data pipelines, ensuring the availability and … integrity of market data. Design and implement tools for data analysis, visualization, and model validation. Work with large datasets to extract meaningful insights that can drive investment decisions. Stay abreast of industry trends and incorporate new technologies to enhance existing systems. Qualifications: Strong proficiency in Python and experience more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

seizing energy investment opportunities. Job Summary The job holder will be responsible for expanding and optimising our data and data pipeline architecture, as well as optimising data flow and collection for cross functional teams. The Data Engineer will support our software … developers, database architects, and Trading teams with data initiatives that will ensure optimal data delivery architecture is consistent throughout ongoing projects. This role requires the jobholder to be self-directed and comfortable supporting the data needs of multiple teams, systems and products. They will … architecture to support our next generation of products and data initiatives. Key Responsibilities: Create and maintain optimal data pipeline architecture Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key more »

Lead Data Engineer I'm currently working on a dynamic opportunity which is looking for a Lead Data Engineer to join and lead thier team! You'll play a pivotal role in bridging engineering best practices with thier data domain. Thier team is committed … integrations and click-stream ingestion. Proven ability in stakeholder management and project leadership. Proficiency in SQL, Python, PySpark. Solid background in data pipeline orchestration, data access, and retention tooling. Demonstrable impact on infrastructure scalability and data privacy initiatives. A collaborative spirit, innovative problem … integrations and click-stream ingestion | Proven ability in stakeholder management and project leadership | Proficiency in SQL, Python, PySpark | Solid background in data pipeline orchestration, data access, and retention tooling | Demonstrable impact on infrastructure scalability and data privacy initiatives | Collaborative spirit | Innovative problem-solving more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … science, and engineering discipline. Experience performing data warehouse architecture development and management. Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality more »

Role Overview The successful candidate will set and execute the data engineering roadmap for International systems to align with the goals of the Lyra global data strategy and Lyra International Technology roadmap. You will be responsible for running a small data engineering team, planning … team members’ workloads and utilising your experience to guide the team to solutions to complex problems. You will be responsible for ensuring that data shared across borders, whether to our parent company, subsidiaries, or approved third parties, observes expected safeguards, technical controls, and minimization techniques to comply with … Manage, scale, and mentor team of data engineers Work with stakeholders on priorities and roadmap for data analytics, customer pipelines, data models, and data infrastructure Design and develop new data flows and integrations to support the international migration to a more »

Role: Mid-Level Data Engineer Business Area: Engineering/Technology Reporting to: Head of Data Location: Hybrid/Remote About Validis Validis is a dynamic fintech company with a leading software platform, built for the extraction and standardisation of accounting data. Validis is both a B2B … and SasS with operations across the UK, US & Australia. We specialize in providing quality data for both audits and financial services, with the guarantee that our product will streamline processes for the financial industry. Our current client base holds a number of blue- chip companies, mainly comprising audit … implementing CI/CD workflows for data engineering projects. Proven ability to leverage CI/CD tools to streamline data pipeline development and deployment. Experience designing and implementing ETL pipelines using tools like Apache Airflow, Luigi, Spark, or similar frameworks (familiarity is a plus). more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … best of open-source data engineering and scientific Python toolset. Our tech stack includes Airbyte for data ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data practices, setting standards, and ensuring compliance. Creating data models and overseeing the implementation of changes to address evolving business needs. Providing leadership in evaluating and implementing master data management and metadata management solutions. Collaborating with IT operations and delivery teams, producing essential data more »

Data Engineer – London - £100,000 Description My client, a prominent figure in the insurance industry are on the hunt for a Data Engineer to join there ever growing Data team. The right individual will have a passion for Data as well as … being able to communicate to non-technical stakeholders. The right Data Engineer will ideally have experience working with Azure however my client are open to seeing people equally AWS/GCP. Responsibilities Designing and implementing scalable and efficient data storage solutions on Azure. Building and maintaining … with data scientists and analysts to understand their requirements and develop optimized data models. Monitoring and troubleshooting data pipeline performance issues. Implementing security measures to protect our data assets. Conducting regular data quality checks and applying necessary fixes. Keeping more »

We Want We are looking for a Staff Backend Engineer to join our Insights Back-End team to help build out data pipeline, data store and API layer of Bazaarvoice data that will power our various client-facing dashboards, raw data reporting, product performance reporting. The ideal candidate has built and operated systems that aggregate and share data sets to support new product offerings and is well experienced in evaluating the right technologies to build those data sets. Success in the role is underpinned by … a strong understanding of the business cases of the product portfolio and building relevant data sets, API layer and access methodologies that support the portfolio goals. What You’ll Be Doing Own, design, implement and support of major components of the unified back end data. Define how data more »

Leading eCommerce client are now searching for a Lead Data Engineer to play a pivotal role in delivering their innovative data strategy. This role will shape our client’s data function, delivering end to end data requirements and designing data pipelines/warehouses to ensure data is streamlined and enhances overall business success. The role: Design, develop and test an array of data migration processes to streamline the conversation from existing system to a new Snowflake infrastructure. Manage the end-to-end data lifecycle, enabling the use of data to influence overall business decisions. Streamline our client’s data processes to ensure that customer journeys are personalised. Ensure the accuracy and reliability of data. Collaborate on business projects, advising on data and data more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure … to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data lineage. The successful candidate will have: A strong background in data engineering using Python Strong understanding of Test Driven Development Experience building Star Schemer Databases more »

Senior Azure Data Engineer - 6 Month Contract - Inside IR35 This is an Azure Data Engineer 6 month contract position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as … having strong Azure Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake … framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

since early 2022. FullCircl was formed in 2021 following the merger of Artesian Solutions and DueDil. In September 2023 FullCircl acquired W2 Global Data Solutions. FullCircl is backed by top-tier investors which include Octopus Investments, Notion Capital, Augmentum Fintech and notable angel investors which include Dr Steve … competitive compensation and benefits and believe employee ownership is a key part of cultivating the mindset we want from everyone. The Role Lead Data Engineer We realise at FullCircl that it takes a dedicated team of brilliant people to continually innovate, push boundaries and exceed expectations. We pride … amazing things. Still led by our three original founders, we’re constantly on the lookout for rising stars. We are seeking a lead data engineer to join our team and to work on a variety of assignments alongside our product and commercial management teams. The role will require more »

Job Title: Data Engineer Job Type: Permanent Location: London or Fully remote Salary: £50,000-£70,000 (Dependant on experience) + Bonus + Benefits Skills: Python, SQL, Posgress SQL, Snowflake, Powershell Roc Search`s client provides real-time decisions and active risk monitoring to enable their customers to … maximize profitability while managing risk. their solutions are continually adapting to changing market conditions. They 20+ years of retail data science expertise and experience. They work with a global base of leading commerce partners, representing a big proportion of all US omnichannel retail sales activity across hundreds of … to operations, finance, marketing, and loss prevention. Their performance-improvement solutions yield measurable results with significant return on investment. About the role The Data Engineer will be part of a team that follows DataOps best practices and owns the standardized data assets and the processes around more »

Eames Consulting are delighted to be supporting an established Investment management company that are looking for a Data engineer. Responsibilities: Develop and maintain complex SQL queries, customize reports, and optimize data usage. Create tools to address business challenges and empower users with third-party tools. Design … and implement cloud-based data platforms, ETL pipelines, and data integration. Collaborate with colleagues to ensure data integrity, quality, and security standards are met. Experience required: 3 years experience working as a Data engineer or similar Strong proficiency in SQL or Data Bricks and database development, including complex query optimization and performance tuning. Comfortable using programming languages such as Python Strong Data Warehousing experience - building data pipelines from scratch Good understanding of open source, big data, and cloud data platforms (e.g. AWS more »

culture where your voice is valued. Ready to explore a career path? Start your journey. Job Description As a member of Egencia’s Data Engineering group, you will tackle Egencia’s most sophisticated and impactful challenges using cutting-edge data technologies. Working in an agile development … mindset, you will contribute to our cloud-based data platform and make sure it scales dynamically and operates reliably. Internal and external clients will use the products you develop to gain in-depth insights into travel programs, financials, and operations. Every day, data scientists will leverage … lots of it. We use Python, Java, Scala, Spark, and SQL. We welcome programmers of all backgrounds as long as you focus on data engineering solutions and attention to quality. Foster and strengthen a deep understanding of vast data sources in the cloud and know precisely more »

and this role will face off with a range of business leads in order to build the pipelines that will ultimately ingest the data for their new strats! Responsibilities: Design and implement scalable ETL solutions (structured and unstructured; batch and streaming) Building data pipelines, standardizing, and … cleansing data for the investment team Define and set best practice standards surrounding data (i.e., data modelling, database design, ETL design, job scheduling and monitoring, etc) Manage the entire data flow, from acquisition to transformation into actionable insights using SQL and Python …/Java Design and maintain APIs Requirements: Degree in Computer Science. 4+ years of professional software development experience. A passion for data and experience in applying that passion to high quality data products. Strong knowledge of software engineering best practices, object-oriented concepts, and the ins more »

Job Title: AWS Data Engineer Work Arrangement: Hybrid (London) 2 to 3 days Rate: £400 - £420/day Duration: 6 months Number of Positions: 2 Role Description: We are seeking an experienced AWS Data Engineer with a minimum of 8 years' experience in Big Data platforms. This role will involve working with cutting-edge technology to build and maintain Big Data pipelines in the cloud. Key Responsibilities: Develop code according to given requirements. Create and execute unit test cases. Assist in backlog grooming. Key Skills/Experience: Extensive experience in developing … Big Data pipelines in the cloud using technologies such as Apache Spark. Expertise in performing complex data transformations using Spark SQL queries. Experience orchestrating data pipelines using Apache Airflow. Proficiency in Git-based version control tools. Proficiency with Linux commands and Bash Scripting. Working more »

learning models through research, such as reviewing academic papers and adapting them for our use. Drive the development of the cutting-edge 3D AI pipeline, leveraging the latest breakthroughs in 3D data processing, 3D vision, and machine learning techniques. This entails system design and comprehensive testing. Identify … and address bottlenecks in data, pipeline, and operations, proactively devising and implementing agile solutions to enhance scalability and optimize performance. Take ownership of project management for AI/Vision initiatives, overseeing end-to-end execution and effectively coordinating data operations. Mentor and empower junior team more »

human feel of our match simulation. Areas you’ll be involved with include, Full Body Inverse Kinematics, Motion Matching, Skeleton Retargeting, Data Pipeline and Compression, Events Editor, Real-Life Match Data 3D Viewer, Statistics and Generative AI. Collaborate with other disciplines, including animations, artists, QA more »