101 to 125 of 316 Data Lake Jobs

understand business requirements and designing integration solutions to connect Dynamics 365 Finance and Operations with other systems such as ProjOps, Dataverse, Data Lake, Synapse, Enterprise Data Warehouse (EDW), or third-party applications. Development: Writing code and developing custom solutions using relevant programming languages and technologies … such as X++, C#, SQL, XML, JSON) to create seamless data flow and functionality between different systems. API Development: Utilizing APIs (Application Programming Interfaces) provided by Dynamics 365 Finance and Operations and other systems to facilitate data exchange and communication between them. Data Mapping … including solutions. T-SQL development skills Cloud Integration specialisms familiar with the range of Azure integration technologies Functions Logic App Azure Data Lake Azure Synapse Azure DevOps Experience in developing solutions within Power platform (Power Apps, Power automate), Dataverse Experience of integration work with payroll/HR more »

BI Developer that has a strong background in Python and Power BI to join the Software Delivery team to strengthen the power of data within the business. The company is moving its data from legacy stores to a strategic data warehouse and data … coding standards, and tools, while also working collaboratively with the team by utilising communication and problem solving skills. Key responsibilities: Ability to build data pipelines that will allow optimal extraction, transformation, and incorporation of data from various data sources. Be able to utilise Power … BI to build dashboards and reports. Understand the Data Virtualisation layer and partner with the Business for continued deployment and Business enhancements. Improve solutions and be able to anticipate where the demand is for the business. Build analytics tools that utilise the data pipeline to provide more »

Data is crucial to providing a first-class service to our customers and managing our business effectively. The Data Quality Team has a proud history of working with colleagues across numerous business units to help influence behaviour and decisions at all levels of the organisation in … expanding its influence into more of the business's key strategic projects in 2024 and beyond. As well as ensuring that business-critical data is fit for purpose, the team is also responsible for the purchase of third party marketing suppression and enhancement data. We continually assess the … business' use of these assets. Strong technical skills encompassing the ability to understand data from source systems and the data lake, interrogate this data competently using SQL-based code, build automated reports in PowerBI and communicate the results in a timely manner to more »

extensive array of digitally enhanced products and services, tailored to satisfy the diverse needs of their customers who are looking for a lead data and analytics specialist to join their team. Responsibilities: Lead the upskilling, coaching, and mentoring of fellow data engineers. Establish and reinforce data engineering best practices and guidelines. Conceptualize, build, and sustain data pipelines for the efficient extraction, transformation, and loading of data from various sources into our data warehouse and data lakes. Uphold data integrity, quality, and consistency throughout the … entire process. Create and enhance ETL/ELT workflows capable of handling substantial data volumes. Collaborate with data analysts, data scientists, and other stakeholders to comprehend their data needs. Develop and implement appropriate data models to support efficient data more »

safe, and designed for the future.Our human touch sets us apart from other consultancies, system integrators and software houses - with people, technology, and data at the heart of who we are, we believe in creating value and sustainability through everything we do for our clients, staff, communities, and … public-sector, Methods is now building a significant private sector client portfolio.Methods was acquired by the Alten Group in early 2022.As a Lead Data Engineer at Methods you will be expected to get involved with projects from conception to delivery - leading and supervising other engineers to meet requirements … JSON, Parquet, XML etc.Experience with or developed understanding of the application of ETL/ELT tools.Experience with NoSQL type environments, Data Lakes, Lake-Houses (Cassandra, MongoDB or Neptune).Experience with distributed storage, processing engines such as Apache Hadoop and Apache Spark.Experience with message brokering/stream processing more »

An opportunity is available through organic growth for an experienced Senior Data Engineer to join a prestigious company renowned for its commitment to excellence in the realm of professional services research and analysis. This organisation stands at the forefront of providing invaluable insights and intelligence to professional services … and leveraging cutting-edge technologies, they empower the team to drive impactful solutions that shape the future of professional services. As a Senior Data Engineer, you'll be part of a dynamic team dedicated to harnessing the power of data to deliver unparalleled value and insights … of schema evolution and/or serialisation. Extensive experience in building and optimizing data pipelines. Experience engineering and maintaining Data Lake, Data Lakehouse, Delta Lake environment. Hands-on familiarity with Azure DevOps, CI/CD pipelines, and release management. Person Specification: Passionate more »

Senior Data Engineer Google Cloud Platform UK Remote Full-time, Permanent £90,000-100,000 + benefits Through their SaaS platform, bolstered by cutting-edge AI and proprietary data ontology, this company demystifies the intricate and evolving landscape of compliance for their customers. With some of … company regularly raises money for various charities and enables everyone in the business to do so Key Responsibilities: Craft GCP (Google Cloud Platform) data architecture for Machine Learning, Data Science and Artificial Intelligence services Develop data pipelines that are cost effective and low latency … team to build apps and products at scale by building the data solutions Integrate products to the company's data lake Educate the wider team on data best practice Take ownership of the data governance, educate the team on data more »

tools. Thorough experience in understanding, analyzing, and white-box testing (end-to-end) of integrations built using/with Azure integration tools like Data factory, LogicApps, Functions, API Management, Data catalog, WebApps, Integration service environment … configuring monitoring & diagnostics, etc., and connectivity & security configurations. Experience in understanding, analyzing, and white-box testing Azure data storage (Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL solutions. In-depth understanding of and experience with SOAP, XML, REST, JSON, EDI, XSLT, Async … identifying and high-level investigating performance issues to pin-point problem areas Experience with MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse, Azure DevOps, Informatica, BI tools, etc., is a big plus. The role will be based in more »

Contract Relevant context A fast growing Energy & Commodities trading company is embarking on a transition from a CTRM-centric technology architecture to a data-centric model, termed as the Digital Data Hub (DDH), to enhance scalability and stability. The first phase involves the integration of a … Allegro to Topaz derivatives integration project. Engage in the development of various test automation frameworks targeting web applications, thin client, microservices, real-time data processors, data analytics/visualizations reports. Ensure the automation test frameworks and tests are aligned with the existing programme test automation strategy … experience, backend test experience. Exposure/experience in cosmos dB and azure event hub. in data quality assurance around Data Lake, Data bricks and ETL. Exposure working with Power BI. Knowledge and experience of implementing test best practice. This is an Outside IR35 more »

Data Tech Lead – AWS/OBIEE/EDW/ETL/ELT/Media – London - Up to £750 inside IR35 … A global media organisation based in London is looking to recruit a Data Tech Lead with a focus on data lake platforms. This role is within an existing team in the business and is looking for someone who can design and develop the business data … demonstrable experience with OBIEE and EDW is required. Previous experience as a Technical Lead or Architect is preferred. Having knowledge of Data Lake, Virtualization, and experience with cloud data platforms is essential. Prior experience leading a team and migrating data warehouses to big more »

years of design and development experience with various relevant Microsoft platforms, tools, technologies, patterns, and techniques related to Azure tools, integration, and data is required. Experience with industry/domains like pharmaceutical, finance, HR, sales, marketing, and manufacturing is highly preferred. Experience with healthcare industry regulations, data … and connectivity & security configurations (certificates, encryptions, etc.). In-depth knowledge and experience with Azure data storage (SQL Server, Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL processes. Knowledge and some experience of MS Office/MS Office 365 suite, SharePoint Online … DevOps and CI/CD pipeline. Experience working with APIs, Postman/SOAPUI tools, MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse, Informatica, BI tools, etc. is a big plus. General business skills: Clear communication to translate and more »

least 4 years of design and development.experience with various relevant Microsoft platforms, tools, technologies, patterns, and techniques related to Azure tools, integration, and data is required.Experience with industry/domains like pharmaceutical, finance, HR, sales, marketing, and manufacturing is highly preferred.Experience with healthcare industry regulations, data … etc., and connectivity & security configurations (certificates, encryptions, etc.).In-depth knowledge and experience with Azure data storage (SQL Server, Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL processes.Knowledge and some experience of MS Office/MS Office 365 suite, SharePoint Online, Power … Azure DevOps and CI/CD pipeline.Experience working with APIs, Postman/SOAPUI tools, MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse, Informatica, BI tools, etc. is a big plus.General business skills:Clear communication to translate and explain more »

regulations, security, operations, and infrastructure teams to create and/or review integration-focused project documents to cover all the relevant aspects including data, tools, security, compliance, and operations requirements. Develop/Program proofs-of-concept. Support developers in problem-solving. Conduct code-reviews. A bachelor's degree … and connectivity & security configurations (certificates, encryptions, etc.). In-depth knowledge and experience with Azure data storage (SQL Server, Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL processes. In-depth knowledge & experience using Visual Studio, with one of the programming languages C# … systems Experience working with Azure DevOps and CI/CD pipeline. Experience working with MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse Experience or knowledge of Informatica, BI tools, etc. is a big plus. The position will be more »

years of design and development. experience with various relevant Microsoft platforms, tools, technologies, patterns, and techniques related to Azure tools, integration, and data is required. Experience with industry/domains like pharmaceutical, finance, HR, sales, marketing, and manufacturing is highly preferred. Experience with healthcare industry regulations, data … and connectivity & security configurations (certificates, encryptions, etc.). In-depth knowledge and experience with Azure data storage (SQL Server, Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL processes. Knowledge and some experience of MS Office/MS Office 365 suite, SharePoint Online … DevOps and CI/CD pipeline. Experience working with APIs, Postman/SOAPUI tools, MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse, Informatica, BI tools, etc. is a big plus. General business skills: Clear communication to translate and more »

years of design and development. experience with various relevant Microsoft platforms, tools, technologies, patterns, and techniques related to Azure tools, integration, and data is required. Experience with industry/domains like pharmaceutical, finance, HR, sales, marketing, and manufacturing is highly preferred. Experience with healthcare industry regulations, data … and connectivity & security configurations (certificates, encryptions, etc.). In-depth knowledge and experience with Azure data storage (SQL Server, Data lake, Synapse, etc.) & access tools, APIs, cloud connectivity, and ETL processes. Knowledge and some experience of MS Office/MS Office 365 suite, SharePoint Online … DevOps and CI/CD pipeline. Experience working with APIs, Postman/SOAPUI tools, MS D365, ServiceNow, Azure Analytics tools, Azure Data lake, Azure Synapse, Azure BYOD & Data verse, Informatica, BI tools, etc. is a big plus. General business skills: Clear communication to translate and more »

more liquid world. The more we share, the more we all gain. Responsibilities Sharegain is currently in the process of seeking a dedicated Data Engineer to become an integral member of our innovative R&D team. You will play a vital role in managing the entirety of our … data processing framework and infrastructure. Your valuable contributions will not only steer data-centric decision-making but also actively contribute to the evolution of Data monitoring and management solutions across our organization. Within this role, your tasks will encompass the design and execution of … teams (R&D, architects, business) to devise and actualize tailored solutions for their unique operational needs Expand the capabilities of the Data Lake by seamlessly incorporating different data sources Requirements "Can do attitude" Bring over 3 years of hands-on experience in the realms of more »

Lead Data Engineer - Azure/Spark/Databricks Role: Lead Data Engineer Location: London (3 days a week) Salary: £85k - £90k A leading technology firm, renowned for connecting people globally, is looking for a skilled Lead Data Engineer to join their expanding team. In … engineering stack. Key Responsibilities: Lead the design, development, and implementation of data solutions using Azure services such as Spark, Data Lake, Synapse, and Databricks. Code in Spark and oversee the technical design aspects of projects, ensuring adherence to engineering best practices. Lead and mentor a … with remote teams, leveraging your experience in leading distributed teams to drive success. Requirements: Extensive experience with Azure services, including Spark, Data Lake, Synapse, and Databricks. Proficiency in coding with Spark and a strong understanding of design aspects. Deep understanding of engineering best practices and the ability more »

Senior Data Engineer– Databricks/Azure/SQL * Sponsorship is not available with this position * Role – Senior Data Engineer Salary - £70k-£85k DoE Location – Manchester (Hybrid) A well established and award-winning Data consultancy are looking to add a Senior Data Consultant … to rapidly to join their talented data team. You’ll be at the centre of everything they do as a business, supporting a range of top tier clients. The position is offering a basic salary up to £85,000 depending on experience as well as a discretionary bonus … Scheme The Successful Senior Data Engineer will have: · Databricks experience · Skilled across Azure; Azure Data Factor, Azure Data Lake and Azure Synapse. · Advanced knowledge in SQL and good knowledge of Python · Project experience of data warehousing and data modelling more »

Senior Data Engineer– Databricks/Azure/SQL * Sponsorship is not available with this position * Role – Senior Data Engineer Salary - £70k-£85k DoE Location – Manchester (Hybrid) A well established and award-winning Data consultancy are looking to add a Senior Data Consultant … to rapidly to join their talented data team. You’ll be at the centre of everything they do as a business, supporting a range of top tier clients. The position is offering a basic salary up to £85,000 depending on experience as well as a discretionary bonus … Scheme The Successful Senior Data Engineer will have: · Databricks experience · Skilled across Azure; Azure Data Factor, Azure Data Lake and Azure Synapse. · Advanced knowledge in SQL and good knowledge of Python · Project experience of data warehousing and data modelling more »

located in Stamford, CT, USA as a hybrid role.Position Description/Requirements The Azure Cloud AI Engineer is a member of the Enterprise Data Services (EDS) team within Global IT Services. In this capacity, the candidate will work with a team of empathetic, autonomy-supported, well-connected, competent … and passionate data analysts, data engineers, data architects, and data scientists (i.e., a data team) responsible for empowering the organization with modern data science and AI tools, technologies, and workflows. The ideal candidate will be highly empathetic, collaborative … and securing Azure solutions.- Strong experience in automating CI/CD pipelines in Azure.- Strong experience with integrating Azure storage (e.g., Data Lake, Cosmos DB, Microsoft SQL Server, etc.) in broader solutions. - Experience in deployment and functionality of container services (e.g., Azure Kubernetes Service). - Good understanding more »

Senior Microsoft Data Engineer Manchester/Hybrid Up to £60,000 per annum Were looking for an experienced Senior Data Engineer specifically working with Microsoft our client who are a leading engineering end to end risk management solutions business. As the Senior Data engineer … you will own the development and implementation of data and reporting solutions, integrations between applications and provide database administration working closely with our IT DevOps … Team. You will do this using the latest Microsoft technologies: SQL Server Integration Services Azure Synapse Analytics, Azure Analysis Services, Azure Data Lake, Azure SQL Database, Azure Data Factory and Power BI (both dashboards and paginated reports) Core Responsibilities: * Taking responsibility for the successful completion more »

Data Engineer - Rapidly growing highstreet brand - Hybrid - up to £60,000 Burns Sheehan is partnered with a rapidly growing high-street chain who have seen great success recently and are in the process of expanding further in the UK and abroad. This will be s a great opportunity … to have a real impact on the strategy and roadmap for data across the business. Working closely with the wider tech teams on a super greenfield project, this is the perfect opportunity to have a real impact on a scaling business. You will be working very closely with … be up to?: Data Integration: Gather data from various departments and tools, integrating it into a central data lake or warehouse. Data Pipeline Development: Design and construct data pipelines to automate data flow, involving ETL processes as more »

the importance of our connection to our pets and the pivotal role they play in our lives. We are looking for a talented Data Engineer to join our dynamic team. In this role, you will leverage cutting-edge technologies to enhance our data capabilities, providing critical … insights that drive our mission forward. If you're passionate about making a significant impact through innovative data solutions, we’d love to hear from you. Your Role: Data Pipeline Management: Design, build, and maintain scalable data pipelines using Azure Databricks. ETL Processes: Develop … and implement robust ETL processes using big data technologies. Data Modelling: Apply strong expertise in data modelling techniques and warehousing solutions to support operational and analytical use cases. Programming and Querying: Use Python for complex data manipulation tasks and SQL to query more »

Publicis Groupe, Epsilon is a leading provider of multi-channel marketing services, technologies, and database solutions. We do more than collect and store data, and we might be the most important Internet company you've never heard of. Join our team for your chance to work in the … physical implementation, and performance tuning. You will be part of a team that is supporting cloud Data Warehouse and Data Lake environments implemented on AWS or Azure for some of Epsilon's most prominent Database Marketing clients. What you'll do: Design, configure, and manage … Data Warehouse Database Systems on Cloud (AWS Redshift, EMR HBase and Snowflake). Monitor and report on performance metrics, system health, and logging compliance on database services or database appliances. Perform database administration tasks including backup and recovery, troubleshooting, performance tuning and SQL optimization. Design database schema, tune more »

Publicis Groupe, Epsilon is a leading provider of multi-channel marketing services, technologies, and database solutions. We do more than collect and store data, and we might be the most important Internet company you've never heard of. Join our team for your chance to work in the … physical implementation, and performance tuning. You will be part of a team that is supporting cloud Data Warehouse and Data Lake environments implemented on AWS or Azure for some of Epsilon's most prominent Database Marketing clients. What you'll do: Design, configure, and manage … Data Warehouse Database Systems on Cloud (AWS Redshift, EMR HBase and Snowflake). Monitor and report on performance metrics, system health, and logging compliance on database services or database appliances. Perform database administration tasks including backup and recovery, troubleshooting, performance tuning and SQL optimization. Design database schema, tune more »