101 to 125 of 127 Permanent Data Lake Jobs

such as Puppet, Chef, or SaltPreferred: Bachelor's Degree in Computer Science, Information Technology, Engineering, or associated discipline Experience working with advanced ETL data workflows including technologies such as AWS EMR, Azure Synapse, Azure Data Factory … Apache Hive/Spark/Airflow Experience with IaC deployment of AKS/EKS/GKE architecture Experience with enterprise Data Lake environments using technologies such as DataBricks or SnowflakeCompetencies: Expert analytical/quantitative, problem-solving, and deductive reasoning skills, experience performing advanced troubleshooting and root cause ...

Provision automation using CI/CD (DevOps Pipelines) and IaC (Terraform) tooling Serve as a technical escalation point Work with our development and data teams to integrate products into a DevOps-managed environment Develop and maintain scripts to automate tool/service deployments to our Hybrid Cloud environment …/Airflow Supervisory experience Experience with IaC deployment of AKS/EKS/GKE architecture is highly desired Experience with enterprise Data Lake environments using technologies such as DataBricks or SnowflakeCompetencies: Expert analytical/quantitative, problem-solving, and deductive reasoning skills, with experience performing advanced troubleshooting ...

bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties … will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML and reporting - Designing ...

with an aspiration to invest £1 billion in research over the next ten years.We have an exciting new opportunity for a Head of Data Platforms to join us and play an instrumental role in leading the vision, evolution, and operational excellence of our enterprise data platform. … Ensuring this platform is robust, trusted, cost-effective, and continually advancing to support BHF’s strategic ambitions, you’ll lead our Data Platform Squad and will strengthen the product-based operating model, and ensure dependable, future-proof data foundations. Key responsibilities for this role include: Bridging ...

with likeminded people on an innovative project. You’ll join a team of 4 to contribute to the build of a greenfield market data platform that will enhance how this systematic fund will trade and analyse global markets. You'll integrate various data sources, including financial … Building a new API for the market data platform. Writing a data dictionary for the platform. Adopting S3 for a datalake solution. If you're developer with a passion for building and supporting market data platforms, apply or reach out via email: megan@saragossa.io ...

/ML, and automation frameworks Proven experience in Designing and implementing AI-based test automation frameworks. NLP, LLMs (e.g., GPT-4, BERT), and data annotation techniques. Tools such as TensorFlow, PyTorch, Hugging Face, or equivalent. Strong programming background: Python, Java, or Scala. Experience with test automation tools like … analysis Knowledge of CI/CD pipelines, DevOps, and tools like Jenkins, GitHub Actions, Azure DevOps, etc. Experience working with structured/unstructured data, data lakes, and data pipelines. Demonstrated success in delivering greenfield AI/ML initiatives from concept to scale. ...

solutions provider in Dublin is seeking a senior Data Platform Engineer to design and build a cutting-edge data lake/lakehouse. This hands-on role requires extensive experience in data platform engineering, high-throughput pipelines, and open-source technologies like Apache ...

implement secure & scalable cloud-native architectures (Microservices,Serverless, Container) using comprehensive AWS stack Build end-to-end IoT solutions from device connectivity through data analytics Develop real-time streaming data pipelines from hardware devices and sensors Design and implement time-series database solutions and data engineeringframeworks Create robust proof-of-concepts (POCs) from ground-up using AWS servicesBig Data & Analytics Platform Development (25%) Implement and optimize big data solutions using Amazon Kinesis, Glue, EMR,Redshift, and Timestream Design data lakes and warehouses for massive IoT data ...

Data Engineer- Highly competitive salary About the Role: Were partnering with a leading technology consultancy that helps organisations harness the power of data to modernise platforms and drive business outcomes. As a Data Engineer, youll be at the forefront of designing and delivering cloud … Google Cloud, turning complex datasets into actionable insights. In this role, youll work on diverse projects, from batch and streaming pipelines to data warehouses, data lakes, and AI-powered analytics platforms. This is a hands-on role where your expertise will guide delivery, shape best practices ...

performant Leverages contemporary advanced analytics capabilities and visualisations Provides simple and efficient BI self-service reporting Supports a single analytical view of our data and information Uses agreed and defined metrics Is built in line with our information management strategy and guiding principles Delivers business value through action … Ensures “explainability” fit for audit in a regulatory controlled environment Working as part of the Reporting & Analytics team in close collaboration with the Data Engineering team and business areas to take responsibility and accountability for collaborative design and build of end-user visualisations, reports, and ad-hoc analysis ...

Lambda, SNS, SQS, Step Functions, EventBridge, API Gateway, Load Balancers, and Aurora DB Implement IAM, KMS, and security best practices Build and manage data pipelines and integrations with Databricks Implement application logging, monitoring, and alerting using tools such asSplunk, Dynatrace, and CloudWatch Collaborate with cross-functional teams including … large-scale data processing Hands-on experience with logging, monitoring, and observability tools (Splunk, Dynatrace, etc.) Strong understanding of data lake concepts Nice to Have Experience working with Snowflake ...

collaborative and inclusive culture. About the role CitNOW Group has rapidly expanded through multiple acquisitions, creating an exciting ecosystem of platforms, applications and data landscapes. We are seeking an experienced Enterprise Architect to define and implement a group-wide architecture strategy that unifies these systems within a cohesive … enterprise architecture, with a proven track record in post-merger and acquisition integration. The ideal candidate will have deep knowledge of integration platforms, data architecture and cloud technologies, combined with excellent communication and stakeholder management skills to influence effectively across diverse business units. This role is critical ...

Senior Data Engineer - Azure Cloud Migration & Systems Integration I am recruiting for a rapidly growing, multi-site healthcare organisation in the middle of a major digital transformation. As their Data & Business Intelligence function continues to expand, they require a Senior/Lead Data Engineer … Azure migration and enterprise integration strategy. You'll be joining at a pivotal time as the business migrates to Azure, modernises its data platform, and connects a complex ecosystem of finance, HR, and core operational systems. The role comes with strong visibility across the business and regular interaction ...

support its Legacy Analytics Modernization initiatives. This role will lead efforts to migrate legacy TIERS analytics and reporting platforms to modern data and analytics solutions in alignment with DIRs Texas Data Management Framework. The Project Manager will oversee complex, multi-stakeholder IT initiatives involving data … track project scope, schedule, budget, risks, issues, and dependencies using established project management tools. Collaborate closely with the Office of the Chief Data Architect and the Office of Data Analytics & Performance to support Access & Eligibility Services reporting modernization. Establish and maintain detailed project plans, work plans ...

numi is proud to partner with a growing global technology consultancy seeking multiple Mid/Senior/Principal Data Engineers to join their team in London on a hybrid basis. An idea of some of the hires we are making: 8 AWS Data Engineers 1 Cloud … Engineer AWS 2 Data Engineers with Oracle 5 Snowflake Engineers with AWS 2 Data Engineers with Azure This consultancy empowers some of the world's biggest businesses by delivering advanced data, AI, cloud, and digital solutions. They are committed to providing a platform ...

Lead Database Administrator Location: Hybrid Salary: Up to £60,000 per annum Be the data powerhouse behind the Buzz Bingo experience! At Buzz, everything we do starts with people, our customers and you. If you’re passionate about tech, love bringing new ideas to life, and enjoy leading … into a club to every click they make online. As our Lead Database Administrator, you’ll play a central role in shaping the data foundations that power unforgettable player experiences across the UK. What you’ll be doing You’ll lead the way on all things data ...

ITIL 4 UK Public Sector deals Familiarity with: AI(Ops), Intelligent Automation, Lean and Analytics Experience with tooling for Discovery, Monitoring and Patching Data lakes Modern Catalog Management ServiceNow, SMAX or Remedy is a plus UK Security Clearance: At least ...

secure designs and governance within ZLAP IT's evolving architecture, ensuring that new and existing technologies adhere to Zurich's Information Security and Data Assurance Policies. They will be accountable for the secure delivery of IT projects, working closely with both ZLAP IT and wider ZLAP teams … . API and integration technologies (Mulesoft, web services). Backend policy administration and servicing platforms (Salesforce, Life400, VPMS, BRC, automated underwriting, data lake, fund administration, document management system). Collaborate with ongoing modernization projects (e.g., Calcs platform, Group Risk policy admin, document storage and rendering, payroll enhancements ...

earliest possible date, a full-time and permanent Software Developer (m/f/d) for scientific applications with a focus on Big Data to strengthen our development team at KWS Saat SE & Co. KGaA in Einbeck, Lower Saxony. The role involves replacing an existing application … C# (.NET Core) and Python Practical experience with distributed systems, ideally in an AWS cloud environment or with Docker and Kubernetes Practical experience with DataLake technologies, such as Databricks or AWS Athena and Apache Spark Practical experience in professional software development (e.g., dependency injection, mocking, TDD, CI/CD pipelines ...

Description: Bachelors degree in Computer Science Engineering or related field or equivalent experience Build and optimize data pipelines and ETL processes Develop and maintain data lakes and warehouses (Azure, AWS, GCP, or Snowflake) Integrate structured and unstructured data from multiple sources Support AI/… model training, deployment, and monitoring Ensure data quality, integrity, and governance standards Collaborate with data scientists on feature engineering and model readiness We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit ...

client in the financial services industry. You will work across cutting‐edge platforms including Payment Networks, Digital Banking, Emerging Payments, and highly scalable data and analytics environments built on Google Cloud Platform, Data Lakes and Power BI. This role involves modern engineering, integration, testing, and hands … deploying Hydra‐based applications. Experience in Tokenization and Cryptogram technology stacks. Desirable Skills/Knowledge/Experience Experience with multi-threading, data binding, dependency properties. Strong scripting experience: UNIX, SQL, PL/SQL, Apache Cassandra. Familiarity with testing frameworks such as JUnit and system testing. Strong debugging skills. ...

trigger next steps 2. Commercial Compliance Automation – to ensure activities and invoicing meet commercial terms and other customer account data 3. Datalake Construction, Analysis and WriteBack – enabling data transfer to new systems, between systems and writing back – covering Office, to SaaS to public data ...

world-leading global quantitative hedge fund are looking for a Data Engineer to support their systematic trading and research platform, with a focus on building, managing, and scaling datasets used directly by quants and traders. You will be responsible for developing automated ETL pipelines, integrating new and alternative … checks, and contributing to the firm’s core data framework while working closely with research and trading teams. Stack: Python, Databases/Datalakes, Git The firm offers a highly collaborative, global environment with strong ownership, close interaction with quants and traders, and the opportunity to work on data ...

throughs in systems throughput by identifying and eliminating bottlenecks Leverage technology such as Python, Golang, AWS, Django, Kubernetes, Bash, Terraform, MySQL, Redis, Clickhouse, Iceberg, DataLake, Istio, Postgresql, and more to advance Klaviyo's platform Champion best practices by actively collaborating with other teams in a culture that values whiteboarding … effort to resolve scalability issues Produce and advocate for preventative, upstream solutions with internal stakeholders and external vendors and dependencies Confidently make informed, data-driven choices in a fast paced environment with competing priorities Who you are You care deeply about building software for the long term, balancing ...

making it structured, accessible, affordable, and actionable. Over the next 3 years, Twilio is rebuilding nearly every component of our observability platform, from data collection to real-time analytics. You will drive core initiatives that move Twilio from fragmented tooling to a unified, OpenTelemetry-first observability stack built … tracing infrastructure, or profiling tools). Lead technical execution for major components of Twilio's observability overhaul, including shift to centralized S3-based data lakes, OpenTelemetry instrumentation, and ClickHouse-backed query engines. Deep proficiency in at least one modern programming language (e.g., Go, Python, Java). Familiarity with ...