451 to 475 of 583 Data Pipeline Jobs in the UK

Bash scripts. - Security & Compliance: Implement and maintain security guardrails, IAM roles, and network security protocols, including Cloud Security Command Center. - CI/CD Pipeline Development: Create and manage containerized environments, frequently utilising Kubernetes (GKE) and Jenkins or GitLab CI. - Data & Monitoring: Set up data storage … solutions, manage data pipelines with BigQuery or Dataflow, and monitor system performance for optimisation Required Technical Skills & Experience - GCP Expertise: Deep hands-on experience with core GCP services (VPC, GKE, Compute Engine, Cloud Storage). - Infrastructure as Code (IaC): Proficiency in Terraform is highly required. - Scripting/Programming ...

quality, security, testing, and deployment. Lead key technical initiatives, contributing code and architecture reviews. Drive integration of software with hardware systems, sensors, and data pipelines. Management & Team Leadership Build, lead, and mentor a multidisciplinary engineering team. Establish effective workflows for development, QA, deployment, and iteration. Foster a collaborative … Docker, Cloudformation, Pulumi, CI/CD with GitHub Actions ML/AI: PyTorch, TensorFlow, scikit‐learn, MLflow for experiment tracking, Zenml for ML pipelines Data: PostgreSQL, DynamoDB, data pipelines with Zenml (We’re open to evolving the stack and expect you to influence technical decisions ...

DevOps Engineer (Data/BioTech)Oxford (1 day per month in office).£60-80k + Benefits. This role cannot offer Visa Sponsorship. Your new company I am exclusively recruiting an AWS DevOps Engineer for a fast-growing, clinical-stage life sciences organisation developing a first … technology to tackle diseases, including cancer and autoimmune conditions.The business is privately funded, entering a critical growth phase, and investing heavily into its data and cloud capability to support scientific discovery and clinical delivery. With active clinical programmes underway, technology is a core enabler rather than a support ...

optimisation of transaction monitoring systems. The successful candidate will play a key role in delivering strategic change across financial crime systems, supporting data analysis, system enhancements, and regulatory-driven initiatives. You will work closely with Compliance, Operations, and Technology teams to ensure systems are robust, scalable, and aligned … with evolving regulatory expectations. Key Responsibilities Support delivery of Financial Crime transaction monitoring transformation initiatives Perform advanced data analysis across payment, client, and screening data sets Support audit requests, regulatory reviews, and remediation activities Assist with system upgrades, regression testing, and new technology evaluations (RFPs/ ...

skilled Senior Developer with expertise in Python, Generative AI, Vector Databases, and Spark. The role focuses on building scalable AI-driven applications and data processing solutions. Python Development Develop robust applications and APIs using Python. Write clean, efficient, and maintainable code. Troubleshoot and resolve complex issues. Generative … scalability. Vector Database Work with vector databases like Pinecone, FAISS, Weaviate. Develop embedding pipelines and semantic search. Optimize indexing and retrieval performance. Spark & Data Processing Build data pipelines using Apache Spark. Handle large-scale data processing. Optimize performance and scalability. CI/CD & DevOps ...

Designing, building, and scaling full stack applications Owning system architecture, deployment, and engineering standards Integrating AI features into products using modern APIs and data-driven approaches Improving CI/CD, environments, and overall delivery capability Tech environment - Not set in Stone Frontend: React, Next.js, TypeScript (or similar) Backend … Node.js (Express, NestJS) or Python (FastAPI, Django) (or similar) APIs: REST, GraphQL (or similar) Databases: PostgreSQL, MongoDB (or similar) AI/Data: Experience integrating AI/ML features via APIs (e.g. OpenAI, Anthropic or similar) Exposure to data pipelines, inference workflows, or retrieval-based systems ...

Title: Data Scientist (Generative AI & LLMs) Location: London(Hybrid role) Employment Type: 12 months contract Role Overview We are seeking a versatile Data Scientist to lead the end-to-end development of AI solutions, with a heavy focus on Generative AI and Large Language Models (LLMs … . You will bridge the gap between business requirements and technical execution, overseeing the entire lifecycle from initial scoping and data engineering to model deployment and prompt optimization. Key Responsibilities Business Alignment: Partner with stakeholders to define project scope, translate business problems into technical specs, and establish clear ...

partnering with a fast-scaling fintech that's building the intelligence layer for debt markets. Think financial analysis, credit data, legal and ESG insights, all delivered at speed to some of the biggest names in banking, asset management and law. They need a Senior Software Engineer to join … backbone of their platform. What you'll be doing Backend-heavy engineering across a Python/FastAPI stack, working on everything from data pipelines and API design to ML/AI service integration. You'll be a proper senior IC, running code reviews, mentoring engineers, and getting involved ...

FTSE100 retailer, building cutting-edge, enterprise-grade algorithmic platforms for global institutions. Our mission is to harness the power of AI to transform data into actionable insights, driving smarter decisions in capital markets, asset management, and fintech innovation. As we expand our core engineering team, we are seeking … concurrency, and performance tuning. Key Responsibilities Lead the design and development of backend services using Python and the Django Rest Framework (DRF) Optimize data processing and I/O-heavy operations, with advanced understanding of Global Interpreter Lock (GIL) and Threading Implement clean, type-safe code using Python ...

Your Responsibilities: Full Stack Development: Design, develop, and maintain end-to-end AI solutions, including Front End interfaces, Back End services, and data pipelines. AI System Development: Implement and optimize AI models, ensuring they are scalable, maintainable, and production-ready. Work on GenAI, Agentic AI, and classic … solutions with existing enterprise systems and ensure seamless operation within the client's technology stack. Collaboration: Work closely with cross-functional teams, including data scientists, software engineers, and business stakeholders, to identify opportunities and deliver robust AI solutions. Technical Leadership: Provide technical guidance and mentorship to junior engineers. ...

week in our Waterloo office Experience level: At least five years in commercial software engineering 🚀 About Orchestra Orchestra is an AI-native data pipelining platform building the future of data orchestration and observability . We have built a declarative framework allowing data engineers … explicitly declare how their data pipelines behave. Think CI/CD but for data teams! Backed by VCs from San Francisco, New York and London with angels from across the data space and beyond. 👷 Engineering at Orchestra We care deeply about our users—their ...

deploying NLP models using the latest techniques to solve real business problems at scale, and exposing them through APIs Working with unstructured data to extract insights and build robust data pipelines Evaluating, tracking, and improving the performance of NLP models through MLOps Driving innovation through research … deploying NLP models using the latest techniques to solve real business problems at scale, and exposing them through APIs Working with unstructured data to extract insights and build robust data pipelines valuating, tracking, and improving the performance of NLP models through MLOps Driving innovation through research ...

Overview You’re passionate about modern data science, staying abreast of the latest news and implementing recent discoveries into your day to day work. You’re an innovator; can we do this better? How can we do things differently? Could we solve this quicker? Your mind is always … occupied by these questions, what can be improved with data science. A company scaling on the back of success, expanding across Europe, needs you. They want your thirst for knowledge, the desire to be better and the confidence to try something new, get it wrong and try again. ...

handling, autoscaling - Ensure graceful handling of failovers, deployments, and edge cases across all services Data Pipelines & Warehouse - Manage the Airbyte replication pipeline from production database to data warehouse - Configure incremental replication for new tables as the product evolves - Maintain the data warehouse (PostgreSQL … autovacuum tuning, memory parameters, capacity planning - Manage RudderStack for event streaming to analytics (Amplitude) and attribution (AppsFlyer) CI/CD & Deployment - Own CircleCI pipeline configuration and reliability - Manage ECS deployment strategies, health checks, and rollout verification - Maintain test environments and cleanup automation WHAT WE'RE LOOKING FOR Must Have ...

spinout building AI-powered clinical decision-support tools certified as Software as a Medical Device (SaMD). Our platform uses real-world EHR data to identify patients at risk of preventable harm — giving NHS clinicians time to act. We are a small, high-ownership team operating … Role You will own and shape the technical core of a live clinical AI platform — working across backend services, ML infrastructure, and hospital data integrations. You will have a direct line to the CTO and CEO from day one, with real influence over architectural decisions on a system ...

Your new role As part of the Data Analytics & AI team, you'll help build and evolve a Snowflake-based data platform. You will design and maintain robust data pipelines and models that transform data from operational systems into trusted, analytics-ready ...

Head of Engineering (Data Platform) Azure & Databricks transformation | Scaling Media Organisation | Hybrid (London) The Company Our client is a fast-growing, technology-led media organisation where data and cloud platforms sit at the heart of how the business operates. They are investing heavily in modern infrastructure … initiatives, and highly connected digital services across the organisation. This hire plays a critical role in shaping how teams access, share, and operationalise data at scale. Why this role stands out: Strategic ownership of cloud and data platforms across the business Direct involvement in enabling ...

processes and testing pipelines to support quick iterative development and system stability Work on ML model development with our team to build robust data pipelines and deploy ML models Analyse large corpora of data and build real-time recommendation systems Work with large amounts of data ...

Python Software Engineer – Data & Research Platforms Client: Elite Prop Trading Firm Location: London (Hybrid) Compensation: Up to £275,000 Base + Bonus Overview An elite prop trading firm is seeking an experienced Python Software Engineer to join a growing engineering team focused on building large-scale data … engineering quality, scalability, and reproducibility in research. The Role As a Python Software Engineer, you will design and build distributed systems and data tooling that support quantitative research at scale. Key responsibilities include: Designing and developing Python-based data and research platforms for large-scale analysis ...

Title: Founding Data Engineer 💰 Salary: up to ~£120k + profit share 📈 Equity: 0.75 - 1% 📍 Location: Old Street (4-5 office days/week) 🔐 Company: Data processing AI Agents 👥 Team : ~5 💸 Funding : Pre-seed About the company This early stage start-up is processing hundreds of thousands … account on regular basis - not a promise of a huge payout IF the company succeeds and sells. About the role You'll own data infrastructure end-to-end - ingestion, transformation, storage, delivery. The current agent system works. Your job is to scale it: more documents, more asset classes ...

strong engineering capability is the bigger priority. What you will do Design and build core platform components including pricing services, APIs, and data pipelines Develop scalable ingestion and processing systems for market, macro, and time-series data Build internal tooling that improves how analytics and trading … platform What you will need Strong software engineering fundamentals across architecture, design, and quality practices 5+ years’ experience in platform engineering, quant development, data engineering, or software engineering within financial services Strong Python experience in production environments Exposure to C++ or Rust would be beneficial for performance-sensitive ...

Responsibilities: Design and build responsive web and mobile experiences for end users. Develop intuitive interfaces for operators and remote supervision, integrating teleoperation, navigation data, and safety features. Create and maintain dashboards and front-end systems that support operational workflows and data-driven decision-making. Develop … manage scalable backend services, including APIs, databases, and real-time data pipelines. Collaborate with AI and autonomy teams to embed advanced models into user-facing products and tools. Work closely with cross-functional teams and take ownership of features from concept through to deployment. Required Skills & Experience: 5+ ...

Responsibilities: Design and build responsive web and mobile experiences for end users. Develop intuitive interfaces for operators and remote supervision, integrating teleoperation, navigation data, and safety features. Create and maintain dashboards and front-end systems that support operational workflows and data-driven decision-making. Develop … manage scalable backend services, including APIs, databases, and real-time data pipelines. Collaborate with AI and autonomy teams to embed advanced models into user-facing products and tools. Work closely with cross-functional teams and take ownership of features from concept through to deployment. Required Skills & Experience: 5+ ...

within Birketts’ core ecosystem, including Microsoft 365 (Copilot), iManage (Document Management), Aderant Expert (Practice Management), Intapp Intake (new client onboarding) and Microsoft Fabric (data platform). Azure is a likely hosting platform for any future ‘Birketts-approved AI environment’, subject to requirements and build vs buy decisions. … repeatable AI-enabled workflows. This role is delivery-oriented: you will design, build and ship incremental improvements, partnering with colleagues across Development, Data, Security, Knowledge Management and Learning & Development to drive adoption, skills sharing and effective change management. You will also support evidence-based build vs buy decisions ...

LGBTQ+ business community. Please do not contact the recruiter directly. JOB DETAILS JOB BAND: D CONTRACT TYPE: Permanent DEPARTMENT: BBC Product Group - Data Platforms LOCATION: Newcastle - Hybrid PROPOSED SALARY RANGE: £65,000 - £80,000 depending on relevant skills, knowledge and experience. The expected salary range for this role … now. Flexible working will be part of the discussion at offer stage PURPOSE OF THE ROLE Join the BBC's Data Sourcing teams and help shape the future of our streaming and analytics platforms. As a Principal Software Engineer, you'll provide technical leadership across critical ingestion ...