1 to 25 of 127 A/B Testing Jobs

Requirements / h3 ul li Bachelor's, Master’s, or PhD or equivalent experience in Computer Science, Machine Learning, Applied Mathematics, Statistics, Financial Engineering, or a related quantitative field / li , li Significant professional experience (typically 7–12+ years) in AI / ML product development … / management, model validation, quantitative research, risk modelling, or related areas / li , li Demonstrated success working with technical teams or senior specialists in high stakes modelling environments / li , li Deep understanding of AI / ML systems including LLMs, agentic architectures, RAG pipelines, credit ...

Core Focus) Own the design and evolution of evaluation frameworks for LLM-based systems (both offline and online) Build and scale A / B testing to measure real customer impact and guide product decisions Define meaningful metrics that connect system performance to customer … clarity to ambiguous problem spaces and turn them into actionable changes to the system Cross-functional Collaboration Work daily with Product, Engineering, and Content / Design to shape the assistant’s behaviour Influence decisions about the product through clear communication, data visualisations, and well-structured impactful proposals. Ensure ...

churn, engagement) Help track performance and support scenario analysis for acquisition and retention initiatives Support the design, analysis, and reporting of A / B tests across the customer lifecycle Help define success metrics and interpret test results to inform future activity Support the creation, maintenance … related quantitative field, with a deep understanding of statistical significance, hypothesis testing, and experimental design (A / B testing). Core Technical Skills: Good working knowledge of SQL for data extraction and analysis (experience with Google BigQuery is a ...

manual effort and support scalable, end-to-end delivery. You will collaborate with cross-functional teams to validate use cases via A / B testing and ensure alignment with customer and business outcomes. You will host regular interlock sessions to review defects, share … delivery, or proactive service enablement. You will have experience leading automation and data driven fault prevention initiatives, including the use of A / B testing to validate improvements. You will have extensive experience and deep knowledge in Fixed and Mobile Network Design, Capacity ...

Responsibilities Own the full website user experience, ensuring journeys are intuitive, frictionless, and aligned to customer needs. Lead CRO activity including A / B testing, user testing, heatmaps, and behavioural analysis. Translate insights into clear recommendations that improve conversion and engagement. … teams and influencing decision-making. Manage the backlog of website improvements and prioritise based on impact. Monitor performance metrics and report on key UX / CRO outcomes. Act as the internal voice of the customer, ensuring digital decisions are insight-led. About You This role suits someone ambitious, proactive ...

experimentation-led environments. The Role Analyse how members use the discount card, identifying behavioural trends, usage patterns, and drop-offs Evaluate partner (merchant / client) performance and engagement with the product Use multiple data sources including customer behaviour data, surveys, pain points, and year-to-year trends Support experimentation … A / B testing, measuring impact and surfacing actionable recommendations Translate complex analysis into simple, compelling insights for product, commercial, and senior stakeholders Present findings clearly, focusing on what the data means and what action to take, not just the numbers Work closely ...

schema review and dataview configuration in CJA - validating that migrated dimensions and metrics produce consistent results between AA and CJA Create and maintain pre / post comparison dashboards for site migrations and feature rollouts - following a wave-based rollout methodology across international locales with baseline measurement … post-launch monitoring Build reporting dashboards for Braze email campaigns, landing page templates, and MarTech experiments - including IP warming tracking, A / B test result visualization, and end-to-end campaign performance reporting. Build these dashboards in Power BI with scheduled data refresh from Databricks ...

from concept through to launch Work with Shopify Liquid, APIs and third-party integrations Develop responsive web experiences using JavaScript, jQuery, HTML and CSS / SCSS Create intuitive and user-friendly backend environments for website administrators Utilise AI development tools to support theme creation and development workflows Support data … migration and synchronisation between Shopify and external platforms Implement website performance optimisation and best practice development standards Assist with A / B testing, SEO implementation and data-driven improvements Collaborate across multiple projects while managing priorities effectively Troubleshoot technical issues and contribute innovative ...

a large-scale consumer app Analyse user behaviour, engagement, and lifecycle trends Own the end-to-end experimentation process (A / B testing) Build dashboards, metrics, and reporting frameworks Identify growth opportunities and product improvements Partner with product and engineering teams … into clear, actionable insights What They're Looking For 5+ years' experience in data or product analytics Strong experience with experimentation (A / B testing, hypothesis testing) Ability to turn complex data into clear business recommendations Experience working in product ...

e.g., NDCG, recall, precision) LLM and RAG evaluation methodologies Grounding and faithfulness evaluation Human evaluation and annotation strategies Online experimentation and A / B testing Establish best practices for offline benchmarking, online experimentation, and reproducible evaluation workflows. Build scalable processes for benchmark creation … systems. Information retrieval and semantic search Retrieval-augmented generation (RAG) Evaluation methodologies for IR and generative AI systems Experimentation frameworks and A / B testing Strong experience with: Vector retrieval and hybrid search architectures. LLM evaluation and AI quality measurement Embeddings, reranking ...

pages including homepage, product pages, collection pages, campaign landing pages, and checkout flows Conduct and interpret user research, heatmap analysis, and A / B test results to inform design decisions and drive continuous improvement Develop and maintain a coherent design system and component … beauty, fashion, or premium lifestyle Strong command of UX principles: information architecture, user flows, interaction design, and accessibility standards Experience with A / B testing methodologies and heatmap tools such as Hotjar - comfortable using data to validate design decisions and iterate with purpose ...

APAC Job Summary The Senior DevOps Engineer serves as a critical execution layer for Mercans AI-native infrastructure strategy and broader product / engineering delivery and automation. Either based in Tartu, Estonia or Remote, reporting directly to the CTO, and collaborating with Product Managers, Engineering Managers, software … focuses on building and maintaining a secure, cost-efficient private cloud environment capable of hyperscale payroll processing and proprietary AI model training / inference, while providing deployment automation, feature flagging, and release orchestration to accelerate Product and Engineering team velocity. Duties and Responsibilities: Platform Automation & CI / ...

Data Analyst Location: Manchester (Hybrid: 1 day per week onsite, typically Wednesdays) Duration: August 2026 - April 2027 (Maternity Cover) Rate: £540 / day / £72 / hour (Inside IR35 / Paid via an Umbrella Company) About the Business We are a global leader … cleansing, validation, aggregation, and reporting of complex datasets to create scalable, long-term analytics solutions. Experimentation & Insight: Design, execute, and analyse A / B (and multivariate) tests to measure the success of new product features and business initiatives. Stakeholder Influence: Autonomously partner with cross-functional ...

decisions. Technology is at our core and innovation is everywhere. But our company is more than datasets, lines of code or A / B tests. We're the thrill of the first night in a new place. The excitement of the following morning. … have the opportunity to be a founding team member. Key Job Responsibilities and Duties: Owns and sets the vision for their product / feature, informed by data, user research and overarching vision of their product area Lays out a 1y strategy to achieve the vision ...

unique opportunity to transform reporting, build behavioural analytics, and work across the whole ecosystem lifecycle. This is a hybrid role (2 / 3 days week) for 13 months with potential to extend. Responsibilities Building robust data foundations: Collaborate with engineering to develop pipelines, improve data integrity … reduce escalations. Advancing AI-native analytics: Leverage Gen AI tools to automate recurring insight generation Enabling experimentation & continuous improvement: Support A / B testing and digital experience evaluations Influencing strategy through storytelling: Communicate insights and trends clearly and persuasively to technical ...

unique opportunity to transform reporting, build behavioural analytics, and work across the whole ecosystem lifecycle. This is a hybrid role (2 / 3 days week) for 13 months with potential to extend. Responsibilities Building robust data foundations: Collaborate with engineering to develop pipelines, improve data integrity … reduce escalations. Advancing AI-native analytics: Leverage Gen AI tools to automate recurring insight generation Enabling experimentation & continuous improvement: Support A / B testing and digital experience evaluations Influencing strategy through storytelling: Communicate insights and trends clearly and persuasively to technical ...

optimisation Partner with Product Managers to conceptualise, prioritise, test, and launch new products and features based on data insights Analyse A / B and multivariate tests to measure the impact of new features, content, and campaigns on user acquisition, engagement, and retention Support digital analytics … beneficial Experience working along side Product Engineering teams Strategic thinker with a strong grasp of experimentation principles, including A / B testing and experimental design methodologies Excellent prioritisation and time management skills; comfortable working to deadlines and managing multiple tasks Confident ...

doing:* Leading analysis across product and customer journeys* Driving insight to improve conversion and user experience* Supporting experimentation and A / B testing initiatives* Working closely with Product, Engineering and Data teams* Analysing customer behaviour, engagement and funnel performance* Contributing to personalisation … actionable recommendations What we're looking for: Strong SQL and Python skills Experience in product analytics, experimentation or digital analytics Strong A / B testing and funnel analysis experience Commercial mindset with strong stakeholder skills Experience in fast-paced digital or consumer-focused ...

Design and deliver scalable reporting, dashboards, and data solutions Partner with cross-functional teams including Product, Engineering, and Business stakeholders Conduct A / B testing and experimentation to evaluate product and feature performance Build and maintain high-quality datasets for advanced analytics Translate … team What We're Looking For Proven experience in Data Analytics or a similar role Strong SQL skills and experience with Python / PySpark Experience with data visualisation tools (e.g. Tableau, Power BI) Hands-on experience with A / B testing ...

role is all about using data to drive smarter customer engagement. You’ll: Analyse campaign performance, retention, and lifecycle activity Lead A / B testing and prove commercial impact (ROI focus) Support the rollout of a central CRM strategy and best … real influence – you won’t just report on campaigns, you’ll shape them. Your Skills & Experience SQL - strong experience Campaign analysis, A / B testing, and retention focus Ability to tie insight to commercial outcomes Experience working with large-scale consumer data Strong ...

decisions. Technology is at our core. And innovation is everywhere. But our company is more than datasets, lines of code or A / B tests. We're the thrill of the first night in a new place. The excitement of the next morning. … talent in the team Foster a great culture that innovates, work together as a team, partner with other Booking.com teams / roles and celebrate unified success Respect the Booking.com values and use them as a guide to the way we work Technology, Craft ...

product: partner with your team to define how we quantify organic impact and control for incremental effects—including in a ‘zero‐click’ / LLM-driven world where clicks alone understate value; align on success metrics and controls, and report performance clearly to marketing stakeholders Optimise performance … country / program by running A / B tests and SEO split-tests and deep analysis into segments (RFM), lifetime values (LTVs), return on investment etc. Improve how your channel forecasts performance and inputs recommendations to the investment framework Own data pipelines by identifying ...

scalability and reliability. Integrate agents with enterprise systems and protocols (MCP servers, A2A protocol, internal APIs). Design and run experiments (A / B tests, multi-armed bandits, uplift models) to measure and optimize model and agent performance. Ensure observability and reliability of deployed agents … deployment). Hands-on experience with AI development frameworks (LangChain, Strands, Amazon Bedrock, AgentCore, or equivalent). Experience with experimentation frameworks (A / B testing, uplift modeling, multi-armed bandits, causal ML). Exposure to machine vision techniques (CNNs, transfer learning, embeddings ...

Product Analyst - £55,000 (Hybrid - London) Industry: Consumer Marketplace / Mobile App Department: Product & Insights We're looking for a Product Analyst to join a fast-growing consumer marketplace used by millions globally. You'll work closely with product managers, designers, and engineers to uncover … Analyse user behaviour and product performance across key journeys Build dashboards and self-serve insights for cross-functional teams Support A / B testing: design experiments, interpret results, recommend next steps Work with data and platform teams to ensure clean, scalable datasets Present ...

client is looking for a proactive and tech-savvy System Support Assistant to join our growing team. If you enjoy solving problems, helping people, and working with technology every day, this could be the perfect opportunity for you. In this role, you’ll be the friendly face behind … advantage Understanding of Google Analytics and implementing code though Google Tag Manager Understanding of different types of Testing including A / B and Multivariate Experience of using A / B testing tools is an advantage Good ...