1 to 25 of 108 A/B Testing Jobs

Requirements / h3 ul li Bachelor's, Master’s, or PhD or equivalent experience in Computer Science, Machine Learning, Applied Mathematics, Statistics, Financial Engineering, or a related quantitative field / li , li Significant professional experience (typically 7–12+ years) in AI / ML product development … / management, model validation, quantitative research, risk modelling, or related areas / li , li Demonstrated success working with technical teams or senior specialists in high stakes modelling environments / li , li Deep understanding of AI / ML systems including LLMs, agentic architectures, RAG pipelines, credit ...

Core Focus) Own the design and evolution of evaluation frameworks for LLM-based systems (both offline and online) Build and scale A / B testing to measure real customer impact and guide product decisions Define meaningful metrics that connect system performance to customer … clarity to ambiguous problem spaces and turn them into actionable changes to the system Cross-functional Collaboration Work daily with Product, Engineering, and Content / Design to shape the assistant’s behaviour Influence decisions about the product through clear communication, data visualisations, and well-structured impactful proposals. Ensure ...

manual effort and support scalable, end-to-end delivery. You will collaborate with cross-functional teams to validate use cases via A / B testing and ensure alignment with customer and business outcomes. You will host regular interlock sessions to review defects, share … delivery, or proactive service enablement. You will have experience leading automation and data driven fault prevention initiatives, including the use of A / B testing to validate improvements. You will have extensive experience and deep knowledge in Fixed and Mobile Network Design, Capacity ...

Responsibilities Own the full website user experience, ensuring journeys are intuitive, frictionless, and aligned to customer needs. Lead CRO activity including A / B testing, user testing, heatmaps, and behavioural analysis. Translate insights into clear recommendations that improve conversion and engagement. … teams and influencing decision-making. Manage the backlog of website improvements and prioritise based on impact. Monitor performance metrics and report on key UX / CRO outcomes. Act as the internal voice of the customer, ensuring digital decisions are insight-led. About You This role suits someone ambitious, proactive ...

from concept through to launch Work with Shopify Liquid, APIs and third-party integrations Develop responsive web experiences using JavaScript, jQuery, HTML and CSS / SCSS Create intuitive and user-friendly backend environments for website administrators Utilise AI development tools to support theme creation and development workflows Support data … migration and synchronisation between Shopify and external platforms Implement website performance optimisation and best practice development standards Assist with A / B testing, SEO implementation and data-driven improvements Collaborate across multiple projects while managing priorities effectively Troubleshoot technical issues and contribute innovative ...

Support evaluation of downstream applications (including GenAI-powered features) where they depend on retrieval quality. Experimentation & Reporting Help design and lead A / B testing and experimentation frameworks for search and ranking improvements. Partner with product and data science to define success metrics … Tableau, Power BI, Looker, matplotlib, seaborn) Experience working with Databricks or similar large-scale data platforms Excellent understanding of experimentation design, A / B testing, and statistical analysis Experience defining and analyzing search / retrieval metrics (e.g., NDCG, recall, precision, ranking metrics ...

e.g., NDCG, recall, precision) LLM and RAG evaluation methodologies Grounding and faithfulness evaluation Human evaluation and annotation strategies Online experimentation and A / B testing Establish best practices for offline benchmarking, online experimentation, and reproducible evaluation workflows. Build scalable processes for benchmark creation … systems. Information retrieval and semantic search Retrieval-augmented generation (RAG) Evaluation methodologies for IR and generative AI systems Experimentation frameworks and A / B testing Strong experience with: Vector retrieval and hybrid search architectures. LLM evaluation and AI quality measurement Embeddings, reranking ...

Data Analyst Location: Manchester (Hybrid: 1 day per week onsite, typically Wednesdays) Duration: August 2026 - April 2027 (Maternity Cover) Rate: £540 / day / £72 / hour (Inside IR35 / Paid via an Umbrella Company) About the Business We are a global leader … cleansing, validation, aggregation, and reporting of complex datasets to create scalable, long-term analytics solutions. Experimentation & Insight: Design, execute, and analyse A / B (and multivariate) tests to measure the success of new product features and business initiatives. Stakeholder Influence: Autonomously partner with cross-functional ...

unique opportunity to transform reporting, build behavioural analytics, and work across the whole ecosystem lifecycle. This is a hybrid role (2 / 3 days week) for 13 months with potential to extend. Responsibilities Building robust data foundations: Collaborate with engineering to develop pipelines, improve data integrity … reduce escalations. Advancing AI-native analytics: Leverage Gen AI tools to automate recurring insight generation Enabling experimentation & continuous improvement: Support A / B testing and digital experience evaluations Influencing strategy through storytelling: Communicate insights and trends clearly and persuasively to technical ...

decisions. Technology is at our core and innovation is everywhere. But our company is more than datasets, lines of code or A / B tests. We're the thrill of the first night in a new place. The excitement of the following morning. … directly related to the Trip Helpfulness aspect of the Connected Trip. Key Job Responsibilities and Duties: Owns and sets the vision for their product / feature, informed by data, user research and overarching vision of their product area Lays out a 1y strategy to achieve the vision ...

conversion rates, average order value, customer retention, and online revenue. Identify and remove friction points across the customer journey. Lead A / B testing and optimisation programmes. Analyse customer behaviour and implement enhancements based on data-driven insights. Analytics & Reporting Monitor and report … heatmapping, customer journey analysis, and testing platforms. Desirable Experience with Shopify, Magento, WooCommerce, or similar e-commerce platforms. Knowledge of SEO, UX / UI, and digital marketing channels. Experience with A / B testing and customer insight tools. Understanding ...

Design and deliver scalable reporting, dashboards, and data solutions Partner with cross-functional teams including Product, Engineering, and Business stakeholders Conduct A / B testing and experimentation to evaluate product and feature performance Build and maintain high-quality datasets for advanced analytics Translate … team What We're Looking For Proven experience in Data Analytics or a similar role Strong SQL skills and experience with Python / PySpark Experience with data visualisation tools (e.g. Tableau, Power BI) Hands-on experience with A / B testing ...

decisions. Technology is at our core. And innovation is everywhere. But our company is more than datasets, lines of code or A / B tests. We're the thrill of the first night in a new place. The excitement of the next morning. … talent in the team Foster a great culture that innovates, work together as a team, partner with other Booking.com teams / roles and celebrate unified success Respect the Booking.com values and use them as a guide to the way we work Technology, Craft ...

. What you'll be doing Own the experimentation roadmap and CRO strategy across the full customer journey Design and run A / B tests across email, website, and direct marketing channels Identify customer pain points and define optimisation opportunities Work with UX, engineering … continuous optimisation Drive a culture of data-driven experimentation across the business What you bring Must-haves Strong experience working in CRO / experimentation / optimisation roles Proven experience owning an experimentation roadmap (not just contributing) Hands-on experience running A / B ...

product: partner with your team to define how we quantify organic impact and control for incremental effects—including in a ‘zero‐click’ / LLM-driven world where clicks alone understate value; align on success metrics and controls, and report performance clearly to marketing stakeholders Optimise performance … country / program by running A / B tests and SEO split-tests and deep analysis into segments (RFM), lifetime values (LTVs), return on investment etc. Improve how your channel forecasts performance and inputs recommendations to the investment framework Own data pipelines by identifying ...

Product Analyst - £55,000 (Hybrid - London) Industry: Consumer Marketplace / Mobile App Department: Product & Insights We're looking for a Product Analyst to join a fast-growing consumer marketplace used by millions globally. You'll work closely with product managers, designers, and engineers to uncover … Analyse user behaviour and product performance across key journeys Build dashboards and self-serve insights for cross-functional teams Support A / B testing: design experiments, interpret results, recommend next steps Work with data and platform teams to ensure clean, scalable datasets Present ...

driven testing and optimisation. What you'll be doing: Building testing strategies and hypotheses Designing and analysing A / B and multivariate tests Delivering insights and recommendations to clients Running client meetings and stakeholder workshops Producing reports, presentations and optimisation roadmaps … What we're looking for: 4+ years' experience in CRO, experimentation or digital analytics Strong A / B testing and optimisation experience Excellent stakeholder management and communication skills Experience with tools such as Webtrends Optimize, VWO, Optimizely, AB Tasty or similar Passion ...

weight UI / UX Designer | Remote (UK ) | £55,000 - £60,000 About the Company We're partnering with a fast-growing, profitable student platform that's just closed a £7m Series A and is gearing up for a US launch. Think … micromanagement . The Role This is a new seat at the table. There has never been a dedicated UI / UX designer in the business, and the opportunity is wide open - both products need you. You'll own the end-to-end design process across ...

execution across multiple digital platforms. The Role Support the build-out of an experimentation centre of excellence Translate business OKRs into A / B testing roadmaps Own test prioritisation, ideation and execution Run experiments end-to-end (design launch analysis insight) Work closely … hands-on role - you'll need to think strategically while still being able to execute and deliver. About You Proven experience running experimentation / CRO programmes Hands-on experience building and analysing A / B tests Strong understanding of experimentation principles & metrics Experience with ...

your channel drives and how it is measured and controlled for incremental effects; report on actual performance to marketing stakeholders Optimise performance per country / program by running A / B tests and deep analysis into segments (RFM), lifetime values (LTVs), return on investment … background working in either product analytics and have ideally worked on invite or referral programs before You have advanced SQL skills; Python / R skills are advantageous; familiar with data pipelines (we use dbt) You’ve successfully set up and run A / B ...

high-impact opportunities in the onboarding flow. Partner with the team to prioritise projects with the highest leverage. Lead experimentation and A / B tests for business onboarding, defining hypotheses and metrics. Analyze results to drive product roadmap recommendations to reduce friction and boost conversion. … environment. You identify high-impact opportunities and make them happen. You triage requests by impact and know when to go deep versus deliver 80 / 20 results. Communication: Excellent verbal and written communicator - you cut through the noise, communicating what matters in a way that people ...

identify and advocate for better alternatives Ensure accurate, impactful, fast decision making with high quality analysis, raising the bar for a / b testing and helping us to iterate faster. Build automated data pipelines and dashboards to minimise repetitive tasks and allow data … write clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT / Airflow Experimentation & Statistics: Experience running A / B tests and analysing the results to drive product decisions Ownership ...

across platforms. Evaluation & Experimentation Leadership Define and operationalise evaluation frameworks for search and generative AI systems, including IR metrics (NDCG, recall, precision) and LLM / RAG evaluation methodologies. Develop grounding and faithfulness evaluation, human evaluation, and annotation strategies. Design online experimentation and A / B … search relevance and ranking systems, semantic search, retrieval‐augmented generation, evaluation methodologies for IR and generative AI, experimentation frameworks and A / B testing. Strong experience with vector retrieval and hybrid search architectures, LLM evaluation and AI quality measurement, embeddings, reranking, and retrieval orchestration. Experience ...

part in: Assessing and implementing robust MLOps framework, governance and best practices as per industry standards Designing and delivering end-to-end AI / ML systems, from data preparation and model development to model deployment, feature stores, model management and monitoring model management Delivering solutions using the latest GenAI … contribute to the business and your own personal growth, through activities that form part of the following categories: Business Development - Build client-ready demos / POVs, support proposals and technical deep-dives, and showcase delivery patterns. Internal contribution - Build reusable assets and frameworks that accelerate delivery across accounts, shape ...

surface insights on where we can win (market selection, segment focus, product areas) Be a strategic partner on competitiveness: proactively translate pricing / competitor signals into clear, actionable insights tailored to the needs of Product, Growth, Pricing and Marketing, and influence prioritisation and follow-through … insights are adopted and drive measurable impact Contribute to, own, create and track OKRs for the Competitive Intelligence / Comparison mission, keeping teams accountable through clear targets, progress tracking, and decision-ready reporting. Own measurement & attribution for comparison-led acquisition: define how we quantify incremental impact from comparison insights ...