Key takeaways<\/h2>\n\n\n- A\/B testing transformed GoDaddy’s product development from opinion-based to data-driven decision-making<\/li>\n
- Failed experiments provide valuable learning opportunities and drive innovation<\/li>\n
- Experimentation culture promotes cross-functional collaboration and faster, more confident decision-making<\/li>\n<\/ul>\n<\/div>\n<\/div>\n
(Editor’s note: This post is the first in a four part series that discusses experimentation at GoDaddy. Check back for the next part in the series.)<\/em><\/p>\nA\/B testing wasn\u2019t always central to product development at GoDaddy. Like many teams, we relied on customer feedback, business priorities, and our best judgment. And while that worked to some extent, we knew we could be more data-driven.<\/p>\n
Over time,\u00a0we’ve built and embraced a culture of experimentation<\/a>\u00a0with A\/B testing being central to how we build products. At any given time,\u00a0GoDaddy teams are running hundreds of experiments across all customer touchpoints<\/a>. However, it\u2019s more than just a validation tool, it serves as a guiding framework for decision-making that shapes our internal product culture. This approach allows us to act faster, with greater confidence, while remaining aligned with our customers’ needs.<\/p>\nWhat’s A\/B testing?<\/h2>\nA\/B testing is a method of comparing two or more versions of a product, feature, or interface to determine which one performs better based on specific metrics. In a typical A\/B test, users are randomly assigned to different groups, with one group serving as the control group and others as experimental variants.<\/p>\n
\n![\"diagram\"](\"https:\/\/www.godaddy.com\/resources\/wp-content\/uploads\/2025\/03\/ab-testing.jpg?size=3840x0\"\/)
<\/figure>\n<\/div>\n\u00a0The diagram shows an overview of the A\/B testing approach<\/em><\/p>\nThis type of experiment is often referred to as a randomized controlled experimentation (RCE). The results are then analyzed using statistical methodologies to determine, with statistical significance, which version leads to better outcomes.<\/p>\n
A\/B testing is a popular RCE type, but not the only one. The following are common RCE types:<\/p>\n
\n- A\/B<\/strong>: comparing two versions (A vs. B). This is the most common type of test used for simple changes, such as a new button color or headline.<\/li>\n
- Multi-variant<\/strong>\u00a0(or A\/B\/n): this is an extension of A\/B testing where you compare three or more versions to see which one resonates most with the audience.<\/li>\n
- Multivariate<\/strong>: instead of testing just one element (like in A\/B testing), you test combinations of multiple changes at once to identify the most effective mix.<\/li>\n
- Champion\/Challenger<\/strong>: where a “Challenger” variant undergoes multiple refinements before being tested against the current “Champion”.<\/li>\n<\/ul>\n
While A\/B testing and its variations rely on randomization for precise comparisons, another approach is Pre\/Post\u00a0testing that measures performance before and after a change without a control group. Though more susceptible to external influences like seasonality, it can still provide valuable insights when randomization isn\u2019t possible.<\/p>\n
Beyond guesswork<\/h2>\nWhen we redesigned\u00a0GoDaddy\u2019s domain transfer flow<\/a>, we assumed simplifying the interface would improve success rates. While this did happen, we soon realized that external factors like seasonality, marketing campaigns, and macro trends made it difficult to isolate the true impact of our changes. Were the results due to our redesign, or something else? The following diagram shows A\/B testing on a simplified\u00a0hierarchy of evidence pyramid<\/a>:<\/p>\n\n![\"\"](\"https:\/\/www.godaddy.com\/resources\/wp-content\/uploads\/2025\/03\/simplified-pyramid.jpg?size=3840x0\"\/)
<\/figure>\n<\/div>\nThat\u2019s where A\/B testing made a difference. Unlike the pre\/post analysis we used before, it allowed us to directly compare two versions under the same conditions. A\/B testing helped:<\/p>\n
\n- give us\u00a0control<\/strong>\u00a0over external factors, so we could pinpoint exactly what was driving improvements.<\/li>\n
- provide more\u00a0confidence<\/strong>\u00a0in our results, so teams iterated faster, making data-driven adjustments to achieve the desired outcome.<\/li>\n
- us\u00a0prioritize<\/strong>\u00a0changes that delivered the greatest impact, ensuring we focused on the most valuable improvements first.<\/li>\n<\/ul>\n
Shifting to data-driven experimentation empowered our product engineering teams to make decisions that not only achieved our targets, but also delivered value to our customers and stakeholders.<\/p>\n
Maximizing learning opportunities<\/h2>\nOne of the biggest lessons we\u2019ve learned about A\/B testing is that statistically significant losses can be just as valuable as statistically significant wins.<\/p>\n
Some of our most valuable customer insights came from failed A\/B experiments. For example, our first iteration of a new\u00a0GoDaddy Auctions bidding flow<\/a>\u00a0aimed to help buyers win expired domains but had the opposite effect. Instead of a setback, the team saw it as a learning opportunity, we:<\/p>\n\n- reviewed results to understand the frictions within the flow.<\/li>\n
- collected additional feedback from surveys and analyzed session recordings.<\/li>\n
- brought together the team to brainstorm solutions based on the findings.<\/li>\n
- developed a new bidding flow experience addressing the identified frictions.<\/li>\n
- launched an improved version, leading to a winning iteration.<\/li>\n<\/ol>\n
This collaborative, data-driven approach not only uncovered the root cause of the issue but also led to an improved iteration that strengthened customer success.<\/p>\n
The following image shows two A\/B experiment iterations:\u00a0<\/p>\n\n![\"graphical](\"https:\/\/www.godaddy.com\/resources\/wp-content\/uploads\/2025\/03\/experiment-example.jpg?size=3840x0\"\/)
<\/figure>\n<\/div>\nA\/B testing is a powerful way to validate ideas based on customer actions. But it\u2019s just one piece of our customer empathy toolkit. To understand both the “what” and the “why”, GoDaddy teams are encouraged to combine A\/B testing with a range of\u00a0tools that offer deeper insights into customer behavior and feedback<\/a>. There’s no stronger motivator to use these tools than a losing experiment. In those moments, we gain the insight to rethink, adjust, and create something better.<\/p>\nThe Power of Velocity<\/h2>\nOne of the biggest benefits of A\/B testing is that it enables our teams to move faster with purpose. That\u2019s meaningful progress. Speed is about how fast you go, velocity is about moving in the right direction. You can be running at full speed, but if you\u2019re going in circles, you\u2019re not getting anywhere. In business, progress matters more than motion.<\/p>\n
The following image illustrates the difference between speed and velocity:\u00a0<\/p>\n\n![\"diagram\"](\"https:\/\/www.godaddy.com\/resources\/wp-content\/uploads\/2025\/03\/velocity.jpg?size=3840x0\"\/)
<\/figure>\n<\/div>\nBefore adopting A\/B experimentation, our product cycles were slower and sometimes disconnected from long-term strategy. We\u2019d spend months building and launching features, only to scramble afterward to analyze metrics. By the time we understood what didn\u2019t work, it was too late to adjust. We were already focused on the next project.<\/p>\n
But velocity isn\u2019t just about moving quickly, it\u2019s about learning in a way that compounds over time. As our CEO Aman noted, a\u00a0healthy experimentation culture balances positive, negative, and inconclusive outcomes<\/a>. If every experiment wins, we might be playing it too safe instead of pushing boundaries. The most impactful experiments don\u2019t just move metrics, they advance our understanding of the customer and shape product strategy. A scattered, one-off experiment might drive a short-term win, but experiments aligned with product themes and clearly defined customer problems lead to stronger product experiences.<\/p>\nCatalyst for Collaboration<\/h2>\nA\/B testing hasn\u2019t just helped us build better products. It transformed how teams collaborate at GoDaddy. Before we adopted a structured approach to experimentation, teams like product, design, engineering, business analytics, and customer success often worked in silos. The A\/B testing process itself has naturally become a unifying force. Each experiment is a cross-functional effort from the start, with everyone involved in defining the hypothesis, designing the test, monitoring results, discussing, iterating and sharing the learnings.<\/p>\n
GoDaddy aspires to build great products and experiences that solve real customer problems. Our gold standard are experiments that use randomization and robust statistical methods that empower teams to:<\/p>\n
- \n
- A\/B testing transformed GoDaddy’s product development from opinion-based to data-driven decision-making<\/li>\n
- Failed experiments provide valuable learning opportunities and drive innovation<\/li>\n
- Experimentation culture promotes cross-functional collaboration and faster, more confident decision-making<\/li>\n<\/ul>\n<\/div>\n<\/div>\n
(Editor’s note: This post is the first in a four part series that discusses experimentation at GoDaddy. Check back for the next part in the series.)<\/em><\/p>\n
A\/B testing wasn\u2019t always central to product development at GoDaddy. Like many teams, we relied on customer feedback, business priorities, and our best judgment. And while that worked to some extent, we knew we could be more data-driven.<\/p>\n
Over time,\u00a0we’ve built and embraced a culture of experimentation<\/a>\u00a0with A\/B testing being central to how we build products. At any given time,\u00a0GoDaddy teams are running hundreds of experiments across all customer touchpoints<\/a>. However, it\u2019s more than just a validation tool, it serves as a guiding framework for decision-making that shapes our internal product culture. This approach allows us to act faster, with greater confidence, while remaining aligned with our customers’ needs.<\/p>\n
What’s A\/B testing?<\/h2>\n
A\/B testing is a method of comparing two or more versions of a product, feature, or interface to determine which one performs better based on specific metrics. In a typical A\/B test, users are randomly assigned to different groups, with one group serving as the control group and others as experimental variants.<\/p>\n
\n<\/figure>\n<\/div>\n\u00a0The diagram shows an overview of the A\/B testing approach<\/em><\/p>\n
This type of experiment is often referred to as a randomized controlled experimentation (RCE). The results are then analyzed using statistical methodologies to determine, with statistical significance, which version leads to better outcomes.<\/p>\n
A\/B testing is a popular RCE type, but not the only one. The following are common RCE types:<\/p>\n
- \n
- A\/B<\/strong>: comparing two versions (A vs. B). This is the most common type of test used for simple changes, such as a new button color or headline.<\/li>\n
- Multi-variant<\/strong>\u00a0(or A\/B\/n): this is an extension of A\/B testing where you compare three or more versions to see which one resonates most with the audience.<\/li>\n
- Multivariate<\/strong>: instead of testing just one element (like in A\/B testing), you test combinations of multiple changes at once to identify the most effective mix.<\/li>\n
- Champion\/Challenger<\/strong>: where a “Challenger” variant undergoes multiple refinements before being tested against the current “Champion”.<\/li>\n<\/ul>\n
While A\/B testing and its variations rely on randomization for precise comparisons, another approach is Pre\/Post\u00a0testing that measures performance before and after a change without a control group. Though more susceptible to external influences like seasonality, it can still provide valuable insights when randomization isn\u2019t possible.<\/p>\n
Beyond guesswork<\/h2>\n
When we redesigned\u00a0GoDaddy\u2019s domain transfer flow<\/a>, we assumed simplifying the interface would improve success rates. While this did happen, we soon realized that external factors like seasonality, marketing campaigns, and macro trends made it difficult to isolate the true impact of our changes. Were the results due to our redesign, or something else? The following diagram shows A\/B testing on a simplified\u00a0hierarchy of evidence pyramid<\/a>:<\/p>\n
\n<\/figure>\n<\/div>\nThat\u2019s where A\/B testing made a difference. Unlike the pre\/post analysis we used before, it allowed us to directly compare two versions under the same conditions. A\/B testing helped:<\/p>\n
- \n
- give us\u00a0control<\/strong>\u00a0over external factors, so we could pinpoint exactly what was driving improvements.<\/li>\n
- provide more\u00a0confidence<\/strong>\u00a0in our results, so teams iterated faster, making data-driven adjustments to achieve the desired outcome.<\/li>\n
- us\u00a0prioritize<\/strong>\u00a0changes that delivered the greatest impact, ensuring we focused on the most valuable improvements first.<\/li>\n<\/ul>\n
Shifting to data-driven experimentation empowered our product engineering teams to make decisions that not only achieved our targets, but also delivered value to our customers and stakeholders.<\/p>\n
Maximizing learning opportunities<\/h2>\n
One of the biggest lessons we\u2019ve learned about A\/B testing is that statistically significant losses can be just as valuable as statistically significant wins.<\/p>\n
Some of our most valuable customer insights came from failed A\/B experiments. For example, our first iteration of a new\u00a0GoDaddy Auctions bidding flow<\/a>\u00a0aimed to help buyers win expired domains but had the opposite effect. Instead of a setback, the team saw it as a learning opportunity, we:<\/p>\n
- \n
- reviewed results to understand the frictions within the flow.<\/li>\n
- collected additional feedback from surveys and analyzed session recordings.<\/li>\n
- brought together the team to brainstorm solutions based on the findings.<\/li>\n
- developed a new bidding flow experience addressing the identified frictions.<\/li>\n
- launched an improved version, leading to a winning iteration.<\/li>\n<\/ol>\n
This collaborative, data-driven approach not only uncovered the root cause of the issue but also led to an improved iteration that strengthened customer success.<\/p>\n
The following image shows two A\/B experiment iterations:\u00a0<\/p>\n
\n<\/figure>\n<\/div>\nA\/B testing is a powerful way to validate ideas based on customer actions. But it\u2019s just one piece of our customer empathy toolkit. To understand both the “what” and the “why”, GoDaddy teams are encouraged to combine A\/B testing with a range of\u00a0tools that offer deeper insights into customer behavior and feedback<\/a>. There’s no stronger motivator to use these tools than a losing experiment. In those moments, we gain the insight to rethink, adjust, and create something better.<\/p>\n
The Power of Velocity<\/h2>\n
One of the biggest benefits of A\/B testing is that it enables our teams to move faster with purpose. That\u2019s meaningful progress. Speed is about how fast you go, velocity is about moving in the right direction. You can be running at full speed, but if you\u2019re going in circles, you\u2019re not getting anywhere. In business, progress matters more than motion.<\/p>\n
The following image illustrates the difference between speed and velocity:\u00a0<\/p>\n
\n<\/figure>\n<\/div>\nBefore adopting A\/B experimentation, our product cycles were slower and sometimes disconnected from long-term strategy. We\u2019d spend months building and launching features, only to scramble afterward to analyze metrics. By the time we understood what didn\u2019t work, it was too late to adjust. We were already focused on the next project.<\/p>\n
But velocity isn\u2019t just about moving quickly, it\u2019s about learning in a way that compounds over time. As our CEO Aman noted, a\u00a0healthy experimentation culture balances positive, negative, and inconclusive outcomes<\/a>. If every experiment wins, we might be playing it too safe instead of pushing boundaries. The most impactful experiments don\u2019t just move metrics, they advance our understanding of the customer and shape product strategy. A scattered, one-off experiment might drive a short-term win, but experiments aligned with product themes and clearly defined customer problems lead to stronger product experiences.<\/p>\n
Catalyst for Collaboration<\/h2>\n
A\/B testing hasn\u2019t just helped us build better products. It transformed how teams collaborate at GoDaddy. Before we adopted a structured approach to experimentation, teams like product, design, engineering, business analytics, and customer success often worked in silos. The A\/B testing process itself has naturally become a unifying force. Each experiment is a cross-functional effort from the start, with everyone involved in defining the hypothesis, designing the test, monitoring results, discussing, iterating and sharing the learnings.<\/p>\n
GoDaddy aspires to build great products and experiences that solve real customer problems. Our gold standard are experiments that use randomization and robust statistical methods that empower teams to:<\/p>\n
- provide more\u00a0confidence<\/strong>\u00a0in our results, so teams iterated faster, making data-driven adjustments to achieve the desired outcome.<\/li>\n
- Multi-variant<\/strong>\u00a0(or A\/B\/n): this is an extension of A\/B testing where you compare three or more versions to see which one resonates most with the audience.<\/li>\n
- A\/B<\/strong>: comparing two versions (A vs. B). This is the most common type of test used for simple changes, such as a new button color or headline.<\/li>\n