User Heat Blog

UserHeat: Free Heatmap Analytics Tool

The “Luna Luna Case,” a case study in massively improving CV (conversion) with Mobile Heatmaps and A/B Testing

MTI Base Design

Case Summary

  • Resulted in 171% improvement of website’s CVR (conversion rate) using continuous A/B tests
  • Visualized the pros and cons of the landing page’s design patterns using Heatmaps

About MTI Ltd.

MTI Ltd. provides more than 100 services in the fields of healthcare, music, e-books etc. In particular, services such as Luna Luna and mujic.jp are its most popular. Its business comes from paid subscriptions, registered paying members pay a few hundred yen per month (about a few dollar) . Luna Luna, a fertility-tracking site to aid conception, had the mission to increase its monthly subscriber rate, thus it decided to review its landing site design. In order to review the design, we conducted a user behavioral analysis, and then multiple design reviews using A/B testing. We produced several alternate designs, and compared them to the original base design.


Analyzing Users behavior via Heatmaps

The landing page is long, as it’s designed for smartphones. The elements are as follows:

  • Header
  • Title
  • Introduction to Luna Luna
  • “TRY” button
  • The “Most Fertile Day” Column
  • The “When Fertility Starts” Column
  • “Did You Know?” Inquiries
  • “Sign Up” Button
  • Detailed description of Luna Luna Family
  • “Sign Up” Button
  • Footer

Analyzing Usage at the Page Bottom

This map shows how far users scroll down the site. Usually, the number of people who read through gradually decreases.

In the base design, about 50% of people see the button that comes after the “Did You Know” section. This is a rather good result as these users are seeing more than half of the page. The number decreases drastically at the descriptive explanation section. Only about 20% of people read all the way to the last button.

Mr. Ohno of MTI Ltd. says, “For the analysis, it’s important to understand the facts without thinking too deeply.” He found out the following 2 things.

  • About half of users see more than half of the page.
  • Almost none read the bottom of the page.

Learning From the “Gaze Heatmaps” Area

By mapping the section where a user has stopped, we can see that the blue section is not seen often, and the red section is quite well-read by users.

In the base design page, the title is hardly seen. The most attractive section that users focus on is “When Fertility Starts” section. The other areas are seen thoroughly, thus the page is well-made.

  • The Header is not seen.
  • The most seen section is “When Fertility Starts.”

Learning From the Last-Clicked Page

This shows where a user clicks to move on to the next page.

There are many buttons in the base design, and the most-clicked button is “Signup” which comes after the “Did You Know” section. Comparing to the bottom area’s map, we see that about 50% of users read that area. The next most-clicked button is the bottom-most “Signup” button. Also, there is another button which is clicked often, “Transferring From the Luna Luna Service.” This indicates that users look for it until the end of the page and find it eventully.

  • Users do not click the first button but the one that shows up after reading a certain amount of content.
  • The last button, where only 10% of users land, is clicked by almost everyone who saw it.
  • It might be important to make the Transfer button as easy-to-find as possible.

The result of design tests and A/B testing

Based on the result of these analyses, we created 4 new design plans.

MTI Design Plan

  • A – Base Design
  • B – The buttons’ location and design are changed, and sections are swapped.
    • Put “Signup” button on the top, where 100% of people look.
    • Moved “Transfer” button to the top where it is more useful (transfers are also conversions), and made the button skeuomorphic.
    • Moved the most-read section to the top, and added a button just below it.
  • C – Very much the same as B, but the button at the top was eliminated.
  • D – Made each section’s content more readable, and added buttons.
    • Users may get confused if there are two buttons at the top, so we kept the “Signup” button.
    • Replaced the visuals in the columns and made layout and title changes to make it easier to understand.
    • Add “Signup” button in-between each section.
  • E – Make buttons stand out more
    • Changed button color to yellow.
    • Rearranged the sections.
    • Removed the site description at the bottom.

MTI Test Result

Effective Design: How and Why

In the heatmap for the base design, the “When Fertility Starts” section stands out in red. This does not necessarily indicate that this section was well-read. Rather, users had stopped scrolling because they are not sure what to do. Therefore, if one were to add a button just below the “When Fertility Starts” button, it won’t appear in red even if it moved to the above section.
mti_abtest_result

It’s often assumed that a button is placed in the top section, it should be clicked often, but here we test the validity of that assumption. The difference between B and C is simply the presence of a button on the top. Looking at the gaze heatmap and final-click map, you can see that the top button is being ignored. In the final-click map, it can be seen that the skeuomorphic button design was very effective. Another important aspect is what information should be included in each section. Buttons placed between each section are not clicked that often. The button which was most clicked was the one in the most-read section. The result of the yellowed-colored button design was not good. Brightly-colored buttons makes you want to click, but many clicked it without thinking, and after being guided to the member registration, tended to go back to the original page.

mti_gaze_click

To summarize what we’ve learned about effective sites using these heatmaps:

  • Do not place buttons at the top.
  • Place buttons close to where the most-read section is.
  • Skeuomorphic, stand-out buttons (but in moderation; otherwise, can have a negative effect)

Plan C fared the best in A/B testing, and satisfied all these points. This change helped Luna Luna improve 171% over a period of a couple months, while revenue increased an additional 20 million yen (about 170 thousand dollar).

Detail is here.

Share on FacebookTweet about this on TwitterShare on LinkedInPin on PinterestShare on RedditShare on Tumblr