- Resulted in 171% improvement of website’s CVR (conversion rate) using continuous A/B tests
- Visualized the pros and cons of the landing page’s design patterns using Heatmaps
About MTI Ltd.
MTI Ltd. provides more than 100 services in the fields of healthcare, music, e-books etc. In particular, services such as Luna Luna and mujic.jp are its most popular. Its business comes from paid subscriptions, registered paying members pay a few hundred yen per month (about a few dollar) . Luna Luna, a fertility-tracking site to aid conception, had the mission to increase its monthly subscriber rate, thus it decided to review its landing site design. In order to review the design, we conducted a user behavioral analysis, and then multiple design reviews using A/B testing. We produced several alternate designs, and compared them to the original base design.
Analyzing Users behavior via Heatmaps
The landing page is long, as it’s designed for smartphones. The elements are as follows:
- Introduction to Luna Luna
- “TRY” button
- The “Most Fertile Day” Column
- The “When Fertility Starts” Column
- “Did You Know?” Inquiries
- “Sign Up” Button
- Detailed description of Luna Luna Family
- “Sign Up” Button
Analyzing Usage at the Page Bottom
This map shows how far users scroll down the site. Usually, the number of people who read through gradually decreases.
In the base design, about 50% of people see the button that comes after the “Did You Know” section. This is a rather good result as these users are seeing more than half of the page. The number decreases drastically at the descriptive explanation section. Only about 20% of people read all the way to the last button.
Mr. Ohno of MTI Ltd. says, “For the analysis, it’s important to understand the facts without thinking too deeply.” He found out the following 2 things.
- About half of users see more than half of the page.
- Almost none read the bottom of the page.
Learning From the “Gaze Heatmaps” Area
By mapping the section where a user has stopped, we can see that the blue section is not seen often, and the red section is quite well-read by users.
In the base design page, the title is hardly seen. The most attractive section that users focus on is “When Fertility Starts” section. The other areas are seen thoroughly, thus the page is well-made.
- The Header is not seen.
- The most seen section is “When Fertility Starts.”
Learning From the Last-Clicked Page
This shows where a user clicks to move on to the next page.
There are many buttons in the base design, and the most-clicked button is “Signup” which comes after the “Did You Know” section. Comparing to the bottom area’s map, we see that about 50% of users read that area. The next most-clicked button is the bottom-most “Signup” button. Also, there is another button which is clicked often, “Transferring From the Luna Luna Service.” This indicates that users look for it until the end of the page and find it eventully.
- Users do not click the first button but the one that shows up after reading a certain amount of content.
- The last button, where only 10% of users land, is clicked by almost everyone who saw it.
- It might be important to make the Transfer button as easy-to-find as possible.
The result of design tests and A/B testing
Based on the result of these analyses, we created 4 new design plans.
- A – Base Design
- B – The buttons’ location and design are changed, and sections are swapped.
- Put “Signup” button on the top, where 100% of people look.
- Moved “Transfer” button to the top where it is more useful (transfers are also conversions), and made the button skeuomorphic.
- Moved the most-read section to the top, and added a button just below it.
- C – Very much the same as B, but the button at the top was eliminated.
- D – Made each section’s content more readable, and added buttons.
- Users may get confused if there are two buttons at the top, so we kept the “Signup” button.
- Replaced the visuals in the columns and made layout and title changes to make it easier to understand.
- Add “Signup” button in-between each section.
- E – Make buttons stand out more
- Changed button color to yellow.
- Rearranged the sections.
- Removed the site description at the bottom.
Effective Design: How and Why
In the heatmap for the base design, the “When Fertility Starts” section stands out in red. This does not necessarily indicate that this section was well-read. Rather, users had stopped scrolling because they are not sure what to do. Therefore, if one were to add a button just below the “When Fertility Starts” button, it won’t appear in red even if it moved to the above section.
It’s often assumed that a button is placed in the top section, it should be clicked often, but here we test the validity of that assumption. The difference between B and C is simply the presence of a button on the top. Looking at the gaze heatmap and final-click map, you can see that the top button is being ignored. In the final-click map, it can be seen that the skeuomorphic button design was very effective. Another important aspect is what information should be included in each section. Buttons placed between each section are not clicked that often. The button which was most clicked was the one in the most-read section. The result of the yellowed-colored button design was not good. Brightly-colored buttons makes you want to click, but many clicked it without thinking, and after being guided to the member registration, tended to go back to the original page.
To summarize what we’ve learned about effective sites using these heatmaps:
- Do not place buttons at the top.
- Place buttons close to where the most-read section is.
- Skeuomorphic, stand-out buttons (but in moderation; otherwise, can have a negative effect)
Plan C fared the best in A/B testing, and satisfied all these points. This change helped Luna Luna improve 171% over a period of a couple months, while revenue increased an additional 20 million yen (about 170 thousand dollar).
Detail is here.