Problem
- The business want to know whether the eCommerce platform is performing sluggishly and if that is impacting revenue
- They want to know whether Technology could be the culprit
- Anecdotal evidence suggests a user experience problem, but the infrastructure team say everything’s fine
Possible Solutions
The best way to find possible solutions is to first fully understand where the real problem actually lies... is the sluggishness reported by some users caused by something at their end (e.g. device types), or at our end (our infrastructure), or somewhere in between (ISPs/network), or even at one of our 3rd parties? Was it happening at particular times of day, or under specific conditions?
To get some factual data to support the anecdotal evidence, I turned to our monitoring tools. We had a number of them, covering the service from different angles: from infrastructure metrics, through application-level metrics, up to service-level metrics for real traffic and synthetic traffic.
Eggplant's RUM (Real User Monitoring) and PA (Performance Analyser) tooling enabled me to dig down into any variance in experience for different groups of users (e.g. device types, browser, OS, connection speed) at any time, and the impact on our revenue.
- RUM gave me the ability to see in real time the impact of any tech issues, and PA enabled us to dig into the detail of why the experience was as it was on each device profile.
- This enabled us to identify 25 performance enhancements to address the user experience on the worst performing devices,
- and to drive the required changes back to the website, which visibly improved experience and conversion.
Metrics
I was able to illustrate, using our own website's metrics, that faster page load times lead to higher conversion rates, as shown in the graph below. Note how iOS smartphones saw faster page load times than Android smartphones, and converted more... and that this same pattern appears for iOS and Android tablets.
Note how the small screen devices have a lower conversion rate than the large screen devices (desktops/laptops and tablets) – this is likely to be due to the different patterns of user behaviour (users are more likely to be researching the products they are considering buying while on their phones, but then choose to sit down with their tablet or laptop when they actually want to place their order).
The only way we really know whether faster load times lead to higher conversion rates would be to change the page load times and see whether this alters conversion rates.
Results
By making small improvements primarily focused at Android and tablet devices, we were able to see those devices move to the left and upwards, relative to the other device profiles.
Outcomes
Based on these small gains, we were able to drive further improvements to page load times.Knowledge Gained
Real data really helps. By using the right tools to instrument this eCommerce website, and finding clear, visual ways to present the data (turning it into information, knowledge and wisdom) I was able to show the business where the real problems lay, and where potential solutions could be focused, thereby driving real results for the business.
Comments