We designed and launched websites for unions Prospect and Bectu in 2019, and have been delighted to support them since with monthly maintenance.
While users access two different websites on the frontend, it is managed with one log-in for administrators on the backend – pretty fancy, and efficient. The sites receive 60,000 visitors on average per month.
We do make little design tweaks as part of our rolling support approach, but since launch we hadn’t reviewed page performance or made any big design changes to the join page – perhaps the most important page on the site, at least to the union. After a recent conversation about how the unions could encourage new members to join, we discussed AB testing for the Join Page and how we might check if we’re serving potential new members the best and most enticing version possible.
AB Whaaat?
If you’re unfamiliar with A/B testing, the headline is that it’s a fun and effective way to test two very different versions of the same webpage or feature, and then select which one to keep in the long-term based on real usage statistics.
Ideally it starts with a bit of usability testing by speaking directly to users and finding out what they think, what is working, and what isn’t. This data leads to the design, implementation, and running a new version of a specific page or feature at the same time as the current version. Users are randomly split into two groups so that when any user visits the page, they’re either served the ‘older’ (A) version, or the newer (B) version.
A/B tests measure the relative performance and click-throughs of each design, to determine which version is the most effective at achieving the desired objective. 💅 Very clever.
The more effective version is determined the winner, with the prize being that it becomes the only version of the page. The loser 💤 is gracefully retired from the site, thanked for all its hard work and destined for the archive design files.
How do users join a union?
We started with one of my favourite activities: a round of usability testing. Users ranged from union reps in their workplace, to people that have only recently started to engage with unions after hearing about 🔥#hotstrikesummer. This an ideal range of testers because it represents the variety of people that typically land on the join page.
We captured user test findings in an Airtable so that we could share it with the client as the tests happened.
There were some insights that were pretty standard and expected from testing, such as the feelings of apathy created by photos of people (even when Prospect had offered a range of lovely faces) to [add another one in here]. More unexpectedly, the most consistent needs that emerged were for users who were less familiar with the union to get a deeper understanding of what the union could do for them, and evidence that the union had supported people in the past.
We fed these results back to our designer, who had already prepared for the work by going into a deep dive of union join pages. He was surprised to find that there are more bad than good versions out there (no judgement, if you’re from a union and want to talk about testing/refining your join page, just get in touch!)
The point of A/B testing is to test a drastically different version. Here’s the before and after, side by side:
Once our developer implemented the new version, we ran a 28-day test on a free trial with VWO to check which would perform better. The results were updated in real time which I loved, and found myself checking it every few days to see which was winning. Looking at the above, which one do you think performed better?
🎉Tada! It was the new version, with some very impressive stats: A 14% increase on clickthroughs to the join journey (i.e. filling out personal and payment details, which was beyond the scope of this testing). The Prospect team were really happy with the result, and with the amazing value investment of just five days to run the testing, do the research and design a new version, implement it, and test it.
We have some plans to run further testing, some of which Prospect will be able to run alone because we’ve built the site in a way that gives them a lot of autonomy over the backend.
Want to boost your site’s performance?
🚀 Do you have a page on your website that you think could be performing better, but you’re not quite sure where to start?
Get in touch with us and we’ll be happy to have a chat…and a quiet hope that you’ll be up for some A/B testing!