I've been thinking a bit about conversions and A/B tests lately. I've recently worked with a client on helping them pick imagery for their new website. In the end, we ended up with two different hero images that we all seemed to agree aligned with the company's values and what we think will speak to potential customers.
The decision was then to A/B tests the two images to see which one converts better. While this is great, I'm wondering how one could create a third option. In order to create an A/B test, you need to run an additional script on your site (so 50% sees images A and 50% sees image B). That script - like any script - adds additional loading time. Ideally I'd like to A/B test two images, but I'd also like to A/B test two versions - one with the script and one without the script.
To understand how much time scripts add to a loading time, try visiting USA Today's new "European Union Expierence" (e.g. without scripts and tracking).
Booking.com A/B tests everything. If something cannot be A/B tested, Booking.com won't do it. There's more than 1,000 A/B tests running at any time.Living a Testing Culture