This is sorta related but since I concluded the test today and this post is here, I thought I'd share.
I ran a 5 way split test for 9 days on my newsletter's signup page (JavaScript Weekly). My original page was the 2nd best performing but an identical page just without the subscriber count got a 8% higher conversion (or about 20% more signups in all) with a 90% confidence at the end of testing.
The worst performer? A signup page with no screenshot preview of the newsletter. Sent conversions from about 37% down to a mere 3% (!!) Lesson learned? Always have visuals or screenshots on pages where you're trying to get people to sign up for things they aren't sure about.
I'm not sure how tongue in cheek that was ;-) but 90% confidence was not a goal, at least. I just got bored after a week and wanted to move on. C'est la vie..
I ran a 5 way split test for 9 days on my newsletter's signup page (JavaScript Weekly). My original page was the 2nd best performing but an identical page just without the subscriber count got a 8% higher conversion (or about 20% more signups in all) with a 90% confidence at the end of testing.
The worst performer? A signup page with no screenshot preview of the newsletter. Sent conversions from about 37% down to a mere 3% (!!) Lesson learned? Always have visuals or screenshots on pages where you're trying to get people to sign up for things they aren't sure about.