Website testing tool flaw exposes experiments

Late last week, a vulnerability in a popular online testing tool was discovered that allows anyone to view what kind of content experiments the site is running — and included in the examples cited was popular TV news site

Optimizely is a service with thousands of clients that allows them to conduct a wide variety of tests involving things such as headline text and size, page layout and more on unsuspecting users. Each time a visitor arrives on a page that’s part of an Optimizely experiment, he or she may be served a variation the the site producers are testing based on a variety of factors (including random selection).

The flaw, which was first reported in VentureBeat, was discovered by a marketing consultant named John McLaughlin, who also developed a tool that lets anyone see what, exactly, any Optimizely client is testing on the site — as well as certain future or past experiments.

One of the examples cited in the VentureBeat article was, which, according to McLaughlin’s tool, is or has been running a variety of tests using Optimizely, including ones appearing to involve the “Watch CNN” box, automatically starting video and more (as well as some innocently named “tests” and “demos”).

You can see a full list of the results of the tool here.

The tests give some interesting insight into what elements CNN considered, at least at one point, a priority on its site — including the live video-only feed of the cable channel and automatically planing video.

Optimizely has issued a statement reinforcing that this issue is not affecting data integrity or security of accounts, but is also planning to roll out a fix for the issue “soon.”

Once this fix is rolled out, it’s likely McLaughlin’s tool will stop working.