Glenn Fleishman tweeted a link to the read-it-later service formerly known as Readability and now known as Pocket, which has posted an article on its blog, claiming that Pocket helps to extend the ‘life span’ of articles which it saves.
It provides some graphs to this effect, seeming to show that Pocket’s activity curve is shallower. All well and good.
Except it’s not. The x-axis of the graph is labelled simply ‘activity.’ No units. There is an asterisk stating that the two curves are on separate scales—naturally, given that BuzzFeed gets much, much more traffic than Pocket—but there’s no indication as to what these scales are, and therefore it’s impossible to check Pocket’s maths and validate their reasoning.
This is disappointing. Pocket is a well-engineered read-it-later service, and one of the best: it is allegedly run by adults working in computing, who should therefore have at least some grasp of statistics. Unless they consider the number of ‘opens’ something has in Pocket to be a trade secret, there is no excuse not to release this data when using a graph to advertise your product. Otherwise, your graph has about as much validity as this one, discussing the lifespan of statistical guff on blogs in terms of watermelons:
I don’t think Pocket set out to deceive the publishers their blog post is aimed at, but this sort of cowboy statistics sets a dangerous precedent.