What happened to the "long tail" theory of commerce on the Internet?

More than 15 years ago, with Amazon.com already on the ascendant, all talk was about how the Internet promised to liberate commerce for both sellers and purchasers. There was no more prominent expression of that optimism than Chris Anderson’s notion of the long tail (“The Long Tail: Why the Future of Business Is Selling Less of More”, 2006, here). However, if you are a Kat reader under the age of 35, you may not even recognize the term.

Since the euphoria of that time, the attitude towards “the long tail” has become more measured, bordering on skeptical. As one observer has stated, we are actually in an era of the “short head” rather than the “long tail”. What is all this talk about anatomy about? Read on.

Consider the following graph. Lots of occurrences regarding a small number of items are likely to happen within the bounds of the head on the left, far fewer occurrences regarding a greater number of items at the tail sloping rightward.

As applied to products, the basic notion is that the Internet worked a fundamental change. In the bricks and mortar world, where physical shelves dominate, the limits of space meant that only a limited variety within a given product category could be offered for sale to the physical customer (the “head”). The name of the game was to try and get your product on the shelf: No shelf, no exposure, no sales.

However, product offerings on the Internet are not constrained by the limits of physical space. Thus, while, in the Internet world, there will continue to be an abundance of sales of the leaders within a product category, the consumer also now has more abundant opportunity to purchase lesser known items (located at the tail). As a result, it is argued, the sales of such niche products, which individually may each be in low demand, can collectively amount to a substantial quantity, provided that there is both adequate supply and are distribution channels to provide such products.

In this way, the Internet seemed to promise a greater number of product offerings, redounding to the benefit of both the supply and demand sides of the relationship. On the one hand, niche product offerings could flourish, while, on the other hand, purchasers could enjoy access to these niche products.

As reported, Anderson took notice of research by Professors Erik Brynjolfsson, Yu (Jeffrey) Hu, and Michael D. Smith They argued that, with respect to the book business, Amazon.com enjoyed substantial sales of less prominent, even obscure books, i.e., niche products located at the tail, which previously were not likely to be offered for sale in bricks and mortar stores.

The idea was that information technology had materially lowered the search costs for such products. E.g., gone was the need to visit numerous second-hand book stores (if they even exist in your area) in the hope that some obscure publishing pearl might be discovered. Search was now to be done in front of the computer screen, enabling more efficient results.

The problem was that when researchers looked for instances of the long tail in action, more and more they are becoming hard-pressed to find them. This Kat conducted his own search and could not find a critical mass of studies that seem to validate the notion. To the contrary, research seems to point to the opposite. A 2018 report summarized by Professor Serguei Netessine in “The Long Tail Theory, Debunked: We Stick with What We Know”, here, describes the situation as follows:

The big message is we didn’t really find any evidence of the long tail effect, and that goes contrary to the theory and contrary to a few studies that were done before us. We found that, if anything, you see more and more concentration of demand at the top.

Take movies, especially the streaming platforms as exemplified by Netflix. [Merpel notes that in the current lockdown environment, the only platform more important than Netflix is Zoom.] Yes, there are more and more movie offerings, so indeed, the number of niche offerings, i.e., the tail, is getting longer. However (at least in a non-COVID-19 world) the actual time that one devotes to watching movies has not really increased.

So, are viewers more likely to watch more of these obscure titles, i.e., the tails, or stay within the comfort zone of hits and blockbusters? Netessine opts for the latter—

…[W]hen people search for what to watch in this increasing product variety, they tend to gravitate much more towards hits. We attribute it to the fact that, first, it’s hard to search this huge product variety. Second, even if you rely on some kind of a recommendation system, which every company uses now, recommendation systems are pretty basic. They only recommend something that somebody else has already watched, so they’re not going to recommend to you niche movies all that much.

From this perspective, gravitating to niche offerings, at least to streamed movies, does not seem to be manifesting itself as the long tail theory predicted.

OK, but what about selling things? Nettesine focuses on the role of recommendation algorithms, whose task it is to get before the customer what she is presumably looking for. Here, as well, niche products may not fare as well as hoped.

This is because, as he explains, such algorithms rely (albeit, not exclusively) on previous purchases. But that is precisely the problem for a niche product: how do you get purchased in the first place? And, in the absence of that initial purchase, how can one fashion the algorithm to reach such niche products?

The upshot is that “[a]s consumers are deluged with a dazzling array of choices, they tend to stick to brands they know.” Seen in this way, the strong brand remains the enemy of the niche product. The theory of the long tail viewed the Internet as the platform for ameliorating this state of affairs. But it does not seem to have quite worked out that way.

The supply of niche products may well increase. However, the behavior dynamics of consumer demand does not appear to have changed in respect to the role of brands, particularly strong brands. This seems to be the case, even as we are no longer constrained by the physical limitations of shelf space. Short head or long tail?

By Neil Wilkof

Top picture by Hay Kranen, who has released it into the public domain.

Bottom picture by Peretz Partensky and is licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license.