Because it isn't there

I've commented before on the sense of taking news articles out of Google's index (there isn't any)...but since Murdoch is planning (or, according to some commentators, is pretending to plan) to take News Corp's content off Google by some time next year the idea is worth another look.

The problem with trying to re-silo content at tis stage in the game is threefold.

First, online consumers do not have a problem finding content or finding specific content. In fact, that's the opposite of the problem we face. Content is dyfunctionally abundant. There is so much of it that the problem is filtering through the billions of pages/articles/songs/photos/films/games/posts/tweets for something useful, entertaining or relevant. Take some of the content away and unless it is actually unique you've made searchers' lives a little bit easier (thanks!).

Second, and relatedly, if I'm looking for commentary on the latest NYTco quarterlies and some of the potentially useful information doesn't show up in my search because it has been deliberately withheld from the engine, I'll never know. People resopond to incentives. Hiding content from me when I search doesn't incentivise me to try any other particular alternative behaviour (I'm going to realise that the article which would have answered all my questions is on the New York Post website but since Google doesn't know about it I have to surf over there and look for it? Just how am I going to know that?).

Third, content in a silo will attract no new readers. Two years ago I didn't read Marginal Revolution, but I found it and now I do. Ditto Unqualified Reservations. Ditto...well, everything in my feedreader. It's an interesting idea to run a paid content business that no-one can find. I wouldn't fancy it myself. Newspapers have enough demographic problems without shutting themselves off to the main potential source of new readers.

Mike Arrington theorises that the end-game is for content-providers to cut deals with other (non-Google) search engines, leaving Google high and dry but the content catalogued and searchable somewhere. Fair enough. Assuming that different providers cut deals with different engines (otherwise it's not a market and we've just swapped the Googlopoly for the Bingopoly, whoop-de-doo) we'd be looking at a potentially massive inconvenience for users with no upside benefits at all as they jumped from engine to engine searching for different content. Happily such a deliberate fragmentation of search ignores the existence of adequate search aggregation tools that pre-emptively obviate the obstacles. Data that can be usefully brought together online generally is, because it can be, because that's how it's most useful (hence indeed, err, Google). Trying at this stage in the game to fragment it all again is no solution, it's just another cycle around while someone knocks up a better aggregator.

Digital channels fragment media. They fragmented music, so now we find songs by the track on iTunes and Spotify instead of buying albums. But where the music industry continues to enjoy inherent commercial upside from its fragmented content (iTunes cost money, Spotify has unskippable ads or subs to pay, ringtones are a goldmine) text content has no such inherent commercial upside as a distributed fragment. Read an article on a newspaper website and you see an ad. Read the same content on Google or in Bloglines and you don't. The problem to solve here is not to stop people finding the stuff and reading it on Google - it's to make money when they see it there, to embed some inherent commercial upside in that experience. RSS banners, universal IntelliTXT, anything that follows the content around and makes money from it wherever it's seen. Hell, even movies manage it with product placement and one day they'll work out how to charge extra for all the millions of times those placements are seen by BitTorrenters.

But take the content off Google and I no longer know it exists, and I'm not going to use three or four different search engines to check every time I have a query. Hide it and it simply isn't there.

Comments

[Noah Brier]

I buy argument one and two, but I'm not sure about three. Marginal Revolution is a great example: Did you find it via search? I know I didn't. Ditto most of the stuff in my feedreader. The vast majority of my time these days goes to reading the stuff that other people are linking to (which, of course, is the stuff that other people are linking to). Now at some point someone needs to find the thing ... Maybe. I mean with the WSJ, it is pretty well known and for the immediate future they can probably rely on direct and linked in traffic to survive. Of course this leaves them in a bit of a pickle when no one knows who they are ... But then again, that's the same argument that was made about using pay walls. Anyway, am making a few roundabout points here. Good stuff, thanks for the post.

[libhomo]

Is Murdoch really going to make Google News more usable by clearing out all of his junk propaganda? That would be the first good thing Murdoch ever did for the web.

[Savi Vila]

Just one thing: How many times appears the same notice or "new" in one Google search? thousands from different sources. At the contrary: How traffic Google search atract to one specific site? Near 20%. Who lose? Rupert.

[Jeffrey Horn]

I'm not sure a new -opoly is inevitable. If another search engine snaps up news, say Yahoo! or whatever, then I'll use said search engine for news and continue to use Google for everything else.

That's why I use Google Scholar for finding articles, Alpha for solving algebra problems, and ICPSR to find public-use data. They serve different markets.

The discerning user doesn't need everything in one place, and I doubt indexing news would topple the arguable monopoly Google has on search.