You are here

Google's Weak Spot - Or Finding the Second Best Search Engine

I've been pondering lately on searches and search engines and in some respects I've gotten frustrated with Google lately. Sure, they come up with thousands or more results and the results are even fairly relevant to the words I typed in. But the results are very commercial, even without Google charging for the very top listings in the search. I believe, it is within this commercialism that the seed of destruction lies for Google, just as it has for previous top search engines. And the problem Google has is that it can't completely change those search results without destroying their basis of business.

"What's that you say? Google showing commercial results at the top will destroy it? How absurd!", you may be saying to yourself right now.

Based on my previous experience and usage of top search engines before Google, I believe that I see a pattern repeating that I've observed before. When the top results are primarily dominated by commercialized, search engine optimized sites, users begin to look for the second best search engine that's still returning 'Mom & Pop' type pages.

Before I go further, I should define what I mean by "commercial" listing. In this context, I am referring to a website that is trying to lure customers and has the ability to pay someone to optimize their page to obtain a better ranking in the search engine results. Unfortunately, I also use the word commercial to refer to businesses, which could include small businesses that are self publishing and not able to optimize their website. I will endeavor to be clear how I am using the term in the following paragraphs, in a vain attempt to provide clarity. :-)

Here in a nutshell is how I see search engines and the steps the good ones take. This does not necessarily cover small search engines that haven't received large exposure, I'm focusing on the big ones of the past and present, such as Web Crawler, Alta Vista, Yahoo!, Meta Crawler and now Google.

  1. First, the search engine starts as an R&D project somewhere, either a business, garage, college or equivalent. Someone has gotten tired of not finding what they want on current top search engines and decides to try and 'index the web and everything else'
  2. Next, there is some slight success in getting better searches. For convenience, the search engine is put on a website and a few friends/colleagues/etc are notified about the search engine
  3. The other people trying the search engine start spreading the word, "Hey, this isn't too bad, try this link" Word begins to spread that here's a new search engine that actually works. Word spreads further and smart web site maintainers find out about the new search engine and start submitting their sites.
  4. As more links are being submitted, by non-commercial sites, the search engine continues to get more visitors and word continues to spread. The search engine might even start to be listed as one of importance for site listings and driving traffic to web sites.
  5. Now, here's where I believe the fatal misstep occurs for search engines, yet, I also believe it is inevitable, if the search engine is going to continue to grow further. Large commercial sites, that pay someone to maintain the website, start optimizing their sites for a site listing in the search engine that is growing more popular. This continues to build data for the search engine, which improves search results. There is also an interest for paying to be in the top results, which helps to fund the search engine to grow further.
  6. As the search engine starts to bring in money, it continues to build up it's database of websites. However, the search results are starting to be skewed towards businesses that can afford to optimize their site for that search engine. However, there are still enough other results, that most people are satisfied with the results
  7. The search engine has now reached critical mass. Listings of all kind are still being added, but priority is given to the optimized sites. The search engine will likely take steps to prevent this, however, major changes could result in breaking all of the results returned and driving users away, which would lower the advertising revenue. Most likely, only minor tweaks will occur and little will change.
  8. Meanwhile, in an R&D location, somewhere else, a developer is getting dissatisifed with the search engine results and starts a project to "index the web"...

I think Google is not quite at the point of critical mass and I am impressed with how they are trying to index other types of knowledge. However, I wouldn't be suprised to see another search engine, of some type, begin to develop prominence in the next 2-4 years. And actually, I think one is already started, at least in the format that I think the next generation search engine will use. And what is this search engine, you might ask? Well, here's what it is, in my best educated, ancedotal guess.

Wikipedia

Wikipedia? But that's not a search engine, it's an on-line encyclopedia and not necessarily an accurate one either.

Yes, that's true, it is based on the idea of an encyclopedia, but I believe it's become more than that. An encyclopedia in the past has been a static, out of date document, concerning itself primarily with historical items. Wikipedia has the potential to be more than that because it has the potential to have so many authors.

And there's more. Do you remember reading those encyclopedia articles and at the end there would be a list of other relevant articles or books? With the web, the relevant articles can also be linked in. But here's the power of Wikipedia, it isn't limited to items that had happened by the time it went to editing, it can include items that are happening now, this minute, this very second.

For example, want to find out more about Hurricane Katrina and places where you could donate to help out? Here's a link to the article about Katrina and down in the External Links and Sources you'll find a list of charitable organizations. Or, heck look up charitable organization and find out the definition, plus see listings to other charitable organizations if you didn't like the ones listed for Katrina.

I believe that Wikipedia is bringing the next stage to search engines, which is providing a context to your search. It's providing the function that the old card catalog in the library used to provide. When you searched in a library, you didn't get a keyword and pull out all of the books that contained that keyword. You either looked in the subject area for related topics, look for an author or look in titles for possible matches. In other words, there was a context associated with your search, which helped narrow it down.

The Yellow Pages also use this idea. When you look under automotive or cars, you get listings for buying, repairing, parts, etc, in other words, businesses that fit a context of what you are looking for. You don't find a bunch of pages that have businesses with the name "Car" in it.

Now I also believe that there are smart people at Google who realize there are things that can be done to improve their searches, but I think in the end, unless they do a major reinvention of their concept of searching, they will fail. I don't think Google will go away, they have way too much potential in so many areas. But if they change how they organize their data and how search results are returned, they risk losing their customer base and their funding, which would have the same effect, only sooner, if they don't return results that are as good as another search engine.

But I kind of hope I'm wrong about Google, as I'm impressed with their idealism and enthusiasm. And Wikipedia may not become the next big search engine, but I think something with a similar focus will come around. And somewhere, right now, a developer is getting dissatisfied with their search results and coming up with an idea to "index the web"...

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer