|
More About Search Engines
by Robert Young, Webmaster Phoenix5
How do search engines search?
How do they select or rank the pages?
Types of sites and two types again
A different type of search: via images
Other types of search engines: meta
Where to learn more
I am hardly an expert but I search the Net daily and this is based on what I've learned in the last couple of years.
Search engines do vary but this is basically what they are and how they work.
How do search engines search?
Contrary to popular belief, search engines don't "search the Web" when you insert a word or phrase. They search an index already created at their site.
An actual Web-search engine is built by their finding Web pages, copying them, filing them at their site and then those pages are indexed, waiting for you to feed in a word or phrase. When you do, it accesses the index that is now stored at their site and gives you the results. This is why they can work so fast.
The extent of a search by a search engine is thus proportional to the number of pages/sites that they have copied, saved and indexed. How it presents the information to you is based on their software amd varies between sites.
Given the size of the Web, storing pages can be, of course, staggering. Google, for example, says it is storing nearly 1.4 billion - yes, billion - Web pages at this writing. It will increase.
How do they select or rank the pages?
When you get the result of your search, the suggested pages are presented in a certain order. Do the same search at another site and they will come up in a different order. So how do they select the order? Well, the answer is complex, controversial and incomplete.
Some search engines go by the number of times that word appears on the page and/or how high the word is on the page. If the word is in the title of the document, it will count more than if it is down towards the bottom. If it appears 10 times, it is worth more than one time. Go figure, but it was the best that some could do at the time.
Others go by how many sites link to that page and/or how often it is accessed.
In short, each has their own criteria and not all are forthcoming as to what it is. (Google will flat out tell you at http://www.google.com/technology/index.html, which is another thing I like about them.)
So the page you may really want could be at the bottom of their list. That is why a good engine is really crucial, to allow you to modify your search and really narrow it down because you can't rely on the order the search engine gives you and you may not have time to go through 5,000 pages.
Types of sites and two types again
There are basically two types of search sites: search engines and directories. For each, they can be broken down further as generic (any topic) and specific (they specialize.)
For example, if you want to search for theater, dance and musical arts, then go to http://www.culturefinder.com. The "All Things Spiritual" Directory at http://www.allspiritual.com/ specializes in "spirituality on the Internet."
If you want to know if there is a search engine/directory for a particular topic, go to the guide to search engines that lets you look at
http://www.searchengineguide.com/searchengines.html. Just remember, they only guide you to search engines/directories and don't actual search the Web.
It has limits. Feeding "prostate cancer" to Search Engine Guide produces only one site, HealthLink
which turned out to be a broken link. Additionally, there are PCa sites that Search Engine Guide doesn't list, such as the resources at National Cancer Institute.
A different type of search: via images
There are now IMAGE search engines that allow you to search for an image. This can be an innovative way to find something because you come in via a visual. There is:
Ditto at http://ditto.com/ and
Google's at http://images.google.com/
Why would you search for an image? Well, try "prostate gland" in Google and you will find drawings, photos and slides. You will also get more junk as it is not as precise. Doing the same in Ditto produces hardly anything, by comparison.
Other types of search engines: meta
There are search engines that try to give a "human interface" where you type in a question. "Ask Jeeves" is one of the more popular ones at
http://www.ask.com/index.asp?
It is a comfortable way to learn to prowl the Web but it has limits. It also gets confused. Asking "What are the signs of prostate cancer?" offered me advice for the astrological sign of Cancer, as well as suggesting some sites more relevant to my request.
Ask Jeeves is an example of a META-search engine. It taps into other search engines. This sound good but don't expect the best service.
Ask Noah (for alternative medicine) is another at http://www.noah-health.org/english/alternative/alternative.html
Metacrawler at http://www.metacrawler.com is one of the better generic meta-search engines.
CNet launched their own at http://www.search.com/ which is getting good reviews.
Mamma at http://www.mamma.com/ leaves a lot untouched.
Dogpile at http://www.dogpile.com/ is equally lacking, in my opinion.
But these might well serve the person learning to search the Web as it can be a daunting task.
Where to learn more
PC World magazine has a one-page comparison of search engines at their Web site that might prove valuable. It is at
http://www.pcworld.com/resource/article/0,aid,17689,pg,7,00.asp
Search Engine Watch caters to Webmasters but also accommodates anyone wanting help with search engines. They are at
http://searchenginewatch.com/
ZDNet has a slightly dated but nice tutorial at
http://www.zdnet.com/zdhelp/stories/main/0,5594,2304949,00.html
CNet has their tutorial (May 2000) at
http://www.cnet.com/internet/0-3817-7-1922932.html
Each of these pages have links that can be of further help.
Conclusion
There is more information out there than one can imagine. The trick is knowing how to find it and the ONLY way to find YOUR search engine is to test them and THEN learn how to use them for some have special features that allow you to really narrow it down. (See the Primer on how to narrow down.)
Good hunting.
Robert Young
8/17/01
|
|
