More information you put into your website - more valuable it becomes - and gets more difficult to navigate ! No matter how well you design your site navigation, visitors will need ingenious ways to find what they are looking for. Site search engines provide a powerful and familiar means to provide that access. Visitors can just type the words and press the Search button in a form, and get a list of all the documents that match those words on your site.

We discussed various issues involved, specially who needs search engines and why in previous issue. In this article, we discuss the tasks involved if your website falls into the category that needs search engine.

There are essentially two ways to go about

• buy a search engine software, install it and develop an in-house program for maintenance.
• Outsource the requirement through hosted search engine

The first option offers more freedom but costs a lot and requires considerable in-house technical expertise.

The second option is less expensive and very easy to implement (anyone with practical knowledge of HTML can implement it - does not require technical skill). However, there are a few issues that need attention before opting for such a service.

Your own Search Engine (Local Software)

For an example of such software, consider the search engine at The Great Indian Bazaar (the top right corner of www.infobanc.com) It has separate provisions for searching buyers, sellers, news items etc. So, if you are searching for Buyers, the search engine will only display BUYERS - no unwanted junk. You are unlikely to get such customization in general purpose search engines.

Though this option gives you maximum choice it also requires considerable understanding of how search engines work and technical skill to implement. Besides, it costs more and needs regular maintenance. This place is too short to discuss the issues involved, we shall instead concentrate on second option.

Outsourced or Remotely Hosted Search Engine

The World Wide Web is a marvelous place - you will find helping hands almost everywhere. For example, you can outsource your search to a remote site search service and let someone else worry about the gory details. The indexer and search engine run on the remote server: they will use a web indexing robot, or spider, to follow links on your site and read the pages, then store every word in the index file on that server (without taking a single Kilobyte from your precious server space).

During search, the visitor fills up a form on your local Web page (just like any search engine) which in turn sends the information to the remote search engine. The remote search engine takes the search terms, matches the words in the index, sorts them according to relevance, and creates an HTML page with the results. When a searcher clicks on the result link, they will see the page from your site, just as though the search came from there. It's easy and painless for practically everyone.

Advantages of Remotely Hosted Search Engine

• No need for server access
  Practically any website can use this service - even if your site is on a shared-host and you have FTP access only.
• No need for technical skill
  You never need to install any software or pay someone else to do so
• Easy administration
  The remote search host will take care of daily maintenance. You simply add/update pages - the remote server will automatically index all new/updated pages and keep the index up-to-date.
• No load on your server resources
  Search engines require significant resources, such as CPU time during searching and indexing, disk space etc. Outsourcing to a remote server moves the load away from you. In addition, these servers are usually in data centers with excellent connectivity and administration
• Minimal initial investment
  Instead of paying for a search engine up front, you can pay a small monthly fee. Some services are free, showing advertising with the search results.
• Easy to switch
  If you aren't happy with your search service, it's easy to switch to another.

The Flip Side - Disadvantages

Like all good things in life - this service also comes with riders you need to be aware of.

• Unwanted Advertising or pay every month You must pay every month or allow your searchers to see other people's advertising
• Less control over indexing
  If your data changes frequently (hourly or daily), most of these services will not index that often.
• Dependent on outside service
  If the service's search engine gets busy, it may delay responses for your site, and there's not much you can do.
• Less capacity
  The remote search services have a page limit, although some will index hundreds of thousands of pages.
• Less freedom of choice
  Each search engine has its own special features, but you have more choices if you plan to run your own engine. For example, some have problems indexing password-protected areas, or word processing file formats, adding a thesaurus or a spellchecker, etc.
• Multi-site indexing
  Most remote services allow you to index just the sites you control (though there are exceptions). With a local search engine, you can index other sites and create a public search portal.

How to start

Whether you opt for local search engine or remotely hosted one - you must first complete few housekeeping tasks to prepare your website. We shall discuss the tasks in next issue as also a review of available Internet resources.

Related Links:

• How to find information from Internet - Part II Search Engines - How they work

Source: FAIDA - Newsletter on Business Opportunties from India and Abroad Vol: 3, Issue 22 September 19' 2002