Search Strategies

A question most people wonder is whether they should take advantage of a search engine's default searching mode or use Boolean Logic and other search features to personally control the retrieval of documents.  The answer depends on the nature of the question and on the searcher's skills. 

Nature of Question
It never hurts to try out a search in the engine's default mode.  At the most, you will lose a couple of minutes, and you may even find ideas to help formulate your Boolean or other query.  You should attempt a search in the default mode if the question is rather general, and you think that there will be a lot of relevant material on the Web, then you might want to let the search engine try to find the information.   Put in a list of terms describing your topic and see what results await you. 

Searcher's Skills
You should know Boolean and other search features well enough to use it effectively.  The less these skills are present, the more the searcher will need to rely on the search engine's default mode.  Some search engines provide pull-down windows or other methods to achieve the effects of Boolean, field searching, proximity, etc.

Because there are so many variables, it is not easy to provide clear, surefire rules for when to use the default mode and when to employ Boolean and other search techniques.  Instead you should:

• Be willing to try each approach (default relevance ranking, Boolean, etc.)
• If one option doesn't work, try another.
• Know the advanced features that are available to you.

< PREVIOUS | NEXT >

Keyword Searching

Before beginning a search, ask yourself the following questions:

1. Who/What?
2. Where?
3. When?
4. How?
5. Why?

All of these five categories will not always apply to a given query, but they are useful to keep in mind when analyzing the major components of a search.

All search engines ignore "stoplist" words because they have minimal informational value and are found so commonly in all language.  
Examples of these words include the, of, a, in, as, if, not, why, never, before, is, it, about, and.

< PREVIOUS | NEXT >

Boolean Searching

(also known as Boolean logic, Boolean algebra)

If a search engine has full Boolean capabilities, it will offer the following search options:

AND operator
You will get hits that contain both terms.

Example:  automobile AND sales would retrieve only those pages where both the word "automobile" and the word "sales" are present on the same document.

OR operator
You will get  more hits than an AND search because you will get results that contain either term.

Example: cats OR dogs would retrieve all documents that have the word "cats" plus all the pages that have the word "dogs"

NOT operator
Excludes a word.

Example: climbing NOT ice would retrieve all pages that have the word "climbing" except those that have the word "ice".

AND NOT
Sometimes you will see this variation of the NOT operator. Documents containing the term after this operator are rejected from the results set. 

Notes

• When using operators, use upper case characters.  Lower case will work with some, but not others.
• Try using Boolean searching even if you are unsure whether the search engine you are using offers it.  If the engine doesn't apply them, chances are it will ignore the operators.

< PREVIOUS | NEXT >

Phrase Searching

The most powerful keyword term is the phrase.   Phrases are combinations of words that must be found in the search documents in the EXACT order as shown.  Enter the phrase with double quotation marks around it.
Example: "solar system"

Some search services provide specific options for phrases, some do not allow them at all, but almost all will allow you to enter a phrase in quotes, ignoring the quotations if not supported.

Note:  Be careful when using phrases in a search.  Some minor variations can cause the search to fail.  Submit phrases in different ways to alleviate this problem.

• The spaces between words are as important as any other character.   If you include a double space between any two words and the phrase typically has only one, the search will fail.
• Sometimes two dashes are used together in web pages.  If you include only one dash, the search engine may miss all those documents that contain two.  

< PREVIOUS | NEXT >

Field Searching

Restricts searches to certain portions of documents or pages, such as the title of the Web site, or words in URL only.   Some search engines provide no field searching capability.

Title Searches

Title searches can be specified using an assigned line or a pull-down menu to restrict the search to certain portions of documents or pages.

Image Searching

Some engines now provide image searching capabilities.  You can browse to a special database of images from the Web and other collections or you can use field searching. 

< PREVIOUS | NEXT >

Date Searching

It probably isn't a good idea to limit your search by date. Within search engines, date usually does not refer to the date of publication of the content of a Web page, but rather to the date when the page was created or last modified.  If no creation or "last modified" date is on the page, the engine may use the date the page was picked up or last visited by the search engine.

< PREVIOUS | NEXT >

Truncation Operators or Wildcards

Allow searching on the stem or root of a word or allowing for variable characters.  Use the asterisk wildcard for truncation.  Generally, you must have a minimum of three characters at the beginning of a word as your stem basis, but truncation seems to work best when the stem is longer, when plurals are represented by an added "s" rather than "ies" or other forms, and the stem is not a root to many other common words. The wildcard tells the search engine to match all characters after it and increases coverage by 50% or more.

Example: bird* search will match the words bird, birds, birding

< PREVIOUS | NEXT >

Proximity Operators

Allow searchers to specify that search terms be near others within a certain number.

The NEAR operator may allow for two words to be a certain distance apart.   Sometimes you can specify the maximum distance, or specify whether the words need to be in a specific order, or if either word can come first.

Example: "global warming" NEAR icebergs   would find records that contain both words no further than ten words apart depending on the search tool.

The BEFORE operator is similar to NEAR.  The first term before this operator has to occur within a specified word distance BEFORE the term on the right side of this operator in order for the document to be scored as a result.

The AFTER operator is also similar to NEAR.   The first term before this operator has to occur within a specified word distance AFTER the term on the right side of this operator in order for the document to be scored as a result.

< PREVIOUS | NEXT >

Name Searching

The ability to search specifically for names of companies and people.  Some search engines can do this because they can distinguish between upper and lower case characters.  If the first letters of words are capitalized, the engine assumes it may be a proper name.

Suggestions

• Use the NEAR operator when available.  This will allow for the name to be inverted and will also allow for middle names.
  Example: Hillary NEAR Clinton will retrieve Hillary Clinton; Hillary Rodham Clinton;  Hillary Diane Rodham Clinton; Clinton, Hillary
  
• Put the name in double quotes.
  
• Enter all the possible variables together linked by the OR operator.
  Example:  "Hillary Clinton" OR "Hillary Rodham Clinton" OR "Clinton, Hillary"

< PREVIOUS | NEXT >

Punctuation and Capitalization

• Not all search engines handle punctuation in the same manner.  You should consult the help file of the search engine you are using.
• Most search engines are insensitive to whether you use upper, lower, or mixed cases in your search.  If you use lower case, most engines will match both upper and lower case. 
• For the few search engines that support capitalization, you can find proper names or place names easier.

Note:  Instead of retyping your query each time, take advantage of the computer's Copy and Paste capability.

< PREVIOUS | NEXT >

Search Tips

Basic Suggestions

• The easiest way to find a Web page without using a search engine is to guess the URL.  Become familiar with the major domain identifiers such as .com, .gov, and country codes.  Organizations will try to register memorable or easy to find addresses, so try a few possible combinations of name and domain identifiers.
  
• Shorten the URL.  You might get to a page with an index to what else is listed on the site.
  
• To find the information you are looking for on a Web page, use Internet Explorer's Edit - Find on this Page command .
  
• Identify exactly what it is that you're looking for.   Try to picture the "perfect" web site, what it would contain, and formulate your search based on that criteria.
  
• Try a broad search first to get an idea of the volume of information on your topic.  Then, use the advanced search options to narrowly focus the search, removing items one-by-one as needed on subsequent searches, depending on results.

Advanced suggestions

• Plan your strategy before picking your search engine.  Decide which features would be helpful such as Boolean logic, parenthesis, truncation, proximity, phrases, etc.  Find out which engines provide the features you need. 
  
• Some search engines evaluate your search from left to right.  But others have a hierarchy of operations where some operators are considered more important than others.  Usually "and"  is given a higher rank than "or".   To solve this problem, you may want to use parentheses.   This is a simple way to ensure the search engines evaluate your search from left to right.
  
• Use the AND operator to link search terms.  Proper left to right evaluation of the query is ensured.
  
• Combine keywords into phrases when possible. 
  
• Truncate words to pick up singular and plural versions.  Think about the roots of the words you're searching for.   
  
• Use nouns and objects as keywords.  Verbs, adjectives, adverbs, and conjunctions are either thrown away by search engines or can be described in many different ways.  Avoid common words.
  
• Put the main subject first.  Some search engines give more weight to the words you list first.
  
• Use synonyms or alternate words that say the same thing.  Do not use too many of them or your other concepts may receive too little attention.  Use the OR operator; this will cover the likely different ways a concept is described.
  
• Use 6 to 8 keywords in a search.  More keywords can reduce the number of documents returned by 99%.
  
• If you do not get good results, try rearranging the order of words.
  
• Use Metasearch tools.  They increase Web coverage by 3 to 4 fold.
  

Check out About.com Web Search for searching tutorials, online help files, and search engine finding aids.

Test your skills with the Internet III Exercise!

< PREVIOUS | NEXT >

Evaluating Search Engines

There are more than 2500 search services on the Web, and there are approximately a dozen major search services not to mention metasearch tools. It will require a great deal of time just to develop an in-depth knowledge of just one search engine.  Spend some time checking out the top five to ten search engines to determine which one works best for you.  Each search engine does not index a majority of web sites, and each of the major search engines can produce a significant number of results that are not found by its competitors.  You should experiment with several search engines, and then choose a few that you are most comfortable with.

Some criteria to consider when selecting a search engine are:

• Size of the database.
• Currency of data.  In an index, how often do the spiders or robots search or in a directory, how many professionals are searching and updating the finding?
• Speed.
• Ease of use of the interface.
• Quality of help tools.
• Search features.
• Clarity and quality of search results.  How many relevant hits the first time you search versus relevant hits after refining the search.
• Are there duplicate "hits"?
• Elimination of invalid and expired hyperlinks.

< PREVIOUS | NEXT >

Popular Search Engines

Check out Search Engine Watch http://www.searchenginewatch.com/ for information on advanced searching and how search engines work. 

< PREVIOUS | NEXT >

The "Invisible" or "Deep" Web

What is it?

Information found in databases on the Web that traditional search engines can not index. For example, traditional search engines are able to access the main page of a library’s web site or a shopping site, however, they are unable to search the contents of a library’s catalog for specific items. It is this information that is considered part of the "invisible" or "deep" web. The "invisible" or "deep" web includes newspaper and magazine archives, dictionaries, specialized search engines, discussion group archives, catalogs, calculators and other interactive web resources.

How are "Invisible" Web search tools different from regular search engines?

They are designed to save you time and frustration when trying to locate information "hidden" in databases on the web. Traditional search engines may index the main page of a database but often you will need to search the web page to find a search form to enter your query. With most search engines you will pull up thousands of sites and have to search through these results to determine which sites are databases. Often, it is hard to distinguish which web pages are databases versus regular web sites because many search engines do not provide this information. With "deep" web search engines, you are searching only for select databases that contain information on your subject. Therefore, you will receive a much smaller number of results, making searching much more manageable.

It is important to note that some very good subject directories on the web, like the Librarian’s Index to the Internet do have direct links to databases and are clearly identified in your search results.

< PREVIOUS | NEXT >

Information that can be found on the "Invisible" or "Deep" web

• General & specialized databases
• Patent & Trademark databases
• Government databases
• Telephone directories
• Dictionaries
• Tools that perform various calculations
• Auction databases

Examples of when to use Invisible Web Search Engines/Tools

• Looking up definitions
• To find out various rates; (mortgage rates, credit card rates, car loan rates, etc.)
• Searching for forms
• Searching for addresses of businesses or people
• Finding a doctor, lawyer, realtor, etc.
• Locating hotel prices
• Whenever you need something calculated

"Invisible" Web Search Tools

• Invisible Web
  www.invisibleweb.com
  
  
• Complete Planet
  www.completeplanet.com

• Internets
  www.internets.com

To test your "Invisible Web skills," take the Invisible Web Exercise.

To find out more about these search tools and other Invisible Web resources, check out the Invisible Web tutorial at www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html

< PREVIOUS | NEXT >

TOP...