By Scott Buresh
Nearly all search engines utilize spiders (which are also known by their original name, robots) to go out and scour the web looking for web pages. These search engine spiders then bring the data back to be indexed by the engine.
Since roughly 1996, individual meta commands have existed that can be used on individual web pages to modify how these search engine spiders behave. The most useful of these commands are fairly universal and respected by almost all search engines. What follows is a list of some of the more popular spider commands and instances in which you might want to use them. (Please note all <> have been replaced with [] for publishing purposes).
[meta name="robots" content="index"]
This meta command is one of the most common ones used – and it is also the least necessary. It tells search engine spiders to come on in and put the page in their indexes. However, all search engines do this by default anyway. Basically, if you want to put it in there for fun, be my guest, but this command is not giving you any special treatment. All search engines are going to index your page, unless you specifically tell them otherwise.
[meta name="robots" content="follow"]
The follow command is different from the index command. It basically requests that the search engine spiders follow the links that are on a particular page. Again, however, this piece of code is completely unnecessary because all search engines are going to follow the links on a page, unless otherwise directed.
[meta name="robots" content="noindex"]
The noindex command, the opposite of the index command, tells search engine spiders not to index the content of a page. It’s important to note, however, that search engine spiders will still follow the links on a page that uses only this command.
When not used for legitimate purposes, this tag can be dangerous because it can put you at risk for penalization by most, if not all search engines. This is because you can use a noindex tag to hide pages with multiple links that you don’t want visitors to see but that you do want all search engines to index.
There are, however, some legitimate uses for the noindex command. For example, if you have a dynamic site, and you’ve created static pages to replace some of your dynamic pages, which can make them easier for search engine spiders to access, you could put a noindex tag on the dynamic version.
As Google mentions in its Webmaster Help Center:
"Consider creating static copies of dynamic pages. Although the Google index includes dynamic pages."
Nearly all search engines utilize spiders (which are also known by their original name, robots) to go out and scour the web looking for web pages. These search engine spiders then bring the data back to be indexed by the engine.
Since roughly 1996, individual meta commands have existed that can be used on individual web pages to modify how these search engine spiders behave. The most useful of these commands are fairly universal and respected by almost all search engines. What follows is a list of some of the more popular spider commands and instances in which you might want to use them. (Please note all <> have been replaced with [] for publishing purposes).
[meta name="robots" content="index"]
This meta command is one of the most common ones used – and it is also the least necessary. It tells search engine spiders to come on in and put the page in their indexes. However, all search engines do this by default anyway. Basically, if you want to put it in there for fun, be my guest, but this command is not giving you any special treatment. All search engines are going to index your page, unless you specifically tell them otherwise.
[meta name="robots" content="follow"]
The follow command is different from the index command. It basically requests that the search engine spiders follow the links that are on a particular page. Again, however, this piece of code is completely unnecessary because all search engines are going to follow the links on a page, unless otherwise directed.
[meta name="robots" content="noindex"]
The noindex command, the opposite of the index command, tells search engine spiders not to index the content of a page. It’s important to note, however, that search engine spiders will still follow the links on a page that uses only this command.
When not used for legitimate purposes, this tag can be dangerous because it can put you at risk for penalization by most, if not all search engines. This is because you can use a noindex tag to hide pages with multiple links that you don’t want visitors to see but that you do want all search engines to index.
There are, however, some legitimate uses for the noindex command. For example, if you have a dynamic site, and you’ve created static pages to replace some of your dynamic pages, which can make them easier for search engine spiders to access, you could put a noindex tag on the dynamic version.
As Google mentions in its Webmaster Help Center:
"Consider creating static copies of dynamic pages. Although the Google index includes dynamic pages."
No comments:
Post a Comment