All Search Engines Love Spiders: How Meta Commands Can Help You Love Them Too

By Scott Buresh

Nearly all search engines utilize spiders (which are also known by their original name, robots) to go out and scour the web looking for web pages. These search engine spiders then bring the data back to be indexed by the engine.

Since roughly 1996, individual meta commands have existed that can be used on individual web pages to modify how these search engine spiders behave. The most useful of these commands are fairly universal and respected by almost all search engines. What follows is a list of some of the more popular spider commands and instances in which you might want to use them. (Please note all <> have been replaced with [] for publishing purposes).

[meta name="robots" content="index"]

This meta command is one of the most common ones used – and it is also the least necessary. It tells search engine spiders to come on in and put the page in their indexes. However, all search engines do this by default anyway. Basically, if you want to put it in there for fun, be my guest, but this command is not giving you any special treatment. All search engines are going to index your page, unless you specifically tell them otherwise.

[meta name="robots" content="follow"]

The follow command is different from the index command. It basically requests that the search engine spiders follow the links that are on a particular page. Again, however, this piece of code is completely unnecessary because all search engines are going to follow the links on a page, unless otherwise directed.

[meta name="robots" content="noindex"]

The noindex command, the opposite of the index command, tells search engine spiders not to index the content of a page. It’s important to note, however, that search engine spiders will still follow the links on a page that uses only this command.

When not used for legitimate purposes, this tag can be dangerous because it can put you at risk for penalization by most, if not all search engines. This is because you can use a noindex tag to hide pages with multiple links that you don’t want visitors to see but that you do want all search engines to index.

There are, however, some legitimate uses for the noindex command. For example, if you have a dynamic site, and you’ve created static pages to replace some of your dynamic pages, which can make them easier for search engine spiders to access, you could put a noindex tag on the dynamic version.

As Google mentions in its Webmaster Help Center:

"Consider creating static copies of dynamic pages. Although the Google index includes dynamic pages."

All Search Engines Love Spiders: How Meta Commands Can Help You Love Them Too

By Scott Buresh

Nearly all search engines utilize spiders (which are also known by their original name, robots) to go out and scour the web looking for web pages. These search engine spiders then bring the data back to be indexed by the engine.

Since roughly 1996, individual meta commands have existed that can be used on individual web pages to modify how these search engine spiders behave. The most useful of these commands are fairly universal and respected by almost all search engines. What follows is a list of some of the more popular spider commands and instances in which you might want to use them. (Please note all <> have been replaced with [] for publishing purposes).

[meta name="robots" content="index"]

This meta command is one of the most common ones used – and it is also the least necessary. It tells search engine spiders to come on in and put the page in their indexes. However, all search engines do this by default anyway. Basically, if you want to put it in there for fun, be my guest, but this command is not giving you any special treatment. All search engines are going to index your page, unless you specifically tell them otherwise.

[meta name="robots" content="follow"]

The follow command is different from the index command. It basically requests that the search engine spiders follow the links that are on a particular page. Again, however, this piece of code is completely unnecessary because all search engines are going to follow the links on a page, unless otherwise directed.

[meta name="robots" content="noindex"]

The noindex command, the opposite of the index command, tells search engine spiders not to index the content of a page. It’s important to note, however, that search engine spiders will still follow the links on a page that uses only this command.

When not used for legitimate purposes, this tag can be dangerous because it can put you at risk for penalization by most, if not all search engines. This is because you can use a noindex tag to hide pages with multiple links that you don’t want visitors to see but that you do want all search engines to index.

There are, however, some legitimate uses for the noindex command. For example, if you have a dynamic site, and you’ve created static pages to replace some of your dynamic pages, which can make them easier for search engine spiders to access, you could put a noindex tag on the dynamic version.

As Google mentions in its Webmaster Help Center:

"Consider creating static copies of dynamic pages. Although the Google index includes dynamic pages."

How to Create Search Engine Friendly Web Site Copy

By Kalena Jordan

Search engines read text and not much else. Because they can’t generally index graphics, search engines rely on the text in web sites to provide information about the site content, which they can compare with search queries.

Webmasters therefore need to use body text on any pages on the site that they want indexed by the search engines and ranked highly for matching search queries. Not graphical text that was created in design software, but actual, visible body text. Not sure if your site uses graphical or body text? A good rule of thumb that I learnt from search engine guru Danny Sullivan is to try and highlight the text with your mouse. If you can drag your mouse over individual words in the text when viewing it in a browser, chances are this is body text and the search engines can read it.

Figure 1



Figure 2

The most important page on which to use body text is the home page. Above is an example of a home page that uses graphical text instead of body text. Figure 1 shows what content the site visitors see, while Figure 2 shows the content a search engine sees and indexes.

How much information about a site’s content does a page like the one above provide a search engine? That’s right, very little. With next to no text to be found, the search engine would have to rely on the page’s Title and META Tags to tell it what the page is about. With such little information to go on, it is unlikely that a search engine would consider this page a relevant match for search queries relating to its content. To remedy this, it is widely recommended that each web page you want listed in search engines should contain at least 250 words of visible body text.

Keyword-Rich Text

While it’s a good idea to use plenty of body text on web pages, if that text doesn’t contain relevant keywords and phrases that people type in to the search engines, there’s not much point, because a site isn’t going to be found for logical search queries anyway. Many web sites make the mistake of including text on their site that is either unrelated to their products and services, or full of marketing-speak like “Internet solutions” or “superior services”. The Internet is plagued with web sites selling particular items without once making reference to those items in their site text. Weird huh?

For a search engine to find a site relevant for a particular search query, it MUST find that search query somewhere in that site. The easiest way to ensure this is to include logical keywords and phrases within the visible text on web pages, as well as in the Title and META tags. The best way for webmasters to find keywords that searchers are actually using is by conducting keyword research of their target market on a site such as Keyword Discovery or WordTracker.

Once it is determined what search terms perspective visitors are commonly typing in to search engines, they can then be compared to the goods and services offered on the site and the body text can be adjusted accordingly. Sites lacking any keyword research tend to use very generic, unfocused body copy, or sales-oriented “hype”. Neither style contributes to high search engine rankings.

Target keywords and search phrases placed strategically throughout your body copy give your pages a much higher ranking potential on search engines for related searches. But it’s not as easy as throwing the keywords into your site text willy-nilly. You must ensure that the keywords are integrated seamlessly so their repetition is unobvious and so that the text flows smoothly for the reader.

Don't compromise the readability of your copy to achieve this - hire an expert copywriter to strike the right balance if need be.

SEO Copywriting

Before writing your web site copy, you should research potential keywords and phrases that your target audience may use in search engines and then narrow the list down to your priority terms for each page, sorted in order of importance. You should then use those target search terms as a basis for the creation of optimized Title and META tags for each page on your site. Once you’ve done that, it’s time to integrate those same target search terms into your visible web page copy. We call this SEO copywriting. But exactly how do we do it?

Speak to Your Audience

Don’t lose site of the reader when writing your body copy. Integrating your keywords is important, but not if you are sacrificing the readability of your site and losing the attention of your audience. Put yourself in their shoes like you did when researching your keywords. What are they looking for? What do they need? How will your product/service help them? Does it represent value for money?

Be emotive when describing your products and services. Describe how your product/service will make them feel or look, how it will improve their lives, give them more time etc. Use trigger words that people respond to such as “free”, “success”, “you”, “cash” etc. Not sure what these are? Check out Words That Sell reports. These reports are perfect if you are targeting a specific industry or profession because they define what keywords people in over 38 industries respond to and what they expect when making a buying decision.

Not sure who your audience is or what they’re looking for? Why not ask them? Use a free survey service such as Survey Monkey to learn more about them so you can write “to” them and not “at” them. You could even draft various styles of body copy and obtain feedback from your site visitors to determine what copywriting style works better for them.

Use Easy to Understand Language

The Internet is no place for verbosity. People are in a hurry - they want to find what they seek quickly and easily with the least hassle possible. You can help them in this quest by ensuring your site pages use simple language and easy to grasp concepts throughout. For example instead of "brand-building web information architects", use "website designers specializing in brand promotion". Keep the large chunks of text on each page to a minimum, using bullet points, white space, graphics, lists and sub-headings to break it up and make it easier to read. This rule of thumb is especially important when creating landing pages for pay per click and other advertising campaigns.

Use examples to get your main points across or to demonstrate your product benefits. Use the old WIIFM (What's In It For Me?) adage when composing your body copy to keep the user's interests at top of mind. Remember your international visitors by incorporating regional word usage (such as organize versus organize or jewelry versus jewellery) and avoid technical jargon that could alienate. Want your visitor to take a particular action? Spell it out for them in plain English, for example Click here to Buy Now, Subscribe to our free newsletter, Bookmark this page now etc. These references are called “Calls to Action”.

Build Your Copy Around Your Keywords

You should always build your page copy around your keywords and not the other way around. If your existing page copy doesn’t contain any of your target search keywords, you’re going to have to rewrite it! Start from scratch if you have to. The secret is to focus. Search engines aren’t going to rank your web site about socks highly if your body copy talks about foot sizes. You need to get specific. It sounds really obvious, but if you sell socks, make sure your site copy has plenty of references to the word socks! If you sell green wool socks, target the phrase "green wool socks" and not "foot apparel in lovely shades of emerald"! Who's going to search for socks using that phrase?

At the risk of sounding like Dr Seuss, if you want to be found for, big socks, small socks, cotton socks and wool socks, then mention them all. Better still, sort your copy into categories based on your various products and services. If you sell wool socks AND cotton socks, then have a page dedicated to each kind. This allows you to target niche keywords within your copy and meet the search engine's relevancy guidelines for related search queries.

Keyword Integration

So imagine you've added plenty of text to your pages and the copy flows well for the reader. You've researched your keywords and phrases and now you're faced with the dilemma of integrating the keywords into your copy. So how do you satisfy the search engine's craving for keywords without interrupting the copy flow for the reader? The answer is: very carefully.

Let's take a look at a practical example. We have a client that specializes in luxury adventure travel. Before I optimized their site, part of the home page copy read like this:

"We specialize in providing vacations for people who want a personal service. We bring to our efforts a fanatical obsession with quality and exclusivity. We also bring a freshness, an outward-going passion for discovery which justifies our growing reputation as one of the world's top travel providers. We can put together packages that include all adventure activities, accommodation, transport and food".

Extensive Keyword Discovery keyword research for the client had determined that the site should target the following key phrases:

• adventure travel

• best adventure vacations

• tailored travel

• overseas adventure travel

• luxury travel packages

So taking our original home page text, the challenge was to integrate these keywords carefully and naturally so as not to disturb the logical flow of the copy and lose the interest of the visitor. Here's how I did it:

"We specialize in providing the best adventure vacations for people who want a personal and tailored travel service. We bring to our efforts a fanatical obsession with quality and exclusivity. We also bring a freshness, an outward-going passion for discovery which justifies our growing reputation as one of the world's top overseas adventure travel providers. We can put together luxury travel packages that include all adventure activities, accommodation, transport and food".

Note that the key phrase "overseas adventure travel" accommodates the phrase "adventure travel" too. Voila! The search engines are happy because the site contains text content relevant to related search queries, the client is happy because we were able to integrate the keywords without distracting the visitor and I'm happy because I know the site is going to rank highly for the client's target search terms.

Linas Bari is designer and developer at Smooth Step studio that offers web design in Marbella

In order to get there yourself, you have to find out about these sites, because they will likely have some juicy, highly sought after information that you need to know. The information you find on these sites will help you build a solid business that will allow you to make money at home online.
How do you find these authority sites? The best way to get ahold of the top internet marketers who really know what they're doing is by joining discussion forums in your niche. For example, the Warrior Forum or Associate Programs Forum have some the top internet marketers in the world that will help you make money at home online.
It is in these discussion forums where you will get and give help, as well as learn the tricks of any online business you may be involved with.
Top internet marketers with authority sites will often have blogs that you can subscribe to to get the latest information on building you own authority site.
Oftentimes, when you subscribe to their newsletter lists, you will get free ebooks with invaluable tips and techniques within.
Be sure to link to as many of these authority sites as you can, even if you have to pay, in order to boost your own website's rankings. When you have good rankings yourself, others will want to link to you.
When you want to make money at home online, it's not all about the rankings you get from links. True authority sites will offer information that is found no place else on the net and is hard to get a-hold of. Much of this information will be absolutely unique to that web-master, and that's what makes it so good.
Becoming an authority site does take time and effort, but if you can survive the tough times in your business, and get past at least one year in business, you are doing far better than most, and are on your way to success.
So you have some links, you have some rank, and you have come up with some good original articles and blog content to add to your site regularly. You've been in business for over a year, are in touch with the experts in your niche, and now what do you do?
For one thing, search engine optimization, or SEO is extremely important if you want to generate traffic and sales to your site. If you use good, researched keywords in your articles and blogs, you are seriously getting there! Find out about SEO and keywords. You won't regret it!
Next, you're going to have to understand the most effective ways to make your blog posts if you hope to make money at home online. Not only do you need quality content and keywords, but you need to get involved with social networking services and get your blogs onto these sites. Find out about blogging, and social bookmarking and networking. You won't regret it!
One of the easiest ways to draw traffic that converts on your site is by using tags that link to Technorati, and other keyword links within your posts that get people to click. Find out about Technorati tags and link baiting! You won't regret it!
Finally, it's important for any business - if you really want to make money at home online – that you keep up with the ever-changing internet. Again, if you stick to some quality forums, you have a good chance of knowing the most current buzz on any business related topic.
Right now, the whole video marketing craze is taking over the net as a top-notch way to advertise. If you're not in on that game, you may fall behind in your marketing efforts. If you stay hip in your market, you have a good chance of becoming an authority marketer and site.
Article written by Liane Bate.

Choosing Page Names For Your Web Site

In this article I would like to comment how to choose web site page names to achieve best possible visitor's experience and search engine ranking.
Firstly one needs to understand the difference between static and dynamic URLs.
Dynamic Pages
Dynamic website page names are used to send variables to a script file in order to perform a task that usually generates dynamic results. Dynamic URLs always contain question mark that separates file name from sent variables. Example name "do.php?a=1&b=2" would pass variable a=2 and b=2 to "do.php" script file that generates some response.
Such technology is very useful for web designers but it down performs in search engine listings. Search engines prefer static URLs that are less likely to change over time.
Static Pages
To explain static page naming we are going to use two example URLs: "about.htm" and "about.php".
Classic example of a static page name consists of name and extension. Page name extension indicates technology ("htm" says that page uses HTML technology) or a programming language ("php" extension indicates that page was generated using PHP server side scripting) used.
Many consider that pages with extension other than "htm" or "html" are worth more than those with server side scripting language indicators (like PHP, ASP and others).
If you will try to search for "search engine optimization" in Google and look at domain names of first 20 web sites, you will notice that URLs with many extensions are present. This indicates that search engines don't really care about extension as long as the page represents useful information for the visitor.
Friendly URLs
Friendly URLs are static page names with no extension at all. Friendly page names are very useful to the user as they are easy to remember (E.g. help, shop, contact …). Page name extension that refers to scripting language/technology used in the page. Extension-less construction protects page names from becoming obsolete if different web technology is going to be used to generate / present them in the future. Such names can be achieved only using URL rewriting technology.
URL rewriting
Rewrite engine is a special service on the server that modifies URLs before they are processed by scripting language. It is used to convert dynamic page names to static ones:

Dynamic page:
index.php?page=news&item=34
Can be changed to static page:
news/34.html
Or simply
news/34

Rewrite engine is very helpful to maintain static page names for complex database driven websites, where parameters have to be sent to generate results. Dynamic pages are usually not cached by web browsers so by using rewrite technology to generate static pages one increases overall web site performance.
Apache web servers have built in component called "mod_rewrite" which allows setting up rewrite engine by uploading RegExp (Regular Expression) commands to ".htaccess" file. For those who use Windows IIS web servers - Free IconicIsapiRewrite plug-in can be installed to achieve URL rewriting.

Linas Bari is designer and developer at Smooth Step studio that offers web design in Marbella