Sunday, January 27, 2008

Does Your Website Measure Up? Part 2: Do Search Engines Like You?

This blog topic is being published as a serial. This is the second of four parts.

Part 2: Do Search Engines Like You?

Most web site owners find a significant amount, if not a majority, of their traffic comes from the major search engines like Yahoo, Google, MSN, AOL, Lycos and a few others. In simple terms, a web site that is set up to standards that meet the expectations of search engines is likely to get more traffic than a web site that is not. This is because those search engine friendly sites will rank higher in the search results, meaning more people will see them in the results and link to them. There are some basic search engine optimization (SEO) techniques that you want to be sure your developers include routinely in the sites and added pages they build for you.

Most search engines respond well to static HTML (Hyper Text Markup Language) pages with lots of text. If you have a site with dynamic URL calls (including search variables, for example), Flash, Frames or other technologies that are not HTML-based, you probably will need SEO specialists to make your site adequately visible to search engines.

Within the current structure of the search engine marketplace, you can improve search results in two basic ways: organically or by paying for placement. Organic SEO refers to using the elements of site and page design and structure that search engines find most acceptable and which produce higher placement in the results based upon those elements alone. This article focuses on organic SEO, but a brief introduction to paid placement is also provided.

Some parts of this topic may get a little technical in terminology. Please don’t let it put you off. Read for the gestalt first. When you finish, use to search for any definitions that you need for clarification. I have provided some hyperlinks for key concepts and definitions if you want a more in-depth view of this subject.

There are also numerous publications for SEO learning. Since the search engines are continually changing the rules, and the different engines have differing approaches, subscribing to a SEO periodical might be good, especially if you are trying to do it yourself or evaluate an SEO employee or vendor. Search Engine News is a good choice for a monthly paid subscription. Search Engine Watch and Pandia Post are good free resources. A review of the results in your favorite search engine for the term “search engine newsletter” or “search engine help” will provide many more paid and free resources to choose from. Just remember when evaluating paid services that this is a labor intensive service and you will typically get what you pay for. Low one-time fees for “search engine submission” are typically of limited value. SEO is an art and requires repeated attention. Be sure to check references and look at the historical site statistics for the reference sites.

Is Your Domain Relevant?

The first step in the SEO game is to pick a domain name that is reflective of the search terms you expect people to use when looking for your products. If you are selling premium ballet shoes, is probably a better choice than Domains that reflect the name of the business and business names that reflect the product or service will help in search engine rankings.

Can Search Engines Spider Your Site?

A critical step is to make sure the content on your site can be found by search engines. Search engines “crawl” or “spider” web sites to look for unique and relevant content associated with certain keywords which are then used to index and weight the sites’ pages. (There will be more on keywords later.) The beginning point is the home page for the site; e.g., index.html. Therefore, in order for a search engine to successfully spider a site, the links from the home page to the sub pages needs to be clear in the home page HTML. There are two very good ways to accomplish this.

Create a site map with all the pages and the links to the pages. Provide one or more links to the site map on the home page. Site maps are terrific and every site should have one. Be sure to keep it up to date as the site changes.
Add a list of all pages as hyperlinks. This is why many sites you visit have a section at the bottom of the page with links to all pages in the site. Again, keep it up to date.

Using one, or preferably both, of these techniques will assure your pages can be readily found by the spiders.

Another critical search engine requirement is the robots.txt file. Your root directory must have a robots.txt file or some search engines will not spider the site. The robots.txt file can be used to disallow access to content by spiders, too. This is frequently done while sites are under development to prevent getting a bad page rank for the domain due to incomplete content.

As a final word on spiders, you can set up software to capture search spider visits and provide valuable statistics by analyzing the spider and the results of the visit. By seeing which pages the spider visited and comparing it to page rankings, the analysis can help determine if a page needs additional optimization effort. This analysis software can also block unwanted spiders like those who visit only to copy email addresses for use in spam campaigns.

Do Your Tags, Links and Names Work Together?

Tags, and the keywords they contain, were primarily what search engines used to rank pages in the early days. Because it did not take site developers long to figure out how to use popular keywords like “sex” in their tags to improve page rankings, the search engines now place far less reliance on tags. However, the repetition of keywords in tags, directory and file names, link names to other pages within the site, and the actual copy or content on the site’s pages is very important. This relationship and frequency analysis by the search engines means that keywords should be carefully optimized in conjunction with their use in other elements of a site’s construction.

Here are the basic keyword elements. I am presenting them to show how they are interrelated, but while I’m at it, I’ve included some best practice “how to” suggestions.

Title tag: Use a few of your most strategic keywords to title your page.
Description tag: Include as many relevant keywords as practical in a description of the site.
Meta tags: List all your keywords here, again remembering to keep them relevant.
Heading tags: Can be added at the top of pages, using keywords.
Alt/Image tags: Use keywords to tag images with relevant identifiers.
Links (hypertext): Use relevant keywords in links to the site’s other pages or files.
Directory Names: Use keywords in directory names. Separate with hyphens.
Filenames: Use keywords in file names. Separate with hyphens.
Site Map: As indicated before, a site map helps the spiders find your pages. It is also another relevant way to include keywords.

If your keywords are relevant and included in tags, names, and content; most search engines will give “extra credit” in the rankings for the repetition. As you will see in a few minutes, the “relevance” is related to the copy, or content, of your site.

A Word About Stuffing

No, I haven’t changed the subject to holiday turkeys. In search engine terminology, “stuffing” refers to the addition of popular keywords that are not relevant to the site content. Search engines will ignore non-relevant keywords, or even penalize sites that use large numbers of “hot” keywords in tags when those keywords are not present in the copy on the site. Similarly, sites that use “invisible” text (text the same color as the page background, for example) to put keywords on pages so it appears to be in the copy are penalized. Some search engines exclude such sites completely from their results. Needless to say, it is not a good idea to use this technique.

Does Your Copy Complement Your Key Words?

Using copy that integrates the strategic words that search engines will use to index your site is essential to placement success. Using text with these strategic keywords at the top of each page, in page and paragraph headings, and at the beginning of paragraphs complements the keyword based tags and names used in the construction of your site. All these uses of the strategic words combine to create a synergistic effect in most search engine algorithms, and the attention to detail and relevant content is rewarded with higher placement in search results from searches using those keywords.

Paying For Search Engine Placement

After all this, budget permitting, you may choose to augment your organic rankings with some form of paid placement. Paid placement is a complex subject and I only intend to introduce it here. For a more detailed discussion, refer to the newsletters reference earlier and look at the descriptions for AdWords and AdSense products on Google’s ads page.

Paid Placement
Paid placement can take many forms. However, the most common is to pay the search engine for preferred placement for certain keywords. The cost can be a function of the number of times the results display the advertiser’s link, the number of searchers who actually click on the link to the site, or a flat periodic fee. Search engines refer to these as sponsored listings, pay per click, and other similar terminology. Most search engines wisely segregate paid results from “organic” results to avoid criticism from their user communities.

Paid Submission
Paid submission or paid inclusion refers to paying a fee for prompt search engine spidering of a web site. It is most frequently used for new sites to expedite appearing in key search engine rankings. Not all search engines provide paid submission. Paid submission does not affect ranking, just inclusion on that specific search engine. Sites will still be ranked by the search engine’s usual algorithms and good organic design is still important. However, if market research shows you that your competition has good success with a particular search engine, you might consider the paid submission approach for a new web site.

Budgeting for keyword “buys” and paid submissions can jumpstart the success of a new site. Ongoing buys of keywords can enhance the success of an established site. However, these techniques are expensive and a minimum budget of several thousand dollars per year will typically be required to have a significant effect, especially with popular keywords.

Traffic Analysis

Knowing how much traffic your site has and your “sales per visitor” can help you evaluate organic and paid search results. If you have 1000 visitors per day and sell $2000 in product per day to 100 visitors, you have a 10% “conversion rate” (visitors who buy) and sales of $200 per customer and $20 per visitor. In the simplest terms, you should be able to evaluate the effect of new initiatives by seeing the change (or lack of change) in these metrics. However, if a site has rapidly growing traffic, has a base of established customer who buy repeatedly, or is constantly changing its product offerings; evaluation of the results accruing from SEO changes becomes more difficult. This is because the other traffic and growth factors are commingled with the SEO factors.

To analyze these complex factors, it is important to know where your traffic comes from and which page they “land” on. Did they link from a search engine; another link source (business partner or advertising resource); or a personal IP address? Did they link to the home page, a product or service page, or a specific informational page like a how-to or success story? It can also be important to know which pages they linked to on your site and which page they were on when they abandoned, or left, the site.

Your web site host should provide some analytic software to get the broad statistics. You can also subscribe to paid analytics that monitor your site and accumulate data for analysis. Ongoing analysis of a site’s dynamics is important to its initial and ongoing success, especially in the realm of search engine results placement. Be sure to budget time and money for this important function.

SEO best practices cover a lot of ground. If you are in charge of a commercial web site that is focused on eCommerce, you must be sure the SEO bases are covered. Similarly, if finding your web site in the search engines has strategic importance to your organization’s operations or marketing success, SEO is a key element of the site design.

The old saying is that if you build a better mousetrap, the world will beat a path to your door. In the twenty-first century, you have to assure your door is easy to find on the web. The world is too big to rely on word of mouth alone to find your mousetrap.