WHITE PAPERS AND ARTICLES

WORKING WITH SEARCH ENGINES VERSUS YOUR USERS
Article appeared in MiBizWest and MiBizSouthwest on May 17, 2004
Written by Charles McGrath, Structure:Interactive

Search engine placement is our clients’ number one concern. Often, their trade names don’t appear on the first page of an AltaVista search. Their application terms don’t rank highly, either. And just as often, a competitor’s site ranks higher.

The first question is "how can we improve." The second question should be "does it matter?" To answer these questions, it is first important to know how search engines work.

How Search Engines Work
Search engines generally work by "crawling" the web. Automated software arrives at a site and clicks through it in much the same way that a user would, checking every link and cataloguing the information it finds. This raw data, just an index of the text on the page, is then evaluated to determine relevance. Each search engine uses a slightly different formula, which it guards closely, to deliver rankings. These formulae also change over time, as the engines try to outsmart the sites that are trying to outsmart them. But despite the cloak and dagger, there are some general rules.

Basically, search engines look for the search phrase in three places: the title of page (as displayed in the browser window); the metadata, or page descriptors, located in the page code and hidden from the reader; and the visible text on the page. If a keyword is found in all three places, it will rank high on the list, and it will (usually) rank higher still based on the number of occurrences in the text. This doesn’t mean that you can just put a phrase fifty times on a page, or use the old trick of microscopic text at the bottom. Search engines have caught on to this.

Some search engines (in particular Google, the current favorite), go further, and try to determine how relevant a page is to you. Google looks to see how many pages link to a particular page, the logic being that if a page is popular with other people, it contains information that will be important to you.

So search engines can offer some surprises. One of our clients has a trade name that is in common use. On AltaVista, their site didn’t appear in the first few pages of results, although the term itself appeared everywhere, often improperly. The results were similar on Yahoo. The client’s product name is so widely used across the Web, in distributor sites, trade sites, and personal sites, that it was impossible for the actual client site to break through the clutter. Google, however, brought our client to the top.

Aside from encouraging all of your users to use Google, there is only so much you can do to improve your rankings, since the deck is stacked against you by the search engines themselves. This means that companies that promise to improve your rankings are generally only going to do the same things a well-designed site would do already.

What can you do to improve your search rankings?
The answer is "not much," and "a lot." As our client found, it is virtually impossible to push a common trademark up through all the clutter on the web. The same goes for broad search terms such as "Insulation," since the phrase could refer to wall insulation, thermos insulation or a down jacket.

So what can we do? First, recognize that, in large part, it just doesn’t matter. Users looking for a company usually try the company name first as a domain name. Thus, someone looking for a trade name is likely to try typing that word in first, and, assuming it’s been registered, will find the site. On the other extreme, users typing in an overly broad term will quickly realize that they need to refine the search.

And that’s where we can be effective. Users turn to search engines to perform tasks, and to get answers to questions. They may first ask about insulation, but what they really want to know is "What’s the best way to insulate and waterproof my basement?" It’s our task, then, to make sure that our sites, and the searchable components on each page, are set up to answer these questions. This means evaluating each page from this standpoint, and tweaking content and metadata appropriately.

It also means de-emphasizing the importance of the homepage. There is no way that the homepage can successfully answer every question, so a site should be constructed with multiple landing points. And since the site can be accessed from any page, navigation and clear site structure become extremely important.

It’s also important to recognize the technical limitations of search engines. A "crawler" is not particularly intelligent, and can only find pages that are linked in HTML to other pages. This means that more sophisticated navigation, like drop-down menus and rollover menus, are invisible to them. Pages that can only be reached through such fancy navigation won’t appear in searches.

Are there any shortcuts? Of course. And, as you’d expect, they cost money. While none of the reputable sites sell rankings, they do sell banner ads linked to particular keywords. In our experience, broad search terms aren’t cost effective, but more targeted market-focused keywords might be worth buying. Worth it, that is, if search engines are where your audience is.

Do search engine rankings matter?
This is a question with an answer that varies from site to site. Most of our clients find that fewer than 20% of their visitors are referred from search engines, and those visitors are more likely to have come to the site by mistake.

On the other hand, our clients tend to be leaders in their markets, and are less likely to be "discovered" in a search engine. If you aren’t well known and can’t afford an external promotional budget, focusing on search engine rankings, and perhaps even buying keywords, could be a cost-effective way to raise your online profile. But if you are fairly well established, monitor your web traffic to see how valuable the search engines are for you. If you’re like many of our clients, you’ll find that, while the numbers are too big to be ignored completely, they are not the most efficient way to drive traffic to your site.

And remember — the basics of building a search-engine-friendly site are the basics of good web design. If you build a site that is designed around the user, that is focused on answering user questions, and that is easy to navigate and to understand, then you have built a site that will also perform well in search engines.

Which is really the answer to the question. Building a site that scores well in search engines is important, but it’s even more important to build a site that will work well for your users.

It’s a relief to find that, in the end, that can mean the same thing.

Charles McGrath is Internet director and creative director at Structure:Interactive. He has won numerous awards for interactive media, and is in demand as an Interface Design Consultant. In addition to his work for Structure:Interactive, he has been involved with interactive projects for the Walt Disney Co. and Mattel.

< back