Posts tagged Demystified

How do the search engines find web sites? Do they wait for you to submit your site? Not any more. Usually search engines index a site whether it is formally submitted or not. Search engines use automated software robots called crawlers or spiders that move through the world wide web following the hyperlinks, from page to page and site to site, recording what they find as they go. They do this continually, checking back with sites already listed to see if they still exist, to find new pages, and to check for new content.

This means that if your web site is linked to from another web site that is already indexed, your site will be indexed on the next pass of the robot! If all of your web site pages are linked properly to each other, they will be indexed as well. In addition, many search engines actually share the same data! So if you get listed on one of them you will eventually be listed on the rest.

The only time you will need to submit your new site is if there is no link from any other web site to yours. At Back2Front, we take care of this for you; however, if you would like to link your site, you can submit your URL here: www.google.ca/addurl/

If you are in a niche market that has its own search engines, like genealogical web sites for example, it is worth submitting your site to specialist search engines, which may not share data with the larger, more general search engines.

So what about the SEO companies that will submit your site to thousands of search engines every month for a year for $375? This service is not only unnecessary, but also may be harmful to your search engine ranking. These companies use robots to submit your site to a list of search engines repeatedly. Most search engine software can detect excessive automated submissions and may actually penalize sites that are submitted multiple times and rank them lower or even black list them.

Of course getting indexed is only half the battle. You need to be listed, but you also need to be well ranked. Ranking a web site is a way of assigning it priority over other sites with similar content. The exact ranking algorithm used by search engine companies is a closely guarded secret. But they do make public the criteria that the calculations are based on and provide basic guidelines to web site designers for successful search engine optimization.

Guidelines for successful search engine optimization: The site must have relevant content to the key words used in the search. The closer the match to the exact key words, the better. The more often, the most emphasized, and the higher on the page the search term appears, the better. The more incoming links from other related web sites, the higher the ranking. The bigger (the more content, the most pages) the higher the ranking. The more current, most often updated, the higher the ranking. The older (longest uninterrupted active time span), the better. It is much more effective in the long run to make your web site as relevant and useful to the people who will search for your product or service as possible than to try and trick the search engines into ranking your web site higher than it should be. In fact, several practices commonly employed to fool search engines now have negative effects. Search engine companies actively research such techniques and developed ways to counter them in the interests of maintaining the accuracy and integrity of their search results.

Some examples of deceptive SEO techniques to avoid are: Very small text not meant to be read by humans but added only for search engines. Same colour text as the background colour (again for search engines only). Excessive use of popular search terms that have nothing to do will your product, like “sex”, or “free”. Misleading or non-relevant text in the meta tags of your site. Adding pages of text that are only long strings full of key words. Participating in link schemes with sites that are not related to your content. Using Javascript to redirect to different content for the user than for the search engine. In short, doing anything with your site that either displays to the search engine different content than the human visitors see, or anything that dilutes or corrupts your unique content is not advisable.

So there you go, there really are no tricks, the rules are straightforward – no more mystery. If your site takes into account all of the above, your site will be well optimized for search engines.

However, there are some Roadblocks to Search Engines that will prevent the search engines from crawling and indexing your site properly that you need to make sure your web site does not have. I will deal with these in the next instalment of Search Engines Demystified.. stay tuned.

by Candace Carter, Back2Front-The Web Site People, 2009

www.back2front.ca

Search Engine Optimization Demystified

Part 3 – Road Blocks to Search Engines In the first part of this series Search Engine Optimization Demystified – Part One, we examined the players in the search engine optimization game. Understanding the industry helps us to see through the hype and separate the actual workings of the search engines from the mythology popularized by those looking to make a quick buck off of SEO services.

In the second part of the series Search Engine Optimization Demystified – Part Two, we looked at how the search engines see your web site. We went through ranking, indexing and the rules to follow to get the best results for excellent natural search engine results.

In this, the third article in this series I will explain how search engines can be prevented from ever getting to your site and how they can be restricted from parts of your web site through bad web development practices. This results in poor search engine ranking and low natural search traffic levels.

Meta Tags There are ways that the informed webmaster can intentionally instruct search engine robots not to index a web site. There are some good reasons to do this. For example, we have a client who only serves the government on construction projects and only uses his web site as an addendum to his bids. He does not want the general public finding his site since answering calls from the public would be a nuisance.

We added this Meta tag to the code of his site: <META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
It does what it looks like it does; it instructs the search engine “robots” not to index this page or follow the links from it.

Now before you get paranoid – I have never seen this done maliciously, but if you have a robots meta tag on your site it should say: <META NAME=”ROBOTS” CONTENT=”INDEX, FOLLOW”>
Since search engines assume this to be the default setting, it is not really necessary to have this tag on your site.

There are also meta tags that can tell the search engines what kind of content to expect on this web site. This may reduce traffic intentionally, for instance: < META NAME =”AUDIENCE” CONTENT=”ADULT”>
This tag is used by search engines to filter search results for people who set their search preferences to exclude explicit content.

You can see the META tags that have been used on your web site by looking at the source code of the site through your browser. For example, with Internet Explorer choose “Source” from the “View” menu. The meta tags will be in the top part of the file between the <head>
tags – if you hit the <body>
tag you’ve gone too far.

Frame Sets A frame set is a set of web pages coded so that parts of your screen show content from different files. This used to be an easy way for an amateur to place a navigation bar on all of the pages of a site. Frame sets are not used by professional webmasters anymore due to the negative impact they have with search engines. Since frame sets first came out in the mid-1990′s, coding has evolved to the point where webmasters can develop web sites faster and more easily without the need for frame sets.

The first page of the site is the most heavily weighted page by search engines. Weighting is done according to various factors, including the amount of relevant text on the page. Since the first page of a frame set is just a set of code instructions to the browser on how to find and use the files for the next page, what a frame set does is to “demote” all of the pages in a site to one level down in the page hierarchy. Search engines use the page hierarchy to determine the importance of the page within your web site. If your first page has no content on it, it will rank poorly, and if all other pages are secondary to the first page they will be ranked even lower.

A typical frame set page looks something like this in the code view:
<HTML>
<HEAD>
<TITLE>A simple frameset document</TITLE>
</HEAD>
<FRAMESET cols=”20%, 80%”>
<FRAMESET rows=”100, 200″>
<FRAME src=”frame1.html”>
<FRAME src=”frame2.jpg”>
</FRAMESET>
<FRAME src=”frame3.html”>
</FRAMESET>
</HTML>

As you can see there is absolutely nothing in the way of actual words in the page that may be indexed by the search engines. You can understand now why utilizing a frame set would depress your search engine rankings.

In addition, search engines like to follow a nice set of hierarchical links in order to index the content of the site. While your menu with all of your links may be contained in frame1.html in the code view above, you are making the search engine go that extra step to find that menu. As we already mentioned, this is effectively how the frame set knocks everything down one level and makes the search engine jump through an extra hoop to get to your content. Search engines expect us as site owners to jump through extra hoops to appease them – they don’t like it at all when the tables are turned and will rank your site accordingly.

Media-Rich Home Pages Media-rich home pages contain Flash, images, video and other multimedia files as the primary content. If you only remember one thing about search engines, remember this: search engines can only read text.

Search engines cannot read images or text written on images. They cannot read text inside of Flash movies or understand voiceover commentary from a video. Pdfs are often encoded as jpeg images and cannot be read by search engines when saved in this format. Search engines can only read text contained in HTML and in meta tags (which are specifically written for search engines).

One common misuse of Flash is the “Splash” or “Landing” page. This is a page that is just a picture or movie with only an “Enter Here” link on it. Since search engines cannot read the content of an image or Flash movie, this page looks blank to them. If the “Enter Here” link is also coded inside the Flash movie the search engines cannot see the link and will not be able to get to any of the pages inside your site regardless of whether they employ Flash or HTML.

Some web sites are written entirely in Flash and are not accessible to search engines as is. There are a few ways around the Flash barrier, like creating an HTML version of the site with meta tags and using additional navigation links in HTML. But why place a barrier to search engines on your site in the first place? We recommend using Flash for decorative purposes. We recommend that it only be used in such a way that if it were removed, search engines would still have everything necessary to index the site correctly.

To test your site’s accessibility to search engines check if you can copy and paste the text from the web browser to your text editor. If the text is written in HTML you will be able to do this. If you can’t, it is likely that search engines will not be able to read this text. A site done entirely in Flash or with images is usually the result of a company web development effort driven primarily by graphic design personnel with little or no input from web development professionals. If you are developing a website for business rather than for a movie, an event, or an art exhibit, you are better off listening to the advice of web development professionals who are trained in SEO concepts for maximum ROI through better search engine placement.

Other Landing Pages A landing page may not be media rich, but may have very little content on it in order to “direct” users to a specific location on the site. In Canada we often see landing pages directing users to click on links for either French or English versions of the site. This can be avoided by using scripts that detect the default language of the browser and direct the user to their preferred language without the need for them to click a link. Using language specific web site addresses is another good practice. Alternatively put the link to the choice of language on every page of the site in the navigation. This allows users to switch languages from any page on the site. This is important once you realize that search engines often display internal pages rather than the home page of a site on their search result pages. Well optimized web sites avoid low content landing pages. Dynamic Content and Menus Search engines cannot read text that is dynamically created when a visitor asks for it. A search engine will follow all of the links it can see on a web site. But a search engine will not type in search terms in a “search box” to see what other content you have in your database. If you have a database driven web site you must have “hard coded” links to the data that the search engine can follow, or much of your web site will be not be indexed.

Search engines cannot read text embedded in JavaScript or any other scripting language that requires the user to do something (like choose from a drop down list of options) to get to new content. Therefore, most of the drop-down type navigation bars you see at the top of web sites are actually barriers to search engines. Unless a search engine spider sees an actual coded link you will lose them. This is actually the most common barrier we will see on a website because inexperienced web design personnel are unaware of the fact that search engines cannot read scripted menus.

As an aside, javascript drop down menus are usually also less human user-friendly due to the fact that they difficult to manually operate, and they provide no navigational reference point since the drop down ‘snaps back up’ disappearing as soon as it is clicked. While there are again various methods to “get around” this obstacle, these are stop-gap measures that can and should be avoided.

Remove the road blocks! As we have seen, there are many web site design practices commonly in use today that put up barriers to search engines. The more experience your webmaster has in SEO the less likely they are to engage in practices that can confuse or mislead search engine spiders. But you, as a web site owner, must make it clear to the developer of your site that SEO is important to you. SEO is time consuming and requires training. Unless you request a search engine friendly web site – and pay for one – you are not likely to get one. If you have any of the above barriers in place on your site, we encourage you to give us a call today to find out how we can get you out of the search engine quagmire that you are likely in.

By Candace Carter, Back2Front – The Web Site People. December 2009

www.back2front.ca