Welcome Guest ( Log In | Register)



 
Reply to this topicStart new topic
> Robots And Sub-domains, Do the Bots see the Sub-domains
jlhaslip
post Jan 30 2006, 11:41 AM
Post #1


A computer once beat me at chess, but it was no match for me at kick boxing.
Group Icon

Group: [MODERATOR]
Posts: 4,085
Joined: 24-July 05
From: Linix, DOS and Windows…the good, the bad and the ugly
Member No.: 9,787
Spam Patrol



I guess I have had a site up for a few months now and the Googlebots and others are beginning to find the site, according to the Web Stats at least.

My question is: when the Bots travel through and spider a site, do they actually see the sub-domains associated with the main domain? And do the contents of the sub-domains affect the schedule for the return of the Bots in the same way as good Content would affect the return? For instance, if you regularly update a sub-domain and keep adding or changing the content, will the Bots be back quicker or more regularly? Are the domain and the sub-domain inter-related with respect to the Bots?

Another question, if I may: similarly, should the Meta-tags for pages other than the Index page by different than the Meta-tags for the Index page? I would think that the Meta-tags for the inside pages could be more specific to the particular page, and if the Meta-tags for the secondary pages are different, will the Bots pick these up and handle the secondary any differently?

(assume the robots.txt file does not exclude bots from sub-domains, of course.) blink.gif
Go to the top of the page
 
+Quote Post
Amezis
post Jan 30 2006, 01:53 PM
Post #2


Privileged Member
*********

Group: Members
Posts: 535
Joined: 14-February 05
From: Oslo, Norway
Member No.: 3,759



robots.txt will not do anything to the subdomains. From what I've heard, some bots sees the subdomains as external sites. But after what I've experienced, that's partially wrong. My site is not indexed, and none of the subdomains are. Seems like the domain is banned for some reasons, and that the subdomains are banned too.

But bots follows external links too, so if you have links to the subdomains from your main page, I'm sure the subdomains will be crawled too.
Go to the top of the page
 
+Quote Post
spy_charly
post Jan 31 2006, 04:35 AM
Post #3


Premium Member
********

Group: Members
Posts: 195
Joined: 18-November 05
Member No.: 14,521



well yep... i think the robots are supposed to search into your subdomains as soon as you register your domain with the search engines.

But another way to do it is to have a refferece in your main page or either in some of your pages registered into your search engines
Go to the top of the page
 
+Quote Post

Reply to this topicStart new topic

Collapse

> Similar Topics

Topics Topics
  1. Do Short Domains Improve Seo?(5)


 



- Lo-Fi Version Time is now: 13th October 2008 - 05:42 PM