Google spidering links

[SteveGibson]

Hi folks.

I've got a question about google and links.

I've got a website where the homepage has been on google for a couple of weeks, but the other pages (linked to via text links from the homepage) have not been picked up.

Is there a reason why?

Does anyone have a simple explanation of how, when and why google follows some links and not others?

Steve

[Mac Yeti]

Steve,

Might help if you post up the website in question? If you don't want to do that then have you tried adding the site to google sitemaps?

https://www.google.com/accounts/ServiceLogin?service=sitemaps&nui=1&continue=https://www.google.com/webmasters/sitemaps/siteoverview%3Fhl%3Den&hl=en

This site can help you generate a sitemap for google
http://www.xml-sitemaps.com/

[Ender]

Yup,

could be a number of things - javascript links or something. If you are using javascript links, it slows things down a little, but the deep pages will usually get picked up once you get enough backlinks.

Even if you are using vanilla html in your links, it can still take time if you don't have enough backlinks.

I also agree a plainish sitemap linked from a footer at the bottom of every page is a good idea.

Cheers,
Ender.

[GNU]

You can define which links are followed in a file called robots.txt which you would normally create, but may already exist on your host. Have a google for tutorials on the subject.

[arcon5]

How have you concluded such?

Because I have known people in the past thinking they have not been indexed because they are not showing in search terms.

Go to google, type site:domain.tld to see what pages have been indexed.

[arcon5]

Oh, also, if you website is new, then it will take time for Google to visit you. It wont visit you as often as an established website.

Inbound links are essential at startup.

[Yorganic]

This is quite a common occurence and, as a general rule Google will look at the home page more often anyway. Site Maps can help - depends on big the site is as to whether its worth the effort.

[SteveGibson]

Thanks for the suggestions, everybody.

I'll not mention the site, but there is a sitemap and the internal links are fine according to the tool on seochat.

Go to google, type site:domain.tld to see what pages have been indexed.

I did that, that's how i know they got the homepage indexed.

Oh, also, if you website is new, then it will take time for Google to visit you. It wont visit you as often as an established website.

Inbound links are essential at startup.

Google has crawled it at least twice (internal links existed both times) as the version currently in the cache is different (newer) to the first version they indexed.

I'm actually pleasantly surprised. I thought it would take far longer to get spidered the second time.

Probably, for the most part, I'm just being impatient.

However, Tin sugested a couple of things which I'll try. Though, I expect it'll take at least a couple of weeks before either of them kick in.

Steve

[coultog]

I bought one of my domains in March and it took until August before google indexed anything other than my homepage.

Hope this helps.