Help in Robots.txt

[snareshiver]

Hi,

I have one important url which is not indexed in Google. To make it indexed faster can i add it in Robots.txt under allow.

[SFD]

This won't make a difference as robots.txt only tells the bots which files to exclude, not force them to include.

Best way to get it indexed would probably be a few internal links, including from the homepage and also some external links.

[snareshiver]

since i am running a shopping site, products are found after many clicks from the Homepage.

So I added the link in the Homepage and submitted sitemap, still not crawled.

[snareshiver]

Thanks for the replies

[dunerider]

since i am running a shopping site, products are found after many clicks from the Homepage. No..... google wants all shopping sites to have one product per page so people are taken direct to the page to buy. if your site does not do this and only has multiple products per page - this is probably why nothing is indexed.

your home page can not possibly be optimised for every product on your site, you need single pages so each product can be optimised to stand a chance of good results

[iamlijo]

the best thing to do is to


1. create a sitemap . Try xml-sitemaps.com

2 .Add your site to google webmasters tool .

www.google (http://www.%3Cb%3Egoogle%3C/b%3E).com/webmasters/

3. verify your site and submit sitemap .

see this video youtube.com/watch?v=hlMTQakgCMM


4 . Last and Important , get some quality backlinks from sites related to your niche. wait 3-4 days , your site will get listed. Else PM me , i can help you.


best of luck