Pages not being indexed

[herodigital]

Ok I have a weird situation with a client site - some pages are not indexed in Google (if I query the exact URL it returns not found).

This in spite of said pages being linked from the HOMEPAGE for over 6 months now.

Any ideas?

P.s. nothing to do with robots, and there is no other form of exclusion. these pages should be indexed.

 

[nitro23456]

following on from my post in your other thread.... are they indexed by Bing? I have stuff indexed in Yahoo but Bing and Google havent touched them. Its not been 6 months though more like 3 weeks, but its bugging me

 

[herodigital]

following on from my post in your other thread.... are they indexed by Bing? I have stuff indexed in Yahoo but Bing and Google havent touched them. Its not been 6 months though more like 3 weeks, but its bugging me

now that's interesting. i've just done a search on the URL in question and i'm getting the same as you - indexed in yahoo but ignored by bing and google.

 

[nitro23456]

now that's interesting. i've just done a search on the URL in question and i'm getting the same as you - indexed in yahoo but ignored by bing and google.

that is!! Iv been trying to figure this out for a while actually. They are obviously seeing something different (or the same and Yahoo different).

 

[pageonetraffic]

Google won't index all pages as a matter of course. It will take what it thinks is most relevant.

I've assumed you've looked up all your pages using the site:domain.co.uk command.

To get the pages indexed, make sure there is at least 200 words of unique content, the title and description must also be unique (from the rest of the site and other sites in your market).

Then social bookmark them using digg, tumbler and others and then email the page from one gmail account to another.

Those 3 things will makes sure the page gets crawled the same day - and the page should appear same day (if its a blog) - a week later if its not.

 

[commodityconsultant]

Use stimator.com to index web page quickly.

 

[herodigital]

Google won't index all pages as a matter of course. It will take what it thinks is most relevant.

I've assumed you've looked up all your pages using the site:domain.co.uk command.

To get the pages indexed, make sure there is at least 200 words of unique content, the title and description must also be unique (from the rest of the site and other sites in your market).

Then social bookmark them using digg, tumbler and others and then email the page from one gmail account to another.

Those 3 things will makes sure the page gets crawled the same day - and the page should appear same day (if its a blog) - a week later if its not.

thanks for this, i've tried the social bookmarking root but it seems to be ineffective. having checked quite a few pages i've got links on recently i've found a lot of them are NOT indexed, which is very worrying.

 

[MASSEY]

I experienced this a few weeks ago.

I would place the domain in " into the google search box and it would not show the page up.

But then i managed to find the site page on page 10 of google.

Like has already been said it must be the fact the pages are too similar.

 

[zomex]

Hello,

As others have said Google won't index every page on a website even if the content, title tag and meta description is unique. I have a website with over 500 pages and can confirm that only about 3/4 of pages are indexed in Google even though everything is unique.

Saying that if your client's website is small then the majority of the pages should be indexed.

Is all of the content unique?

Jack

 

[headshotlondon]

I have exprerienced similar thing recently. I think I ll try to change meta data and maybe then the pages will get indexed. I ll also have a look at copyscape in case there is a duplicate content.

 

[herodigital]

in ref to content being similar and so forth - i watched a video with matt cutts last week and he stated that the number of results returned was an estimation, but he didnt say anything about whether google indexes an entire site or not.

i would have assumed that, as the greatest search engine, they would naturally spider and index and entire site.

@massey do you mean that the page did not show under an exact search but you found it by clicking through the listings when searching for other pages? that's weird.

 

[fisicx]

What incentive is there for Google to index/rank the pages?

It may be that G has visited and followed the links (check your log files) but decided they aren't worth including in the results pages

 

[herodigital]

What incentive is there for Google to index/rank the pages?

It may be that G has visited and followed the links (check your log files) but decided they aren't worth including in the results pages

thats interesting, i've only ever heard of G not bothering with pages that are blatent spam and/or against their terms of service in some way. i can assure you that none of the pages i know of meet those conditions!

 

[fisicx]

OK, but has google actually visited the pages?

When was the homepage last visited? Are any of the pages in the index or just a random selection? Are the links only from the homepage or is there an internal linking structure? Have you checked the webmaster tools to see if any isses have been highlighted? Have you deleted the sitemap to clear any blockages? Have you checked robots.txt to make sure there are no prohibitions...

 

[pageonetraffic]

Use google webmaster tool and yahoo webmaster tool, add the domain manually. Search engine robot will crawl all the pages after adding them to search engine directory.

I wouldn't add your site manually. Your site has already been crawled.. so this is not the issue.. the problem is the pages showing up in the index.

Google won't index every page... it hasn't got the time or resources todo it.. so it will choose whats relevant based on authority.

You can increase the authority by doing what I described in my previous post.

Then if you have already SB'd the page... then create 1x PR1 or PR2 backlink to it... (but no more). Remember you need to wait two weeks for your actions to show up. Don't go crazy and create loads of links otherwise it might get buried.

Lastly make sure the page is not 3 layers deep. ie.. it should be a top level or 2nd level page

 

[SFD]

Does the site use wordpress?

When I upgraded one of my sites recently it added a canonical tag to a lot of pages, pointing bag to the homepage. This means they never got indexed and it took me a while to find the issue. Removing the tag meant they were indexed over the following week or so.

 

[richlaff]

The search engine robots travel over the network and index as many of all the billions of web sites and pages that they can but they have the priorities. Sites that have high link popularity is crawled more frequently, as new sites can take days, weeks and even months to be get picked up by the spiders and to get indexed.

 

[DanHaynes]

email the page from one gmail account to another.

Wow! that really works? Google are constantly scanning you emails? Well I will try it for a particularly stubborn page of mine and see...