Duplicate Content Issue wont go away!!

cody44 · Apr 4, 2012

Hi Guys,
I have been testing a site out for a client and found that it contained Duplicate content issues I uploaded a htaccess file with the following information:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.gkfirstaidtraining\.co.uk$
RewriteRule (.*) http://www.gkfirstaidtraining.co.uk/$1 [R=301,L]

I then did a check on the site and the issue seemed to have been resolved. That was until I put my site details into SEOMoz and low and behold it still indicates that I have the Duplicate content issue!

The site has been crawled after I uploaded the htaccess file and I know that SEOMoz have crawled the site with the htaccess file present.
All other Duplicate Content programs I check give it the thumbs up but SEOMoz Open Site Explorer still says Duplicate content!!

Can someone put me in the right direction on how to resolve this as I just seem to be making more changes to the htaccess with different coding but SEOMoz still says 'Duplicate Content!.

nitro23456 · Apr 4, 2012

Without admitedly having .htaccess comitted entirely to memory, that code looks like you are redirecting nothing.

terryuk · Apr 4, 2012

What are you trying to do?? Does your http://domain redirect to http://www.?

I don't know what Seomoz class as duplicate content although this may do the job;

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

try that..

cody44 · Apr 4, 2012

terryuk said:
What are you trying to do?? Does your http://domain redirect to http://www.?

I don't know what Seomoz class as duplicate content although this may do the job;

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

try that..

Thanks Terryuk I will try that and see what happens

cody44 · Apr 4, 2012

cody44 said:
Thanks Terryuk I will try that and see what happens

Hi Terry UK

Have done a quick test at the following website
http://www.virante.com/seo-tools/duplicate-content
And below is the error message!!!!!

Default Page Check: FAILED
You have not standardized your default pages meaning the following versions of your url return a 200/OK Header, which may cause duplicate content issues. The following extensions work:
http://www.gkfirstaidtraining.co.uk/index.htm
http://www.gkfirstaidtraining.co.uk/

terryuk · Apr 4, 2012

You need to work that out yourself I'm afraid - not a complete free helping hand. Good luck

Jack Clark · Apr 4, 2012

You need to do a 301 redirect from (www) dot gkfirstaidtraining (dot) co (dot) uk/index.htm to www (dot)gkfirstaidtraining(dot)co(dot)uk/. That should sort out the duplicate content issue for that page. Just make sure that all of the other pages on the site don't behave in the same way. (apologies for the obscure URLs, but I am still not allowed to post URLs until I have 15 posts).

Let me know if you need anymore help,

Jack

cody44 · Apr 4, 2012

Have reverted back to my original code in the .htaccess file:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.gkfirstaidtraining\.co.uk$
RewriteRule (.*) http://www.gkfirstaidtraining.co.uk/$1 [R=301,L]

This is passed on http://www.virante.com/seo-tools/duplicate-content

so just need to get my head round why it fails on SEOMoz open site explorer!!

Thanks for the direction Terryuk

trev.pope · Apr 4, 2012

Just to be sure your site is not on a Windows Server is it?

Jack Clark · Apr 4, 2012

The reason that you are getting a dupe content flag is because you have two versions of the same content at different URLs. You can access you homepage at /index and at the root, which means that Google will treat the pages separately.

to fix this you need to do a 301 redirect from /index to the root.

Let me know if you need more info,

Thanks,

Jack

cody44 · Apr 4, 2012

Jack Clark said:
The reason that you are getting a dupe content flag is because you have two versions of the same content at different URLs. You can access you homepage at /index and at the root, which means that Google will treat the pages separately.

Hi Jack,

If I am understanding this the code below is directing non www to www?

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.gkfirstaidtraining\.co.uk$
RewriteRule (.*) http://www.gkfirstaidtraining.co.uk/$1 [R=301,L]

So surely this will redirect the /index.htm page to www?

I have even put into the Meta Tags the following:
<link rel="canonical" href="http://www.gkfirstaidtraining.co.uk" />

trev.pope · Apr 4, 2012

cody44, I do not believe that is your issue.

Are you on a Windows Server?

cody44 · Apr 4, 2012

I have put the canonical meta tag in and was figuring out the htaccess coding - I have just received a crawl report in csv from SEOMOZ which I requested yesterday morning, it indicates no duplicate content issues.

So I think I best do another crawl report leaving everything in place and see what the crawl report says - if sqeaky clean and all good I will post the content of the htaccess file in case anyone else has a similar issue.

cody44 · Apr 4, 2012

trev.pope said:
cody44, I do not believe that is your issue.

Are you on a Windows Server?

Hi Trevpope...................yes im on Windows!!! what effect could that have on the duplicate content?

nitro23456 · Apr 4, 2012

nothing, your .htaccess might not work as it is for apache and not iis. But of course you could be WAMPing it?

cody44 · Apr 4, 2012

nitro23456 said:
nothing, your .htaccess might not work as it is for apache and not iis. But of course you could be WAMPing it?

Thanks will keep a heads up on that.

I think I will wait for the next SEOMoz crawl and see what comes out in the wash before I go breaking anything else!!!

RevaxMedia · Apr 4, 2012

cody44 said:
Hi Guys,
I have been testing a site out for a client and found that it contained Duplicate content issues I uploaded a htaccess file with the following information:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.gkfirstaidtraining\.co.uk$
RewriteRule (.*) http://www.gkfirstaidtraining.co.uk/$1 [R=301,L]

I then did a check on the site and the issue seemed to have been resolved. That was until I put my site details into SEOMoz and low and behold it still indicates that I have the Duplicate content issue!

The site has been crawled after I uploaded the htaccess file and I know that SEOMoz have crawled the site with the htaccess file present.
All other Duplicate Content programs I check give it the thumbs up but SEOMoz Open Site Explorer still says Duplicate content!!

Can someone put me in the right direction on how to resolve this as I just seem to be making more changes to the htaccess with different coding but SEOMoz still says 'Duplicate Content!.

Rel canonical = URL on the page, if too many, then you will need to use the .htaccess

p.s SEOmoz does not abide by other robot rules such as robots.txt - try GWT this will give you a better insight.

RevaxMedia · Apr 4, 2012

cody44 said:
Hi Trevpope...................yes im on Windows!!! what effect could that have on the duplicate content?

.htaccess won't work unless you have ISAPI Rewrite installed - so I am led to believe.

In this case from www to nonwww create your website in Plesk as non www, create a sub domain of www and redirect all www urls to non www via IIS.

cody44 · Apr 4, 2012

RevaxMedia said:
Rel canonical = URL on the page, if too many, then you will need to use the .htaccess

p.s SEOmoz does not abide by other robot rules such as robots.txt - try GWT this will give you a better insight.

Thanks RevaxMedia,

Have been looking into GWT for a while - Will download it and see whats in the box.

RevaxMedia · Apr 4, 2012

Hi Cody, Google Webmaster Tools is very easy to install and free - all you need to do is upload their random html file to your server and verify ownership then it can give you more in depth information regarding your website - far more thorough than SEOmoz

cody44 · Apr 4, 2012

RevaxMedia said:
Hi Cody, Google Webmaster Tools is very easy to install and free - all you need to do is upload their random html file to your server and verify ownership then it can give you more in depth information regarding your website - far more thorough than SEOmoz

Never realised it was that powerful. Thanks I will check out over the weekend while eating my easter eggs!!!!

open sesame · Apr 5, 2012

I use Seomoz on a pro subscription because it's the nuts, if they are saying you got a problem then you have.

How about you dump the index.htm links in all those sitemaps you got in the footer and anywhere else you have it on the site!

Cookie Consent

Essential

Analytics

Marketing

Duplicate Content Issue wont go away!!

Jack Clark

Jack Clark

RevaxMedia

RevaxMedia

RevaxMedia