UKBF Homepage Returning 403 Forbidden Response (only some User Agents)

Discussion in 'Feedback & Help' started by zigojacko, Mar 20, 2019.

Thread Status:
Not open for further replies.
  1. zigojacko

    zigojacko I say it how it is Full Member - Verified Business

    3,637 1,170
    I don't know if you are aware, and I don't know what you've done or if it is even intentional but the UKBF homepage returns a '403 Forbidden' response for *some* bots.

    Two of which are GoogleBot and BingBot... (not Yahoo! (Slurp(?) though, you seem to like them).

    I don't really care, it's just annoying that a link to UKBF is persistently being flagged as a broken link in a client's site audit every week (and that is the SEMrush bot by the way).
     
    Posted: Mar 20, 2019 By: zigojacko Member since: Dec 7, 2009
    #1
  2. Darren_Ssc

    Darren_Ssc UKBF Ace Free Member

    1,030 290
    Probably a cloudflare firewall rule that has been configured wrong? Could also be the firewall working correctly and blocking fake googlebot? (since I can replicate this behaviour, having just checked).

    They can fetch as googlebot in search console to confirm though.

    All SEO Bots should be blocked btw, they are a menace.
     
    Last edited: Mar 20, 2019
    Posted: Mar 20, 2019 By: Darren_Ssc Member since: Mar 1, 2019
    #2
Thread Status:
Not open for further replies.