• Playing with the cold callers, scammers and scrapers Oct 23, 2017
    Views: 587

    I have a few systems set up on my directory site https://www.uksmallbusinessdirectory.co.uk/ which monitors when the same IP address is visiting certain pagers too often.

    When this happens the visitor is directed to certain pages which look very similar to the pages they are visiting so that they don't notice.

    I currently have it set up so that when the same IP hits the individual business listings pages too often they get redirected to honeypot business listing pages instead.

    Sometimes they notice after a few visits or after ringing my honeypot listings a few times, sometimes they waste hours or days even visiting fake listings pages without even knowing.

    Occasionally I get what are probably scrapers trying to scrape the content of the individual business listings page, and as long as they aren't causing my server any problems I just leave them to scrape the pages that are deliberately set up to be of no use to them.

    In the last 3 or 4 days I have had a Bangladesh IP visiting my site 30,000 odd times a day.
    I wonder how long it will take them to realise they have just been scraping the content from the same 50 pages in the last few days :)

    I will probably block them from the whole server soon, but doing it this way makes it easier to keep an eye on them.
You need to be logged in to comment