Blackhole For Bad Bots
Reference: blackhole
Free prestashop module to automagically ban bad robots who doesn't follow robots.txt instruction
This is very simple free module for prestashop and thirtybees platforms. It is based on a very simple idea:
-
you instruct all robots visiting your website NOT to open specific url
-
this module will add hidden link from all pages on your website to this forbidden page. This link is perfectly visible to all robots, but normal visitors will not notice it at all (without looking into web page source code)
-
when anyone access this forbidden page, his IP address will be immediately added to blacklist
-
blacklisted visitor are forbidden from viewing content from your website
-
shop administrator is notified about new entries to blacklist. They will receive email with WHOIS information about the visitor - his IP address, location, network, etc.
And that’s it. This trap will not affect any good robots who are following robots.txt
directives. On other hand, all bad bots and crawlers will be eventually trapped and forbidden from ever collecting information from your site again.
Module activation
- edit
robots.txt
file in the root directory
Before you install this module, you need to edit your robots.txt
file, and add following two lines
User-agent: *
Disallow: /blackhole/
-
install module
-
optionally, you can test it by navigating to
http://www.yourdomain.com/blackhole/
. You should be banned from your own site. To lift the ban, reset module from your back office.
Moderation
At the moment there isn’t any UI to manage blacklist. If you want to remove some IP address from blacklist, you have to make manual changes in database table called PREFIX_blackholebots_blacklist
Great idea. Yet to test.
Lets hope that those bad bots wont learn how to go around the trap.
Top
Very effective
So simple, yet so effective!
The idea behind this module is so simple yet the reality is that it traps bas bots so well. I've already had one of them banned from our site who fell into the trap, although how smart the bots become to navigate around the black hole remains to be seen, although I suspect if the bot were that intelligent it would be a legit one that would respect the robots file anyway, too marks from me!!.
Simple and effective
Simple and great
-
version 1.0.2 released on 2019-02-22
Initial release