Forums

Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: Robot Not Honoring JUBlockIP Blacklist

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2017

  • Borders
  • Borders's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
I have a situation where a robot / spider / whatever is not honoring JUBlockIP's Blacklist entry. I've put this I.P. address into the Blacklist box and saved the update but it continues to disregard the blacklist. Any ideas on how to stop this?
The administrator has disabled public write access.

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2018

  • admin
  • admin's Avatar
  • Offline
  • Administrator
  • Posts: 1641
  • Thank you received: 178
  • Karma: 43
It seems your put the wrong IP address of robot/spider.
How do you check that robot/spider still can craw your site?
The administrator has disabled public write access.

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2023

  • Borders
  • Borders's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
I use a tool from JoomlaTools called DocMan. There is a facility in this that sends me an email every time someone / something "downloads" a file that is maintained under DocMan. The I.P. address of the user that downloads the file is given along with date / time, browser, and several other pieces of information. In this case, the robot / spider accessed roughly 30 documents in just under 10 minutes (all with the same I.P. address). The address was 5.9.97.244. So then I used Network Solutions WHOIS tool to find out to whom the address belongs and was returned the following:

NetRange: 5.0.0.0 - 5.255.255.255
CIDR: 5.0.0.0/8
OriginAS:
NetName: RIPE-5
NetHandle: NET-5-0-0-0-1
Parent:
NetType: Allocated to RIPE NCC
RegDate: 2010-11-30
Updated: 2010-12-13
Ref: whois.arin.net/rest/net/NET-5-0-0-0-1

OrgName: RIPE Network Coordination Centre
OrgId: RIPE
Address: P.O. Box 10096
City: Amsterdam
StateProv:
PostalCode: 1001EB
Country: NL
RegDate:
Updated: 2013-07-29
Ref: whois.arin.net/rest/org

I then put the entire NetRange ("5.0.0.0 - 5.255.255.255") into the Blacklist area of the JUBlockIP plugin thinking that the robot / spider would be blocked going forward. To my surprise the next day, I received 20 more notification emails from DocMan saying that the same I.P. address (5.9.97.244) had hit my URL again.

I think I've done everything correctly to have blocked this address with your tool but it seems to be ignoring JUBlockIP. Can you offer any further suggestions or approaches to blocking I.P. s which ignore your product.
Last Edit: 3 years 4 months ago by Borders. Reason: I left off a position in the I.P. address of the NetRange
The administrator has disabled public write access.

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2024

  • admin
  • admin's Avatar
  • Offline
  • Administrator
  • Posts: 1641
  • Thank you received: 178
  • Karma: 43
Try to use this:
5.0.0.0-5.255.255.255

No space.
The administrator has disabled public write access.

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2029

  • Borders
  • Borders's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
I already have the suggestion you made in place (see attached file which I copied from the Blacklist window).

Is it possible that robots / spiders are ignoring your blacklist? Would a "robots.txt" file in one of my Joomla directories work any better?

File Attachment:

File Name: Blacklist.txt
File Size:1 KB


Any other suggestions?
The administrator has disabled public write access.

Robot Not Honoring JUBlockIP Blacklist 3 years 4 months ago #2030

  • admin
  • admin's Avatar
  • Offline
  • Administrator
  • Posts: 1641
  • Thank you received: 178
  • Karma: 43
Let's download the new version, i've add the option, so you can check the IP via URL in test mode.
So you can see how the robots will be blocked by JU BlockIP, just turn on Test mode then use the url like this:
http://www.yoursite.com?ipaddress=ipaddress_of_robot
The administrator has disabled public write access.
  • Page:
  • 1
  • 2
Time to create page: 0.344 seconds
Powered by Kunena Forum