• 1

Read this first!

We do not monitor these forums. The forum is provided to exchange information and experience with other users ONLY. Forum responses are not guaranteed.

However, please submit a ticket if you have an active subscription and wish to receive support. Our ticketing system is the only way of getting in touch with RSJoomla! and receiving the official RSJoomla! Customer Support.

For more information, the Support Policy is located here.

Thank you!

TOPIC: Googlebot banned as repeat offender!

Googlebot banned as repeat offender! 12 years 1 month ago #21757

Hello,
today i noticed that my site completely disappeared from Google. So i went to webmaster tools and realized that google is blocked from my website. And off course google bot ip autobanned in Blacklist as "repeat offender"!
How that can be? What kind of security extention which bans google?!
There was no alert generated, i got NOTHING about it, and my alert level is medium, but so far i received only unsuccesful login attempt emails, nothing else. There's 91 IPs in Blacklist already as repeat offender and i use this extention for only 2 weeks. I suspect there's many of them there by mistake, as googlebot. Now i'm really scared to use your extention and thinking to ask for the refund, because the software is really expensive, but it can completely damage your google results without you knowing about that!!
;(
The administrator has disabled public write access.

Googlebot banned as repeat offender! 12 years 1 month ago #21762

  • octavian
  • octavian's Avatar
  • OFFLINE
  • RSJoomla! Official Staff
  • Posts: 783
  • Thank you received: 110
Automatic blacklisting turned on (it is off by default) without verifying if your website has any pages that might trigger a false alert is a bad idea and it leads to problems such as these (blocking legitimate users).
What you can do is:
1) Go to Firewall Configuration > set "Enable automatic blacklisting" to "No".
2) Remove all blacklisted IPs from the "Blacklist/Whitelist" section.
3) Go to Google Webmaster Tools and check which pages were not crawled due to errors - try to access them yourself and see if there are any error messages encountered.

My guess is that this is caused by the "Verify system variables for PHP injections" setting - you may have a link on your website that looks like a remote or local file inclusion attempt.
Please note: my help is not official customer support. To receive your support, submit a ticket by clicking here
Regards,
RSJoomla! Development Team
The administrator has disabled public write access.

Googlebot banned as repeat offender! 12 years 1 month ago #21777

Hello,
Ofcourse 1st thing i did is turned it off and removed all ips.
But to be more clear what should i do now to restore my pagerank?
May be you'd consider to auto-whitelist googlebot in your extention by default? It's good thing to do, no? I hardly can imagine someone who will buy your extention to block google. And as i see i'm not the only one who experience this problem. You can't expect from (quite expensive) security extention to make things like that.
Another thing: why it haven't sent the notification to me about blocking any of those ips? If i'd see that i would definitely check it and remove!
The administrator has disabled public write access.

Googlebot banned as repeat offender! 12 years 1 month ago #21826

  • octavian
  • octavian's Avatar
  • OFFLINE
  • RSJoomla! Official Staff
  • Posts: 783
  • Thank you received: 110
Google has a whole range of IPs that are subject to change anytime so IP filtering is out of the question. The only whitelisting option remaining would be by User Agent. However, RSFirewall! is open-source - that means, anyone can read and understand its code. If we add by default a whitelisting for the Google user agent, it's only a matter of time before attackers learn that and disguise themselves as Google (some of them already do that) and thus rendering RSFirewall! completely useless.
But that's besides the point. You have errors on your website that need to be addressed - they will end up blocking a lot more legitimate users than just Google, so having a whitelist for Google is just part of the solution, not the solution.
I've already said that you can use Google Webmaster Tools to check which pages triggered errors on your website.
Another way (if you're using RSFirewall! R44) would be to go to Firewall Configuration > Active Scanner and set "Log all blocked attempts" to "Yes". If you don't see this option, please re-download the RSFirewall! package again from your account and update your installation (no information will be lost in the process). This will log every URL blocked by RSFirewall! and you can see which ones trigger a protection and why.
I cannot tell you when your pages will show up again since this is entirely at the search engine's discretion - you can check Google Webmaster Tools for more information on when your website was last crawled.
Please note: my help is not official customer support. To receive your support, submit a ticket by clicking here
Regards,
RSJoomla! Development Team
The administrator has disabled public write access.
  • 1

Read this first!

We do not monitor these forums. The forum is provided to exchange information and experience with other users ONLY. Forum responses are not guaranteed.

However, please submit a ticket if you have an active subscription and wish to receive support. Our ticketing system is the only way of getting in touch with RSJoomla! and receiving the official RSJoomla! Customer Support.

For more information, the Support Policy is located here.

Thank you!