Page Index Toggle Pages: [1] 2 
Topic Tools
Hot Topic (More than 10 Replies) Google bot (Read 2,768 times)
mod-two
YaBB Newcomer
*
Offline



Posts: 3
Google bot
Oct 31st, 2010 at 5:24am
Post Tools
How can I stop google bot from filling my eror log.

It makes it difficult to see our members who may be having log in problems.
  
Back to top
 
IP Logged
 
Captain John
Ex Member


Re: Google bot
Reply #1 - Oct 31st, 2010 at 3:47pm
Post Tools
mod-two wrote on Oct 31st, 2010 at 5:24am:
How can I stop google bot from filling my eror log.


Why is the Google bot causing errors ?
  
Back to top
 
IP Logged
 
mod-two
YaBB Newcomer
*
Offline



Posts: 3
Re: Google bot
Reply #2 - Nov 1st, 2010 at 3:28am
Post Tools
I have my error log set to 50 results.

Almost twice a day googlebot fills 80% of it with attempts to access our information.

I lke to view our members who may have trouble logging in so I can help.

And also track three constant forum sabatuers that were banned.

I find the constant goglebot intrusions makes my monitoring duties more time consuming.
  
Back to top
 
IP Logged
 
Captain John
Ex Member


Re: Google bot
Reply #3 - Nov 1st, 2010 at 3:45am
Post Tools
mmmmm I have 9 forums running ... with not 1 error caused by any search engine spider.
  
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #4 - Nov 17th, 2010 at 5:51pm
Post Tools
I asked the same question a few months ago and got the same response.

My error log is really useless.
  
Back to top
 
IP Logged
 
Captain John
Ex Member


Re: Google bot
Reply #5 - Nov 18th, 2010 at 12:48am
Post Tools
Maybe why you both got the same answer is that search engines don't cause errors.
  And maybe like mod-two, RonS2 didn't answer questions asked.
  
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #6 - Nov 18th, 2010 at 2:20am
Post Tools
Quote:
Why is the Google bot causing errors ?


If I knew why I wouldn't need to ask.

Quote:
Maybe why you both got the same answer is that search engines don't cause errors.




« Last Edit: Nov 18th, 2010 at 2:28am by ------ »  

ScreenHunter_006.jpg ( 68 KB | 54 Downloads )
ScreenHunter_006.jpg
ScreenHunter_007_001.jpg ( 76 KB | 49 Downloads )
ScreenHunter_007_001.jpg
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #7 - Nov 18th, 2010 at 2:22am
Post Tools
and more
  

ScreenHunter_008_001.jpg ( 42 KB | 50 Downloads )
ScreenHunter_008_001.jpg
ScreenHunter_009_001.jpg ( 83 KB | 54 Downloads )
ScreenHunter_009_001.jpg
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #8 - Nov 18th, 2010 at 2:25am
Post Tools
And finally
  

ScreenHunter_010.jpg ( 32 KB | 53 Downloads )
ScreenHunter_010.jpg
Back to top
 
IP Logged
 
Captain John
Ex Member


Re: Google bot
Reply #9 - Nov 18th, 2010 at 4:20pm
Post Tools
mmm These access reports are not errors.  They are simply reporting actions and these particular are as you know from a bot.
  These were added to the error log just for knowledge of what is not being reported/found/indexed by these spiders.  This sometimes help admins know why and what is being index or not indexed for rankings on the web.
  
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #10 - Nov 18th, 2010 at 10:23pm
Post Tools
If you re-read this thread John, you will find that you are the only one who ever said these were errors. The problem still remains that google-bot has rendered the error log useless.
« Last Edit: Nov 18th, 2010 at 10:24pm by ------ »  
Back to top
 
IP Logged
 
Captain John
Ex Member


Re: Google bot
Reply #11 - Nov 19th, 2010 at 12:29am
Post Tools
------ wrote on Nov 18th, 2010 at 10:23pm:
google-bot has rendered the error log useless.

Quote:
help admins know why and what is being index or not indexed for rankings on the web.

  Not useless .......  The error.log can be increased to show more .. if one search engine returns to many access entries and of course the more private a forum is the more access restrictions that will be reported, verifying that data is secure.
  
Back to top
 
IP Logged
 
Carsten
Ex Member


Re: Google bot
Reply #12 - Nov 19th, 2010 at 1:19am
Post Tools
Guess it makes some sense if you only want "actual" errors in the error log. You could exclude search bot activity from the error log. Never really had that problem on any of my boards, but i think something like this might help:

In Subs.pl find
Code (Perl)
Select All
sub fatal_error_logging {
 

and add after
Code (Perl)
Select All
	my $user_host = (gethostbyaddr(pack("C4", split(/\./, $user_ip)), 2))[0];
	if (-e "$vardir/bots.hosts") {
		fopen(BOTS, "$vardir/bots.hosts") || &fatal_error("cannot_open","$vardir/bots.hosts", 1);
		my @botlist = <BOTS>;
		fclose (BOTS);
		chomp(@botlist);
		foreach (@botlist) {
			$_ =~ /(.*?)\|(.*)/;
			$bot_name = $1;
			$bothost = qq~$user_host#$ENV{'HTTP_USER_AGENT'}~;
			if($bothost =~ /$bot_name/i) { return 1; }
		}
	} 


As mentioned i could not test it - so no guaranties.
  
Back to top
 
IP Logged
 
------
Senior Member
Beta Testers
****
Offline



Posts: 528
Re: Google bot
Reply #13 - Nov 20th, 2010 at 12:31am
Post Tools
Thanks Carsten

I'll give it a try and get back to you.

Will this run on 2.4?
  
Back to top
 
IP Logged
 
Carsten
Ex Member


Re: Google bot
Reply #14 - Nov 20th, 2010 at 12:04pm
Post Tools
------ wrote on Nov 20th, 2010 at 12:31am:
I'll give it a try and get back to you.

Looking forward to your feedback Smiley

------ wrote on Nov 20th, 2010 at 12:31am:
Will this run on 2.4?

Yep - should make no difference.
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 
Topic Tools