Home > Forums > Products > MetaTraffic Beta

User login

MetaTraffic 2.0 Boxshot

Suggestion - Robots

Posted by adamjanzen [July 22, 2004]

Let me say first that this is becoming a really great product...working on convincing others to migrate to this...

Have a quick suggestion...maybe have an option to disable/enable displaying robot information...for example, the number of visitors when I upgraded didn't include robots (thanks to those who helped on that one!), but as I continue to try Beta6, it seems to be logging them. Don't really want to delete all of the robot data (through the options), but would like to selectively show or hide that stuff.

Don't know if it's just me.... :)

Suggestion - Robots

Submitted by adamjanzen on Mon, 08/30/2004 - 12:28.

Anybody have a workaround for this? GoogleBot is crawling like there's no tomorrow; of course with seperate IPs.

Any ideas?

Suggestion - Robots

Submitted by TechStud on Tue, 08/31/2004 - 06:17.

quote:Originally posted by adamjanzen
Anybody have a workaround for this? GoogleBot is crawling like there's no tomorrow; of course with seperate IPs.

Any ideas?

You could Block their IP's!

TechStud ,
[url="http://www.techstud.com"]www.TechStud.com[/url]

Suggestion - Robots

Submitted by b.e.wilson on Tue, 08/31/2004 - 14:46.

Check your definitions. The default behavior of v2 is to not report robots in the main pagers, but to list them instead in the Robots section. If your definitions were not put into the database properly the program won't know how to recognize a robot and treat it like a normal user.

The new MSNBot crawls even faster than Googlebot. Wait 'till that arrives.

Suggestion - Robots

Submitted by cdegroot on Tue, 08/31/2004 - 16:25.

Googlebot recently changed its user agent so that it is not detected properly by older beta versions. I believe the latest beta should have an updated definitions file that has corrected this. If you download it, grab the definitions.txt file and update your defifinitions in the maintenance section, this should fix the problem.

Regards,
~Chad

Suggestion - Robots

Submitted by adamjanzen on Wed, 09/01/2004 - 21:22.

Hey,

I checked the definition file...the comment says 2004/05/04 and I downloaded the Beta7 again and the definition file is the same size and date.

Any ideas?

Suggestion - Robots

Submitted by cdegroot on Wed, 09/08/2004 - 20:23.

This will be updated soon. In the meantime, replace the line in the definitions.txt file that starts with Googlebot with the following line:

GoogleBot||^Googlebot/2\.\d \(\+http://www\.(googlebot|google)\.com/bot\.html\)$||||http://www.google.com/bot.html||3

Make sure the line isn't wrapped. Save the file and update your defs in the maintenance section. That should fix your problem.

Regards,
~Chad