p5-WWW-RobotRules-6.02nb6

Basic Info:

Perl 5 module database of robots.txt-derived permissions

homepage: http://search.cpan.org/dist/WWW-RobotRules/
license: gnu-gpl-v2 OR artistic
install size: 26288
build date: 2017-11-09 20:54:47 +0000
package file: p5-WWW-RobotRules-6.02nb6.tgz
package size: 11608

Description:

The Perl 5 module WWW::RobotRules parses /robots.txt files as specified
in "A Standard for Robot Exclusion", at
http://www.robotstxt.org/wc/norobots.htmls
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.

The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited.
The same WWW::RobotRules object can be used for one or more parsed
/robots.txt files on any number of hosts.

Homepage:
http://search.cpan.org/dist/WWW-RobotRules/

Provides:

Requires:

Depends:

p5-URI>=1.27
perl>=5.26.0<5.28.0

Package List

File NameDateSize
lib/perl5/vendor_perl/5.26.0/WWW/RobotRules/AnyDBM_File.pm2012-02-15 12:473617 bytes
lib/perl5/vendor_perl/5.26.0/WWW/RobotRules.pm2012-02-18 06:0511155 bytes
lib/perl5/vendor_perl/5.26.0/x86_64-linux-thread-multi/auto/WWW/RobotRules/.packlist2017-11-09 13:54241 bytes
lib/perl5/vendor_perl/man/man3/WWW::RobotRules.32017-11-09 13:548002 bytes
lib/perl5/vendor_perl/man/man3/WWW::RobotRules::AnyDBM_File.32017-11-09 13:543273 bytes