p5-WWW-RobotRules-6.02nb7

Basic Info:

Perl 5 module database of robots.txt-derived permissions

homepage: http://search.cpan.org/dist/WWW-RobotRules/
license: gnu-gpl-v2 OR artistic
install size: 26460
build date: 2019-01-26 01:32:49 +0000
package file: p5-WWW-RobotRules-6.02nb7.tgz
package size: 11624

Description:

The Perl 5 module WWW::RobotRules parses /robots.txt files as specified
in "A Standard for Robot Exclusion", at
http://www.robotstxt.org/wc/norobots.htmls
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.

The parsed files are kept in a WWW::RobotRules object, and this object
provides methods to check if access to a given URL is prohibited.
The same WWW::RobotRules object can be used for one or more parsed
/robots.txt files on any number of hosts.

Homepage:
http://search.cpan.org/dist/WWW-RobotRules/

Provides:

Requires:

Depends:

p5-URI>=1.27
perl>=5.28.0<5.30.0

Package List

File NameDateSize
lib/perl5/vendor_perl/5.28.0/WWW/RobotRules/AnyDBM_File.pm2012-02-15 12:473617 bytes
lib/perl5/vendor_perl/5.28.0/WWW/RobotRules.pm2012-02-18 06:0511155 bytes
lib/perl5/vendor_perl/5.28.0/x86_64-linux-thread-multi/auto/WWW/RobotRules/.packlist2019-01-25 18:32241 bytes
lib/perl5/vendor_perl/man/man3/WWW::RobotRules.32019-01-25 18:328088 bytes
lib/perl5/vendor_perl/man/man3/WWW::RobotRules::AnyDBM_File.32019-01-25 18:323359 bytes