yascrapy用golang和Python联合完成的高性能分布式爬虫
需积分: 18 72 浏览量
2019-08-10
07:47:04
上传
评论
收藏 166KB ZIP 举报
yascrapy用golang和Python联合完成的高性能分布式爬虫.zip （76个子文件） 
yascrapy-master 
MANIFEST.in 18B benchmark bench_utils.py 530B bench_request_queue.py 2KB bench_rabbitmq.py 2KB bench_filter_queue.py 873B __init__.py 0B bench_response.py 2KB bench_mongo.py 1020B bin yascrapy_worker 4KB yascrapy_cache 7KB yascrapy_producer 2KB conf migrate_rabbitmq.json 239B common.json 413B rotate_log.conf 112B proxy.json 408B migrate_ssdb.json 142B core.json 363B docs conf.py 9KB api_guide.rst 2KB installation.md 3KB getting_started.md 3KB
requirements.txt 62B Makefile 8KB index.rst 462B source conf.py 9KB api_guide.rst 2KB installation.md 3KB getting_started.md 3KB index.rst 462B _static 
yascrapy.png 40KB _static yascrapy.png 40KB tools delete_spider.py 3KB migrate_rabbitmq.py 4KB rabbitmqadmin.py 2KB proxyconfig.py 4KB ssdbadmin.py 3KB proxyadmin.py 1KB rabbitmq_to_ssdb.py 4KB migrate_ssdb.py 2KB responseadmin.py 2KB yascrapy ssdb.py 2KB utils.py 2KB rabbitmq.py 6KB bloomd.py 388B plugins handle_proxy.py 1KB __init__.py 0B handle_error.py 5KB mongo.py 2KB request_queue.py 11KB tests test_mongo.py 913B test_error_handler.py 4KB test_proxy_handler.py 928B
test.html 69KB test_config.py 2KB zh_404_err.html 6KB test_response.py 5KB 404_err.html 25KB response_queue.py 8KB __init__.py 0B config.py 1KB base.py 14KB filter_queue.py 2KB libs __init__.py 0B pybloomd.py 19KB sample_spider sample_worker.py 1KB items.py 92B __init__.py 86B settings.py 911B sample_producer.py 771B docker bin rabbitmq-start 503B startall 286B bloomd.conf 120B Dockerfile 2KB setup.py 978B .gitignore 217B README.md 241B资源评论
