crawly:Crawly，一个用于Elixir的高级Web爬网和抓取框架
需积分: 13 164 浏览量
2021-05-12
03:39:39
上传
评论
收藏 96KB ZIP 举报
crawly-master.zip （85个子文件） 
crawly-master 
mix.exs 3KB .github FUNDING.yml 66B .tool-versions 21B scripts hex.sh 322B .circleci config.yml 617B lib crawly pipeline.ex 2KB fetchers httpoison_fetcher.ex 366B splash.ex 2KB fetcher.ex 519B response.ex 430B pipelines csv_encoder.ex 1KB experimental send_to_ui.ex 848B validate.ex 1KB json_encoder.ex 762B write_to_file.ex 4KB duplicates_filter.ex 2KB api.ex 2KB loggers send_to_ui_backend.ex 1KB spider.ex 2KB requests_storage requests_storage.ex 5KB requests_storage_worker.ex 3KB utils.ex 7KB data_storage data_storage.ex 4KB data_storage_worker.ex 2KB parsed_item.ex 433B worker.ex 6KB application.ex 1KB settings.ex 2KB middlewares unique_request.ex 782B auto_cookies_manager.ex 1KB domain_filter.ex 907B robotstxt.ex 796B request_options.ex 861B user_agent.ex 899B engine_sup.ex 637B manager.ex 7KB manager_sup.ex 586B request.ex 2KB engine.ex 8KB crawly.ex 4KB config test.exs 1KB config.exs 2KB dev.exs 15B LICENSE 11KB .formatter.exs 171B test crawly_test.exs 1KB worker_test.exs 8KB pipelines experimental send_to_ui_test.exs 711B write_to_file_test.exs 3KB csv_encoder_test.exs 857B json_encoder_test.exs 493B validate_test.exs 1KB duplicates_filter_test.exs 2KB utils_test.exs 2KB data_storage_test.exs 2KB test_utils.ex 2KB test_utils.exs 347B settings_test.exs 1KB request_storage_test.exs 3KB manager_test.exs 5KB test_helper.exs 48B engine_test.exs 4KB middlewares domain_filter_test.exs 2KB auto_cookies_manager_test.exs 2KB request_options_test.exs 464B robotstxt_test.exs 554B user_agent_test.exs 918B unique_request_test.exs 506B api_test.exs 649B engine_tests.exs 527B data_storage_worker_test.exs 1KB README.md 6KB mix.lock 10KB docs 
index.html 776B .nojekyll 0B favicon.ico 31KB _site index.md 0B README.md 310B .gitignore 141B documentation configuration.md 6KB ethical_aspects.md 486B experimental_ui.md 1KB tutorial.md 11KB basic_concepts.md 16KB http_api.md 611B资源评论
