crawler-scala
需积分: 9 64 浏览量
2021-07-04
05:31:01
上传
评论
收藏 86KB ZIP 举报
crawler-scala-master.zip （77个子文件） 
crawler-scala-master 
.gitignore 337B project plugins.sbt 111B build.properties 18B Build.scala 3KB example-app src main scala Main.scala 3KB crawler-lib src main resources logback.xml 1KB protos models_http.protobuf 888B reference.conf 5KB scala serialization Serializable.scala 0B SerializationUtils.scala 3KB protos com blikk serialization HttpProtos.java 157KB model ESJsonTransformations.scala 2KB CrawlItem.scala 162B Messages.scala 650B Responses.scala 1KB Requests.scala 2KB rabbit RabbitData.scala 4KB RabbitConsumer.scala 693B RabbitPublisher.scala 3KB processors StatusCodeFilter.scala 1KB StatsCollector.scala 363B FrontierSink.scala 804B ESRabbitRiverTransformer.scala 816B CrawledItemSource.scala 932B CrawlStats.scala 637B RequestExtractor.scala 3KB UrlNormalizer.scala 656B DuplicateFilter.scala 5KB RabbitMQSink.scala 3KB TerminationSink.scala 829B utils Logging.scala 395B LoanPattern.scala 422B test resources reference.conf 1KB scala testutils TestHttpServer.scala 2KB LocalRabbitMQ.scala 1KB TestConfig.scala 388B MockCrawler.scala 1KB AkkaSingleNodeSpec.scala 746B crawler-backend src main resources application.conf 2KB scala CrawlService.scala 2KB Config.scala 1KB PerDomainFlow.scala 726B RabbitThrottler.scala 4KB Messages.scala 371B ResponsePublisher.scala 2KB Frontier.scala 3KB CrawlServiceLike.scala 4KB FrontierThrottler.scala 3KB main Main.scala 1KB README.md 1KB circle.yml 387B crawler-test src test resources 
pageWithLinks.html 178B logback-test.xml 792B application.conf 1KB scala app RabbitPublisherSpec.scala 1KB integration StatusCodeFilterSpec.scala 1KB MultiInstanceSpec.scala 1KB DuplicateFilteringSpec.scala 2KB SimpleCrawlSpec.scala 3KB TerminationSinkSpec.scala 1KB models HttpRequestSpec.scala 2KB ESJsonTransformationsSpec.scala 2KB processors DuplicateFlowSpec.scala 1KB StatsCollectorSpec.scala 1KB StatusCodeFilterSpec.scala 887B RequestExtractorSpec.scala 5KB DuplicateFilterSpec.scala 504B RabbitMQSinkSpec.scala 2KB PersistentDuplicateFilterSpec.scala 3KB backend RabbitThrottlerSpec.scala 4KB FrontierSpec.scala 2KB ConfigSpec.scala 401B FrontierThrottlerSpec.scala 3KB ESRabbitRiverTransformerSpec.scala 1KB IntegrationSuite.scala 3KB blikk-crawler.sublime-project 833B docker Dockerfile 316B资源评论
