# geo-spider
crawl all GEO metadata, features:
1. crawl platforms
2. crawl samples
3. crawl series
4. incremental crawling
Table of Contents
1. [installation](#installation)
2. [output file format](#output-file-format)
3. [logs](#logs)
4. [platforms](#platforms)
- [denovo crawling](#platforms-denovo-crawling)
- [incremental crawling](#platforms-incremental-crawling)
5. [samples](#samples)
- [denovo crawling](#samples-denovo-crawling)
- [incremental crawling](#samples-incremental-crawling)
6. [series](#series)
- [denovo crawling](#series-denovo-crawling)
- [incremental crawling](#series-incremental-crawling)
## installation
```
pip install geo-spider
```
## output file format
geo-spider saves files in jsonlines form,
Refer to [this site](https://jsonlines.org/) for details.
## logs
geo-spider default generate logs to geo-spider.log(current directory)
in WARNING level, you can customize by `-d` and `-l` options.
1. `-d` to enable debug mode
2. `-l` specify customized log file
```
geo-spider -d -l new-geo-spider.log <sub-command>
```
## platforms
### platforms denovo crawling
```
geo-spider platforms -o platforms.jl
```
### platforms incremental crawling
If you have a crawled platforms jsonlines file:
```
geo-spider platforms -cf platforms.jl -o new-platforms.jl
```
If you have multiple platforms jsonlines files:
```
geo-spider platforms -cd platforms -o new-platforms.jl
```
## samples
### samples denovo crawling
### samples incremental crawling
### series
### series denovo crawling
### series incremental crawling
PyPI 官网下载 | geo-spider-0.0.2.tar.gz
版权申诉
178 浏览量
2022-01-27
19:47:22
上传
评论
收藏 4KB GZ 举报
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
最新资源
- 基于JavaScript和CSS的随寻订购网页设计源码 - web-order
- 基于MATLAB的声纹识别系统设计源码 - VoiceprintRecognition
- 基于Java的微服务插件集合设计源码 - wsy-plugins
- 基于Vue和微信小程序的监理日志系统设计源码 - supervisionLog
- 基于Java和LCN分布式事务框架的设计源码 - tx-lcn
- 基于Java和JavaScript的茶叶评级管理系统设计源码 - tea
- IMG_5680.JPG
- IMG_0437.jpg
- 基于Java的JAVA项目分析工具设计源码 - JAVAProjectAnalysis
- top888.json
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈