# file_io
Deterministic File Lib to make working with Files across Object Storage easier
## Quickstart
```python3
!pip install --upgrade git+https://github.com/trisongz/file_io.git
!pip install --upgrade file-io
from fileio import File, Auth
# Auth object is for setting ADC if needed
Auth(adc='/path/to/adc.json')
'''
Recognized File Extensions
.json - json
.jsonl/.jsonlines - jsonlines
.csv - csv
.tsv - tsv with "\t" seperator
.txt - txtlines
.pkl - pickle
.pt - pytorch
.tfrecords - tensorflow
'''
# Main auto classes
File.open(filename, mode='r', auto=True, device=None) # device is specific to pytorch. Set auto=False to get a barebones Posix via Gfile
File.save(data, filename, overwrite=False) # if not overwrite, will attempt to append for newline files
File.load(filenames, device=None) # yields generators per file, meaning you can have different file types
File.download(url, dirpath=None, filename=None, overwrite=False) # Downloads a single url
File.gdown(url, extract=True, verbose=False) # uses gdown lib to grab a google drive drive
# Main i/o classes (Not Binary)
File.read(filename) # 'r'
File.write(filename) # 'w'
File.append(filename) # 'a'
# Binary
File.wb(filename) # 'wb'
File.rb(filename) # 'rb'
# Batch downloaders
File.batch_download(urls, directory=None, overwrite=False) # downloads all urls into a directory, skipping if overwrite = True and exists
File.batch_gdown(urls, directory=None, extract=True, verbose=False) # downloads all gdrive urls to a directory
# Extension Specific
# .json
File.jsonload(filename)
File.jsondump(dict, filename)
# .jsonl/.jsonlines (Single File)
File.jlg(filename)
File.jlw(data, filename, mode='auto', verbose=True)
# Multifile Readers
# .jsonl/.jsonlines
File.jgs(filenames)
```
### Changelogs
May 21, 2021 - v0.1.5
- Added TSV/CSV Write Methods
- File.csvwrite
- File.tsvwrite
May 20, 2021 - v0.1.4
- Hotfix for file.split_file(s) method to also return resulting filenames with `output_files` key
May 20, 2021 - v0.1.3
- Py Version Requirement Fix
May 19, 2021 - v0.1.2
- Minor Fixes
- Added Methods for Splitting Files/Items
- File.calc_splits
- File.split_items
- File.split_file
- File.split_files
May 12, 2021 - v0.1.1
- Minor Fixes
- Added Method
- File.fmv
May 12, 2021 - v0.1.0
- Refactored Library
- Organized Methods
- Added MultiThreaded Wrapper
- `from fileio import MultiThreadPipeline`
- Added gsutil wrapper method
- File.gsutil
- Added Methods for Yaml
- File.yload
- File.yloads
- File.ydump
- File.ydumps
- File.yparse
- Updated Methods for Json
- File.jsonload
- File.jsonloads
- File.jsondump
- File.jsondumps
- File.jp
- File.jwrite
- File.jg
- File.jgs
- Updated Methods for Jsonlines
- File.jll
- File.jlp
- File.jldumps
- File.jlwrite
- File.jlwrites
- File.jlg
- File.jlgs
- File.jlload
- File.jlw
- File.jlsample
- Updated Methods for Text
- File.textload
- File.textwrite
- File.textread
- File.textlist
- Added Methods for Requests
- File.rget
- File.rpost
- File.reqsess
- Added Methods for URL Encoding/Decoding
- File.urlencode
- File.urldecode
- Added Methods for Hashing
- File.hash
- File.checkhash
- Added Methods to Disable/Enable TQDM
- File.enable_progress
- File.disable_progress
- Added Utility Methods
- File.cat
- File.backup
- File.findir
- File.append_ext
- File.copydir
- File.dirglob
- File.absdir
- File.get_local
- File.finalize
- File.print
- File.set_printer
- Fixed/Updated Methods
- File.isfile
- File.download
- File.batch_download
- File.pexists
- File.whichpath
- File.copy
- File.bcopy
- Added TFDSIODataset
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
共17个文件
py:9个
txt:4个
pkg-info:2个
资源分类:Python库 所属语言:Python 资源全名:file_io-0.1.5.tar.gz 资源来源:官方 安装方法:https://lanzao.blog.csdn.net/article/details/101784059
资源推荐
资源详情
资源评论
收起资源包目录
file_io-0.1.5.tar.gz (17个子文件)
file_io-0.1.5
PKG-INFO 6KB
fileio
src
__init__.py 37KB
__init__.py 916B
_version.py 105B
utils
multi.py 2KB
__init__.py 291B
ops.py 4KB
logging.py 3KB
ds.py 3KB
setup.cfg 38B
setup.py 1KB
README.md 4KB
file_io.egg-info
PKG-INFO 6KB
requires.txt 170B
SOURCES.txt 343B
top_level.txt 7B
dependency_links.txt 1B
共 17 条
- 1
资源评论
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功