# resumableds
A Python class that supports Data Science projects.
resumableds supports you in writing data science scripts including save/resume functionality.
Data can be saved and resumed avoiding unnessary retrievals of raw data from data storages.
The data directory structure is inspired by cookiecutter-data-science (https://drivendata.github.io/cookiecutter-data-science/).
The class also supports the statement 'Analysis is a DAG' (https://drivendata.github.io/cookiecutter-data-science/#analysis-is-a-dag).
resumableds is written in pure Python and it is intended to be used within Jupyter notebooks.
It however can also be useful in Python scripts or script pipelines.
### Example
```
proj1 = RdsProject('project1') # create object from class (creates the dir if it doesn't exist yet)
proj1.raw.df1 = pd.DataFrame() # create dataframe as attribute of proj1.raw (RdsFs 'raw')
proj1.defs.variable1 = 'foo' # create simple objects as attribute of proj1.defs (RdsFs 'defs')
proj1.save() # saved attributes of all RfdFs in proj1 to disk
```
This will result in the following directory structure (plus some overhead of internals):
- <output_dir>/defs/var_variable1.pkl
- <output_dir>/raw/df1.pkl
- <output_dir>/raw/df1.csv
Note, pandas dataframes are always dumped as pickle for further processing and as csv for easy exploration. The csv files are never read back anymore.
Later on or in another python session, you can do this:
```
proj2 = RdsProject('project1') # vars and data are read back to their original names
proj2.defs.variable1 == 'foo' # ==> True
isinstance(proj2.raw.df1, pd.DataFrame) # ==> True
```
PyPI 官网下载 | resumableds-0.4.5.tar.gz
版权申诉
75 浏览量
2022-01-29
18:06:25
上传
评论
收藏 12KB GZ 举报
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
最新资源
- Pytorch-pytorch深度学习教程之前馈神经网络.zip
- Pytorch-pytorch深度学习教程之线性回归.zip
- Pytorch-pytorch深度学习教程之基本操作.zip
- 基于QT的地图可视化桌面系统后台数据库为MySQL5.7源码.zip
- 基于simulink的PLL锁相环系统仿真【包括模型,文档,参考文献,操作步骤】
- 基于EM-GMM模型的目标跟踪和异常行为检测matlab仿真【包括程序,注释,参考文献,操作步骤,说明文档】
- 2109010044_胡晨燕_选课管理数据库设计与实现.prj
- 帕鲁介绍的PPT备份没什么好下的
- demo1-202405
- 两种方式修改Intel网卡MAC地址
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈