# TensorFlow Datasets
TensorFlow Datasets provides many public datasets as `tf.data.Datasets`.
[![Kokoro](https://storage.googleapis.com/tfds-kokoro-public/kokoro-build.svg)](https://storage.googleapis.com/tfds-kokoro-public/kokoro-build.html)
[![PyPI version](https://badge.fury.io/py/tensorflow-datasets.svg)](https://badge.fury.io/py/tensorflow-datasets)
* [List of datasets](https://github.com/tensorflow/datasets/tree/master/docs/datasets.md)
* [Try it in Colab](https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/overview.ipynb)
* [API docs](https://www.tensorflow.org/datasets/api_docs/python/tfds)
* [Add a dataset](https://github.com/tensorflow/datasets/tree/master/docs/add_dataset.md)
**Table of Contents**
* [Installation](#installation)
* [Usage](#usage)
* [`DatasetBuilder`](#datasetbuilder)
* [NumPy usage](#numpy-usage-with-tfdsas-numpy)
* [Want a certain dataset?](#want-a-certain-dataset)
* [Disclaimers](#disclaimers)
### Installation
```sh
pip install tensorflow-datasets
# Requires TF 1.12+ to be installed.
# Some datasets require additional libraries; see setup.py extras_require
pip install tensorflow
# or:
pip install tensorflow-gpu
```
### Usage
```python
import tensorflow_datasets as tfds
import tensorflow as tf
# tfds works in both Eager and Graph modes
tf.enable_eager_execution()
# See available datasets
print(tfds.list_builders())
# Construct a tf.data.Dataset
ds_train, ds_test = tfds.load(name="mnist", split=["train", "test"])
# Build your input pipeline
ds_train = ds_train.shuffle(1000).batch(128).prefetch(10)
for features in ds_train.take(1):
image, label = features["image"], features["label"]
```
Try it interactively in a
[Colab notebook](https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/overview.ipynb).
### `DatasetBuilder`
All datasets are implemented as subclasses of
[`DatasetBuilder`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetBuilder.md)
and
[`tfds.load`](https://www.tensorflow.org/datasets/api_docs/python/tfds/load.md)
is a thin convenience wrapper.
[`DatasetInfo`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetInfo.md)
documents the dataset.
```python
import tensorflow_datasets as tfds
# The following is the equivalent of the `load` call above.
# You can fetch the DatasetBuilder class by string
mnist_builder = tfds.builder("mnist")
# Download the dataset
mnist_builder.download_and_prepare()
# Construct a tf.data.Dataset
ds = mnist_builder.as_dataset(split=tfds.Split.TRAIN)
# Get the `DatasetInfo` object, which contains useful information about the
# dataset and its features
info = mnist_builder.info
print(info)
tfds.core.DatasetInfo(
name='mnist',
version=1.0.0,
description='The MNIST database of handwritten digits.',
urls=[u'http://yann.lecun.com/exdb/mnist/'],
features=FeaturesDict({
'image': Image(shape=(28, 28, 1), dtype=tf.uint8),
'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10)
},
total_num_examples=70000,
splits={
u'test': <tfds.core.SplitInfo num_examples=10000>,
u'train': <tfds.core.SplitInfo num_examples=60000>
},
supervised_keys=(u'image', u'label'),
citation='"""
@article{lecun2010mnist,
title={MNIST handwritten digit database},
author={LeCun, Yann and Cortes, Corinna and Burges, CJ},
journal={ATT Labs [Online]. Available: http://yann. lecun. com/exdb/mnist},
volume={2},
year={2010}
}
"""',
)
```
### NumPy Usage with `tfds.as_numpy`
As a convenience for users that want simple NumPy arrays in their programs, you
can use
[`tfds.as_numpy`](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_numpy.md)
to return a generator that yields NumPy array
records out of a `tf.data.Dataset`. This allows you to build high-performance
input pipelines with `tf.data` but use whatever you'd like for your model
components.
```python
train_ds = tfds.load("mnist", split=tfds.Split.TRAIN)
train_ds = train_ds.shuffle(1024).batch(128).repeat(5).prefetch(10)
for example in tfds.as_numpy(train_ds):
numpy_images, numpy_labels = example["image"], example["label"]
```
You can also use `tfds.as_numpy` in conjunction with `batch_size=-1` to
get the full dataset in NumPy arrays from the returned `tf.Tensor` object:
```python
train_ds = tfds.load("mnist", split=tfds.Split.TRAIN, batch_size=-1)
numpy_ds = tfds.as_numpy(train_ds)
numpy_images, numpy_labels = numpy_ds["image"], numpy_ds["label"]
```
Note that the library still requires `tensorflow` as an internal dependency.
## Want a certain dataset?
Adding a dataset is really straightforward by following
[our guide](https://github.com/tensorflow/datasets/tree/master/docs/add_dataset.md).
Request a dataset by opening a
[Dataset request GitHub issue](https://github.com/tensorflow/datasets/issues/new?assignees=&labels=dataset+request&template=dataset-request.md&title=%5Bdata+request%5D+%3Cdataset+name%3E).
And vote on the current
[set of requests](https://github.com/tensorflow/datasets/labels/dataset%20request)
by adding a thumbs-up reaction to the issue.
#### *Disclaimers*
*This is a utility library that downloads and prepares public datasets. We do*
*not host or distribute these datasets, vouch for their quality or fairness, or*
*claim that you have license to use the dataset. It is your responsibility to*
*determine whether you have permission to use the dataset under the dataset's*
*license.*
*If you're a dataset owner and wish to update any part of it (description,*
*citation, etc.), or do not want your dataset to be included in this*
*library, please get in touch through a GitHub issue. Thanks for your*
*contribution to the ML community!*
*If you're interested in learning more about responsible AI practices, including*
*fairness, please see Google AI's [Responsible AI Practices](https://ai.google/education/responsible-ai-practices).*
*`tensorflow/datasets` is Apache 2.0 licensed. See the `LICENSE` file.*
没有合适的资源?快使用搜索试试~ 我知道了~
PyPI 官网下载 | tfds-nightly-1.0.1.dev201903250105.tar.gz
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 76 浏览量
2022-01-30
07:38:58
上传
评论
收藏 334KB GZ 举报
温馨提示
共220个文件
py:173个
txt:43个
pkg-info:2个
资源来自pypi官网。 资源全名:tfds-nightly-1.0.1.dev201903250105.tar.gz
资源推荐
资源详情
资源评论
收起资源包目录
PyPI 官网下载 | tfds-nightly-1.0.1.dev201903250105.tar.gz (220个子文件)
setup.cfg 38B
README.md 6KB
PKG-INFO 1KB
PKG-INFO 1KB
dataset_builder.py 33KB
feature.py 21KB
dataset_info.py 20KB
splits.py 19KB
subword_text_encoder.py 17KB
dataset_builder_test.py 17KB
dataset_info_generated_pb2.py 16KB
splits_test.py 16KB
download_manager.py 15KB
text_encoder.py 15KB
file_format_adapter.py 15KB
mnist.py 14KB
test_utils.py 13KB
open_images.py 12KB
dataset_builder_testing.py 12KB
registered.py 12KB
download_manager_test.py 12KB
text_encoder_test.py 11KB
document_datasets.py 11KB
dataset_info_test.py 10KB
coco.py 10KB
sequence_feature.py 9KB
resource.py 9KB
librispeech.py 9KB
features_test.py 9KB
sequence_feature_test.py 8KB
moving_sequence.py 8KB
py_utils.py 8KB
starcraft.py 7KB
celeba.py 7KB
squad.py 7KB
subword_text_encoder_test.py 7KB
wmt.py 7KB
cifar.py 7KB
downloader.py 7KB
translation_feature.py 6KB
flores.py 6KB
ted_hrlr.py 6KB
class_label_feature.py 6KB
file_format_adapter_test.py 6KB
image_feature.py 6KB
lm1b.py 6KB
nsynth.py 6KB
downloader_test.py 6KB
open_images.py 6KB
imagenet.py 6KB
download_and_prepare.py 6KB
dataset_utils.py 6KB
multi_nli.py 6KB
extractor.py 6KB
tf_compat.py 6KB
translation_feature_test.py 6KB
bair_robot_pushing.py 6KB
imdb.py 6KB
tf_utils.py 5KB
image_folder.py 5KB
ted_multi.py 5KB
registered_test.py 5KB
wmt_enfr.py 5KB
chexpert.py 5KB
omniglot.py 5KB
text_feature.py 5KB
dataset_builder_beam_test.py 5KB
class_label_feature_test.py 5KB
dataset_utils_test.py 5KB
extractor_test.py 5KB
colorectal_histology.py 5KB
celebahq.py 4KB
titanic.py 4KB
kaggle.py 4KB
resource_test.py 4KB
cifar.py 4KB
checksums.py 4KB
cats_vs_dogs.py 4KB
quickdraw.py 4KB
diabetic_retinopathy_detection.py 4KB
image_feature_test.py 4KB
caltech.py 4KB
lsun.py 4KB
svhn.py 3KB
naming_test.py 3KB
py_utils_test.py 3KB
setup.py 3KB
moving_mnist.py 3KB
text_feature_test.py 3KB
rock_paper_scissors.py 3KB
wmt_ende.py 3KB
horses_or_humans.py 3KB
imagenet.py 3KB
flowers.py 3KB
util.py 3KB
caltech.py 3KB
lazy_imports.py 3KB
colorectal_histology.py 3KB
starcraft.py 3KB
api_utils.py 3KB
共 220 条
- 1
- 2
- 3
资源评论
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 类和对象知识点练习及其参考答案
- C++职工管理系统:本教程主要利用C++来实现一个基于多态的职工管理系统
- 基于原生微信小程序实现的课堂考勤系统的设计与实现
- 商道融绿、润灵环球ESG评级数据(2015-2023年).xlsx
- 商道融绿、润灵环球ESG评级数据(2015-2023年).dta
- 基于 GDAL 与 PROJ4 的遥感图像处理软件,使用 Qt 构建课程设计
- 图形化界面采用Easyx编写,实现对哈夫曼树的显示操作
- 使用后端开发框架Spring Boot构建应用程序.pdf
- 基于Boson的计算机网络实验:RIP和IGRP的配置
- 在线教育系统 JAVA+Vue+SpringBoot+MySQL
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功