tensorflow-1.13.0-rc2

# Graph Transform Tool ## Table of Contents * [Introduction](#introduction) * [Using the Graph Transform Tool](#using-the-graph-transform-tool) * [Inspecting Graphs](#inspecting-graphs) * [Common Use Cases](#common-use-cases) * [Optimizing for Deployment](#optimizing-for-deployment) * [Fixing Missing Kernel Errors on Mobile](#fixing-missing-kernel-errors-on-mobile) * [Shrinking File Size](#shrinking-file-size) * [Eight-bit Calculations](#eight-bit-calculations) * [Transform Reference](#transform-reference) * [add_default_attributes](#add_default_attributes) * [backport_concatv2](#backport_concatv2) * [flatten_atrous_conv](#flatten_atrous_conv) * [fold_batch_norms](#fold_batch_norms) * [fold_constants](#fold_constants) * [fold_old_batch_norms](#fold_old_batch_norms) * [freeze_requantization_ranges](#freeze_requantization_ranges) * [fuse_convolutions](#fuse_convolutions) * [insert_logging](#insert_logging) * [merge_duplicate_nodes](#merge_duplicate_nodes) * [obfuscate_names](#obfuscate_names) * [quantize_nodes](#quantize_nodes) * [quantize_weights](#quantize_weights) * [remove_attribute](#remove_attribute) * [remove_device](#remove_device) * [remove_nodes](#remove_nodes) * [rename_attribute](#rename_attribute) * [rename_op](#rename_op) * [round_weights](#round_weights) * [sparsify_gather](#sparsify_gather) * [set_device](#set_device) * [sort_by_execution_order](#sort_by_execution_order) * [strip_unused_nodes](#strip_unused_nodes) * [Writing Your Own Transforms](#writing-your-own-transforms) * [Transform Functions](#transform-functions) * [Pattern Syntax](#pattern-syntax) * [ReplaceMatchingOpTypes](#replacematchingoptypes) * [Parameters](#parameters) * [Function Libraries](#function-libraries) * [Registering](#registering) ## Introduction When you have finished training a model and want to deploy it in production, you'll often want to modify it to better run in its final environment. For example if you're targeting a phone you might want to shrink the file size by quantizing the weights, or optimize away batch normalization or other training-only features. The Graph Transform framework offers a suite of tools for modifying computational graphs, and a framework to make it easy to write your own modifications. This guide is structured into three main parts, first giving some tutorials on how to perform common tasks, second a reference covering all of the different transformations that are included, together with the options that apply to them, and third a guide to creating your own transforms. ## Using the Graph Transform Tool The Graph Transform tool is designed to work on models that are saved as GraphDef files, usually in a binary protobuf format. This is the low-level definition of a TensorFlow computational graph, including a list of nodes and the input and output connections between them. If you're using a Python API to train your model, this will usually be saved out in the same directory as your checkpoints, and usually has a '.pb' suffix. If you want to work with the values of your trained parameters, for example to quantize weights, you'll need to run [tensorflow/python/tools/freeze_graph.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py) to convert the checkpoint values into embedded constants within the graph file itself. You call the Graph Transform tool itself like this: ```bash bazel build tensorflow/tools/graph_transforms:transform_graph bazel-bin/tensorflow/tools/graph_transforms/transform_graph \ --in_graph=tensorflow_inception_graph.pb \ --out_graph=optimized_inception_graph.pb \ --inputs='Mul:0' \ --outputs='softmax:0' \ --transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") remove_nodes(op=Identity, op=CheckNumerics) fold_old_batch_norms ' ``` The arguments here are specifying where to read the graph from, where to write the transformed version to, what the input and output layers are, and what transforms to modify the graph with. The transforms are given as a list of names, and can each have arguments themselves. These transforms define the pipeline of modifications that are applied in order to produce the output. Sometimes you need some transforms to happen before others, and the ordering within the list lets you specify which happen first. Note that the optimization `remove_nodes(op=Identity, op=CheckNumerics)` will break the model with control flow operations, such as `tf.cond`, `tf.map_fn`, and `tf.while`. ## Inspecting Graphs Many of the transforms that the tool supports need to know what the input and output layers of the model are. The best source for these is the model training process, where for a classifier the inputs will be the nodes that receive the data from the training set, and the output will be the predictions. If you're unsure, the [`summarize_graph`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/graph_transforms/summarize_graph_main.cc) tool can inspect the model and provide guesses about likely input and output nodes, as well as other information that's useful for debugging. Here's an example of how to use it on the [Inception V3 graph](http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz): ```bash bazel build tensorflow/tools/graph_transforms:summarize_graph bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=tensorflow_inception_graph.pb ``` ## Common Use Cases This section has small guides for some of the most frequently-used transformation pipelines, aimed at users who want to quickly accomplish one of these tasks. A lot of them will use the Inception V3 model for their examples, which can be downloaded from [http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz](http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz). ### Optimizing for Deployment If you've finished training your model and want to deploy it on a server or a mobile device, you'll want it to run as fast as possible, and with as few non-essential dependencies as you can. This recipe removes all of the nodes that aren't called during inference, shrinks expressions that are always constant into single nodes, and optimizes away some multiply operations used during batch normalization by pre-multiplying the weights for convolutions. ```bash bazel build tensorflow/tools/graph_transforms:transform_graph bazel-bin/tensorflow/tools/graph_transforms/transform_graph \ --in_graph=tensorflow_inception_graph.pb \ --out_graph=optimized_inception_graph.pb \ --inputs='Mul' \ --outputs='softmax' \ --transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") remove_nodes(op=Identity, op=CheckNumerics) fold_constants(ignore_errors=true) fold_batch_norms fold_old_batch_norms' ``` The batch norm folding is included twice because there are two different flavors of batch normalization used in TensorFlow. The older version was implemented with a single BatchNormWithGlobalNormalization op, but it was deprecated in favor of a more recent approach using individual ops to implement the same computation. The two transforms are in there so that both styles are recognized and optimized. ### Fixing Missing Kernel Errors on Mobile The mobile version of TensorFlow is focused on inference, and so by default the list of supported ops (defined in [tensorflow/core/kernels/BUILD:android_extended_ops](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/BUILD) for Bazel and [tensorflow/contrib/makefile/tf_op_files.txt](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/makefile/tf_op_files.txt) for make builds) doesn't include a lot that are training related. This can cause `No OpKernel was registered to support Op`