PyPI 官网下载 | torchlayers-nightly-1622940382.tar.gz

 -------------------------------------------------------------------------------- | Version | Docs | Tests | Coverage | Style | PyPI | Python | PyTorch | Docker | |---------|------|-------|----------|-------|------|--------|---------|--------| | [](https://github.com/szymonmaszke/torchlayers/releases) | [](https://szymonmaszke.github.io/torchlayers/) |  | [](https://codecov.io/gh/szymonmaszke/torchlayers) | [](https://codebeat.co/projects/github-com-szymonmaszke-torchlayers-master) | [](https://pypi.org/project/torchlayers/) | [](https://www.python.org/) | [](https://pytorch.org/) | [](https://hub.docker.com/r/szymonmaszke/torchlayers) | [__torchlayers__](https://szymonmaszke.github.io/torchlayers/) is a library based on [__PyTorch__](https://pytorch.org/) providing __automatic shape and dimensionality inference of `torch.nn` layers__ + additional building blocks featured in current SOTA architectures (e.g. [Efficient-Net](https://arxiv.org/abs/1905.11946)). Above requires no user intervention (except single call to `torchlayers.build`) similarly to the one seen in [__Keras__](https://www.tensorflow.org/guide/keras). ### Main functionalities: * __Shape inference__ for most of `torch.nn` module (__convolutional, recurrent, transformer, attention and linear layers__) * __Dimensionality inference__ (e.g. `torchlayers.Conv` working as `torch.nn.Conv1d/2d/3d` based on `input shape`) * __Shape inference of custom modules__ (see examples section) * __Additional [Keras-like](https://www.tensorflow.org/guide/keras) layers__ (e.g. `torchlayers.Reshape` or `torchlayers.StandardNormalNoise`) * __Additional SOTA layers__ mostly from ImageNet competitions (e.g. [PolyNet](https://arxiv.org/abs/1608.06993), [Squeeze-And-Excitation](https://arxiv.org/abs/1709.01507), [StochasticDepth](www.arxiv.org/abs/1512.03385>)) * __Useful defaults__ (`"same"` padding and default `kernel_size=3` for `Conv`, dropout rates etc.) * __Zero overhead and [torchscript](https://pytorch.org/docs/stable/jit.html) support__ __Keep in mind this library works almost exactly like PyTorch originally__. What that means is you can use `Sequential`, __define your own networks of any complexity using `torch.nn.Module`__, create new layers with shape inference etc. _See below to get some intuition about library_. # Examples For full functionality please check [__torchlayers documentation__](https://szymonmaszke.github.io/torchlayers/). Below examples should introduce all necessary concepts you should know. ## Basic classifier __All__ `torch.nn` modules can be used through `torchlayers` and __each module with input shape__ will be appropriately modified with it's input inferable counterpart. ```python import torchlayers as tl class Classifier(tl.Module): def __init__(self): super().__init__() self.conv1 = tl.Conv2d(64, kernel_size=6) self.conv2 = tl.Conv2d(128, kernel_size=3) self.conv3 = tl.Conv2d(256, kernel_size=3, padding=1) # New layer, more on that in the next example self.pooling = tl.GlobalMaxPool() self.dense = tl.Linear(10) def forward(self, x): x = torch.relu(self.conv1(x)) x = torch.relu(self.conv2(x)) x = torch.relu(self.conv3(x)) return self.dense(self.pooling(x)) # Pass model and any example inputs afterwards clf = tl.build(Classifier(), torch.randn(1, 3, 32, 32)) ``` Above `torchlayers.Linear(out_features=10)` is used. It is "equivalent" to original PyTorch's `torch.nn.Linear(in_features=?, out_features=10)` where `in_features` will be inferred from example input input during `torchlayers.build` call. Same thing happens with `torch.nn.Conv2d(in_channels, out_channels, kernel_size, ...)` which can be replaced directly by `tl.Conv2d(out_channels, kernel_size, ...)`. __Just remember to pass example input through the network!__ ## Simple image and text classifier in one! * We will use single "model" for both tasks. Firstly let's define it using `torch.nn` and `torchlayers`: ```python import torch import torchlayers as tl # torch.nn and torchlayers can be mixed easily model = torch.nn.Sequential( tl.Conv(64), # specify ONLY out_channels torch.nn.ReLU(), # use torch.nn wherever you wish tl.BatchNorm(), # BatchNormNd inferred from input tl.Conv(128), # Default kernel_size equal to 3 tl.ReLU(), tl.Conv(256, kernel_size=11), # "same" padding as default tl.GlobalMaxPool(), # Known from Keras tl.Linear(10), # Output for 10 classes ) print(model) ``` Above would give you model's summary like this (__notice question marks for not yet inferred values__): ```python Sequential( (0): Conv(in_channels=?, out_channels=64, kernel_size=3, stride=1, padding=same, dilation=1, groups=1, bias=True, padding_mode=zeros) (1): ReLU() (2): BatchNorm(num_features=?, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (3): Conv(in_channels=?, out_channels=128, kernel_size=3, stride=1, padding=same, dilation=1, groups=1, bias=True, padding_mode=zeros) (4): ReLU() (5): Conv(in_channels=?, out_channels=256, kernel_size=11, stride=1, padding=same, dilation=1, groups=1, bias=True, padding_mode=zeros) (6): GlobalMaxPool() (7): Linear(in_features=?, out_features=10, bias=True) ) ``` * Now you can __build__/instantiate your model with example input (in this case MNIST-like): ```python mnist_model = tl.build(model, torch.randn(1, 3, 28, 28)) ``` * Or if it's text classification you are after, same model could be built with different `input shape` (e.g. for text classification using `300` dimensional pretrained embedding): ```python # [batch, embedding, timesteps], first dimension > 1 for BatchNorm1d to work text_model = tl.build(model, torch.randn(2, 300, 1)) ``` * Finally, you can `print` both models after instantiation, provided below side by-side for readability (__notice different dimenstionality, e.g. `Conv2d` vs `Conv1d` after `torchlayers.build`__): ```python # TEXT CLASSIFIER MNIST CLASSIFIER Sequential( Sequential( (0): Conv1d(300, 64) (0): Conv2d(3, 64) (1): ReLU() (1): ReLU() (2): BatchNorm1d(64) (2): BatchNorm2d(64) (3): Conv1d(64, 128) (3): Conv2d(64, 128) (4): ReLU() (4): ReLU() (5): Conv1d(128, 256) (5): Conv2d(128, 256) (6): GlobalMaxPool() (6): GlobalMaxPool() (7): Linear(256, 10) (7): Linear(256, 10) ) ) ``` As you can see both modules "compiled" into original `pytorch` layers. ## Custom modules with shape inference capabilities User can define any module and make it shape inferable with `torchlayers.infer` function: ```python # Class