# Keras Position Embedding
[![Travis](https://travis-ci.org/CyberZHG/keras-pos-embd.svg)](https://travis-ci.org/CyberZHG/keras-pos-embd)
[![Coverage](https://coveralls.io/repos/github/CyberZHG/keras-pos-embd/badge.svg?branch=master)](https://coveralls.io/github/CyberZHG/keras-pos-embd)
Position embedding layers in Keras.
## Install
```bash
pip install keras-pos-embd
```
## Usage
### Trainable Embedding
```python
import keras
from keras_pos_embd import PositionEmbedding
model = keras.models.Sequential()
model.add(PositionEmbedding(
input_shape=(None,),
input_dim=10, # The maximum absolute value of positions.
output_dim=2, # The dimension of embeddings.
mask_zero=10000, # The index that presents padding (because `0` will be used in relative positioning).
name='Pos-Embd',
))
model.compile('adam', keras.losses.mae, {})
model.summary()
```
(Note that you don't need to enable `mask_zero` if you would concatenate other layers like word embeddings with masks)
### Sin & Cos Embedding
The [sine and cosine embedding](https://arxiv.org/pdf/1706.03762) has no trainable weights. The layer has three modes, it works just like `PositionEmbedding` in `expand` mode:
```python
import keras
from keras_pos_embd import TrigPosEmbedding
model = keras.models.Sequential()
model.add(TrigPosEmbedding(
input_shape=(None,),
output_dim=30, # The dimension of embeddings.
mode=TrigPosEmbedding.MODE_EXPAND, # Use `expand` mode
name='Pos-Embd',
))
model.compile('adam', keras.losses.mae, {})
model.summary()
```
If you want to add this embedding to existed embedding, then there is no need to add a position input in `add` mode:
```python
import keras
from keras_pos_embd import TrigPosEmbedding
model = keras.models.Sequential()
model.add(TrigPosEmbedding(
input_shape=(None, 100),
mode=TrigPosEmbedding.MODE_ADD, # Use `add` mode (default)
name='Pos-Embd',
))
model.compile('adam', keras.losses.mae, {})
model.summary()
```
PyPI 官网下载 | keras-pos-embd-0.10.0.tar.gz
版权申诉
5 浏览量
2022-01-12
18:28:25
上传
评论
收藏 5KB GZ 举报
挣扎的蓝藻
- 粉丝: 12w+
- 资源: 15万+