用于适配低版本transformers的tokenizer

解决问题： TypeError: TextEncodeInput must be Union[TextInputSequence,Tupele[InputSequence, InputSequence]] 使用方法： pip install transformers_old_tokenizer-3.1.0-py3-none-any.whl from transformers_old_tokenizer import AutoTokenizer