Skip to content
/ token2index Public

A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and Tensorflow.

token2index.readthedocs.io

License

GPL-3.0 license
51 stars 6 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Kaleidophon/token2index

Error
Looks like something went wrong!

About

A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and Tensorflow.

token2index.readthedocs.io

Topics

python nlp deep-learning tensorflow numpy transformers pytorch indexing transformer rnn seq2seq deeplearning token rnns stoi w2i t2i itos i2t i2w

Resources

Readme

License

GPL-3.0 license
Activity

Stars

51 stars

Watchers

4 watching

Forks

6 forks
Report repository

Releases 4

Extend functionalities to tokenized corpus (#4) Latest
Oct 28, 2020
+ 3 releases

Packages

No packages published

Languages