BERT encoding is very slow - Help

4 次查看（过去 30 天）
Zzz
Zzz 2021-5-7
回答： Ralf Elsas 2023-2-26
I've been following this github: https://github.com/matlab-deep-learning/transformer-models which is the MATLAB implementation of BERT.
While trying to encode my text using the tokenizer, following this script, I realize that BERT encoding takes very long to work on my dataset.
My dataset contains 1000+ text entries, each of which is ~1000 in length. I noticed that the example csv used in the github contains very short description text. My question is: how can we perform text preprocessing using BERT encoding? And how we can speed up the encoding process?
Thanks!

请先登录，再回答此问题。

采纳的回答

Divya Gaddipati
Divya Gaddipati 2021-5-13
Here are a few things that you can try to speed up the tokenizer, which were suggested by the GitHub repo author (you can also find this information here):
1. Remove redundant white-space tokenization in BasicTokenizer
2. Convert basic tokenized tokens to UTF32 in one call in FullTokenizer, and modify WordPieceTokenizer to accept UTF32 as input.
3. Only call sub.string() once in WordPieceTokenizer.
4. Remove input validation in WhitespaceTokenizer which may be called many times.
If the issue still exists, you could also create a new issue on the GitHub page itself.

更多回答（1 个）

Ralf Elsas
Ralf Elsas 2023-2-26
Hello! For everybody dealing with this issue - it can be easily solved: fastBERTtokens

类别

在 Help Center 和 File Exchange 中查找有关 Modeling and Prediction 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by