No changes when using function erasePunctuation to remove digits.

2 次查看（过去 30 天）
Ismat Mohd Sulaiman
评论： Ismat Mohd Sulaiman 2021-7-5
I'm trying to remove the digits in my document that has been tokenized.
However, using the erasePunctuation function, I didn't see any changes (no digits were removed) to the updated document. I've checked the type, and the tokenizer does recognize these tokens as digits. Please help. Thanks.
The output:

请先登录，再回答此问题。

回答（1 个）

Cris LaPierre
Cris LaPierre 2021-3-16
编辑：Cris LaPierre 2021-3-16
erasePunctuation still only erases punctuation, not numbers. The 'digits' specification tells it what type of token to remove punctuation from. See the description here.
You could try to remove digits using the following.
tkD = tokenDetails(cleanDoc);
cleanDoc = removeWords(cleanDoc,tkD{tkD.Type=="digits"});

类别

在 Help Center 和 File Exchange 中查找有关 Text Data Preparation 的更多信息

标签

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by