Sentence parsing can be done one word at a time using strtok. However, sometimes it is useful to (efficiently) extract all words into a cell array in one function call. The function allwords.m does exactly this.
Spaces, white space (tabs), carriage returns, and punctuation characters are all valid separator characters by default. In this example, I had a period at the end, as well as multiple spaces between some words.
str = 'The quick brown fox jumped over the lazy dog.';
words = allwords(str)
words =
'The' 'quick' 'brown' 'fox' 'jumped' 'over' 'the' 'lazy' 'dog'
This utility can also work on any integer vector. The default separators for numeric vectors are [-inf inf NaN], but you can assign any separators you desire. Here, parse a string of integers, with only NaN elements as the separator.
str = [1 2 4 2 inf 3 3 5 nan 4 6 5];
words = allwords(str,nan);
words{1}
ans =
1 2 4 2 Inf 3 3 5
words{2}
ans =
4 6 5
Finally, allwords is efficient. For example, on a random numeric string of length 1e6, allwords parses it into over 90000 distinct "words" in less than 0.5 seconds.
str = round(rand(1,1000000)*10);
tic
words = allwords(str,[0 10]);
toc
Elapsed time is 0.455194 seconds.
There were over 90000 different words that were extracted
numel(words)
ans =
90310
The longest word had length 104.
max(cellfun(@numel,words))
ans =
104
引用格式
John D'Errico (2024). allwords (https://www.mathworks.com/matlabcentral/fileexchange/27184-allwords), MATLAB Central File Exchange. 检索来源 .
MATLAB 版本兼容性
平台兼容性
Windows macOS Linux类别
标签
致谢
参考作品: wordcount
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!版本 | 已发布 | 发行说明 | |
---|---|---|---|
1.1.0.0 | Speed enhancement for character strings, plus I added a reference to the wordcount function. |