How to do one hot encoding of unusual letters in matlab?

2 次查看（过去 30 天）
Sanjana Sankar
Sanjana Sankar 2019-7-19
评论： Sanjana Sankar 2019-7-31
I have a table of german characters. I would like to one hot encode them so that i can input it to a neural network. How should i go about?
The problem is some of the characters are not accepted by matlab as characters. For example, 'ä' 'ö' 'ü' 'ß'
Regardless, I would like to know how to one hot encode any character from a TABLE in matlab.
Thanks in advance!

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2019-7-19
编辑：Walter Roberson 2019-7-19
https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/ describes One-Hot Encoding (a term I was not aware of)
In MATLAB, see https://www.mathworks.com/help/deeplearning/ref/vec2ind.html
You might want to first construct a list of permitted characters, and map the input into an offset in that list. That will potentially save you from wasting bits on characters such as Œœ that you are not using.ŒŒ
  5 个评论
显示 3更早的评论
Walter Roberson
Walter Roberson 2019-7-30
permitted = ['A' : 'Z', 'a' : 'z', 'ä', 'ö', 'ü', 'ß', ' ', '.' ] ;
[found, idx] = ismember(YourText, permitted);
assert(all(found), 'unpermitted character detected')
OneHot = ind2vec(idx) ;
Sanjana Sankar
Sanjana Sankar 2019-7-31
OneHot = full(ind2vec(idx,max(idx)));
This is exactly what I was looking for and I got it from your code. Thank You! This works :)

请先登录，再进行评论。

更多回答（1 个）

Guillaume
Guillaume 2019-7-30
file containing the characters that I want to one hot encode. Do you know how I can go about now
%phoneme_set: A cell array of phonemes to one-hot encode
assert(numel(phoneme_set) > 64, 'Cannot one-hot encode more than 64 phonemes with a 64-bit integer')
phoneme_set(:, 2) = num2cell(2 .^ uint64(0:size(phoneme_set, 1)-1))
  3 个评论
显示 1更早的评论
Guillaume
Guillaume 2019-7-30
number binary pattern (64 bits)
2^0 0000000000000000000000000000000000000000000000000000000000000001
2^1 0000000000000000000000000000000000000000000000000000000000000010
2^2 0000000000000000000000000000000000000000000000000000000000000100
...
2^63 1000000000000000000000000000000000000000000000000000000000000000
This is what one-hot encoding is. As I commented, this is useful for FPGAs and similar which operate at the bit level. On the generic processor of a computer, it's a complete waste of space but my answer does what you asked.
You could encode the one-hot encoded numbers that I generate as a vector of 0 and 1 (double) for even more waste of space:
phoneme_set(:, 3) = num2cell(fliplr(eye(size(phoneme_set, 1), 64)), 2)
Note that the binary pattern of the 0s in that encoding is 0000000000000000000000000000000000000000000000000000000000000000b and of the 1s is 0011111111110000000000000000000000000000000000000000000000000000b.
However, I suspect that what we have here is a XY problem. You have some unspecified problem doing something and you think you can solve it by using one-hot encoding (without really understanding what it means) so ask about one-hot encoding instead of your actual problem.
Sanjana Sankar
Sanjana Sankar 2019-7-30
I have to feed characters to a neural network. And the most common method is one hot encoding. But I haven't found any examples on how it is done in MATLAB.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Gaussian Process Regression 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by