Generate Text with Deep Learning "Invalid training data. Labels must not contain undefined values" ERROR

Question

mark palmer 2022-11-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1860303-generate-text-with-deep-learning-invalid-training-data-labels-must-not-contain-undefined-values-e

评论： mark palmer 2022-12-1

I am using the Generate Text with Deep Learning Matlab example, here

https://www.mathworks.com/help/deeplearning/ug/generate-text-using-deep-learning.html

It works fine when I use the Shakespeare text provided in the example, but none of my texts are accepted. I always get the error: "Invalid training data. Labels must not contain undefined values."

My text and code provided below.

filename = 'RWE Nature.txt';
textData = fileread(filename);
textData = replace(textData,"  ","");
textData = split(textData,[newline]); % USE NEWLINE TO SPLIT TEXT INTO CELLS
% textData = textData(5:2:end);
textData(1:5) % 154 X 1 string array
startOfTextCharacter = compose("\x0002");
whitespaceCharacter = compose("\x00B7");
endOfTextCharacter = compose("\x2403");
newlineCharacter = compose("\x00B6");
textData = startOfTextCharacter + textData;
textData = replace(textData,[" " newline],[whitespaceCharacter newlineCharacter]);
uniqueCharacters = unique([textData{:}]); % '!'(),-.:;?ABCDEFGHIJKLMNOPRSTUVWYabcdefghijklmnopqrstuvwxyz¶·'
numUniqueCharacters = numel(uniqueCharacters); % 62
%
numDocuments = numel(textData); % 154 SONNETS, 89 PARAGRAPHS IN MAYER
XTrain = cell(1,numDocuments);
YTrain = cell(1,numDocuments);
for i = 1:numel(textData)
    characters = textData{i};
    sequenceLength = numel(characters);
    
    % Get indices of characters.
    [~,idx] = ismember(characters,uniqueCharacters);
    
    % Convert characters to vectors.
    X = zeros(numUniqueCharacters,sequenceLength);
    for j = 1:sequenceLength
        X(idx(j),j) = 1;
    end
    
    % Create vector of categorical responses with end of text character.
    charactersShifted = [cellstr(characters(2:end)')' endOfTextCharacter];
    Y = categorical(charactersShifted);
    
    XTrain{i} = X;
    YTrain{i} = Y;
end
% textData{1}
inputSize = size(XTrain{1},1);
numHiddenUnits = 200;
numClasses = numel(categories([YTrain{:}]));
layers = [
    sequenceInputLayer(inputSize)
    lstmLayer(numHiddenUnits,'OutputMode','sequence')
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];
options = trainingOptions('adam', ...
    'MaxEpochs',500, ...
    'InitialLearnRate',0.01, ...
    'GradientThreshold',2, ...
    'MiniBatchSize',77,...
    'Shuffle','every-epoch', ...
    'Plots','training-progress', ...
    'Verbose',false);
% Train the network.
'a'
net = trainNetwork(XTrain,YTrain,layers,options);
'b'
% Generate text using the trained network.
generatedText = generateText(net,uniqueCharacters,startOfTextCharacter,newlineCharacter,whitespaceCharacter,endOfTextCharacter)
'end'
function generatedText = generateText(net,uniqueCharacters,startOfTextCharacter,newlineCharacter,whitespaceCharacter,endOfTextCharacter)
    numUniqueCharacters = numel(uniqueCharacters);
    X = zeros(numUniqueCharacters,1);
    idx = strfind(uniqueCharacters,startOfTextCharacter);
    X(idx) = 1;
    generatedText = "";
    vocabulary = string(net.Layers(end).Classes);
    maxLength = 500;
    while strlength(generatedText) < maxLength
        % Predict the next character scores.
        [net,characterScores] = predictAndUpdateState(net,X,'ExecutionEnvironment','cpu');
        % Sample the next character.
        newCharacter = datasample(vocabulary,1,'Weights',characterScores);
        % Stop predicting at the end of text.
        if newCharacter == endOfTextCharacter
            break
        end
        % Add the character to the generated text.
        generatedText = generatedText + newCharacter;
        % Create a new vector for the next input.
        X(:) = 0;
        idx = strfind(uniqueCharacters,newCharacter);
        X(idx) = 1;
    end
    generatedText = replace(generatedText,[newlineCharacter whitespaceCharacter],[newline " "]);
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ben 2022-11-28

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1860303-generate-text-with-deep-learning-invalid-training-data-labels-must-not-contain-undefined-values-e#answer_1114103

There are a few issues to fix this:

The call to Y = categorical(charactersShifted) needs to include a valueset that includes all the unique characters in your dataset, Y = categorical(charactersShifted,allUniqueCharacters)
To make that work with the uniqueCharacters variable you need to convert it to the same class as charactersShifted, a string.
The endOfTextCharacter will need to be included too, otherwise it'll become an <undefined> category in Y.
Finally the logic charactersShifted = [cellstr(characters(2:end)')' endOfTextCharacter]; might prepend an empty "" when characters was only 1 character long. That will make Y have length 2, but X have length 1 and you'll get a sequence length mismatch when you try to train.

I think training should work once you resolve these things. Hope that helps.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

mark palmer 2022-12-1

(Ironically?) I get an error message when I try to Accept your answer, but thanks, I'll give this all a try.

请先登录，再进行评论。

Generate Text with Deep Learning "Invalid training data. Labels must not contain undefined values" ERROR

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Generate Text with Deep Learning "Invalid training data. Labels must not contain undefined values" ERROR

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论