Problems in transfering SlowFast gesture recognition model to my own task.

Question

Yusheng Hao 2023-4-6

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1942329-problems-in-transfering-slowfast-gesture-recognition-model-to-my-own-task

编辑： Yusheng Hao 2023-4-6

When I transfered SlowFast model to my own task， some problems happened.

Firstly, I can successfully run the case titled by "Gesture Recognition using Videos and Deep Learning" provided by Matlab.

Then, I want transfer the model to my own task. I prepared my own training set as follows:

Compared with default 4 classes (clapping, noAction, somethingElse,wavingHello), there are 6 classes in my task(One, Two, Three, Four, Five, Snap).

Seondly, I began to train the model with my own training set. Everything is ok until the function of createMiniBatchQueue() is called. Specifically, when I run the following code section.

params.ModelFilename = "slowFastPretrained_fourClasses.mat";
if doTraining
    epoch = 1;
    bestLoss = realmax;
    accTrain = [];
    lossTrain = [];
    iteration = 1;
    start = tic;
    trainTime = start;
    shuffled = shuffleTrainDs(dsTrain);
    % Number of outputs is two: One for RGB frames, and one for ground truth labels.
    numOutputs = 2;
    mbq = createMiniBatchQueue(shuffled, numOutputs, params);
    
    % Use the initializeTrainingProgressPlot and initializeVerboseOutput
    % supporting functions, listed at the end of the example, to initialize
    % the training progress plot and verbose output to display the training
    % loss, training accuracy, and validation accuracy.
    plotters = initializeTrainingProgressPlot(params);
    initializeVerboseOutput(params);
    while iteration <= params.NumIterations
        % Iterate through the data set.
        [dlX1,dlY] = next(mbq);
        % Evaluate the model gradients and loss using dlfeval.
        [gradients,loss,acc,state] = ...
            dlfeval(@modelGradients,slowFast,dlX1,dlY);
        % Accumulate the loss and accuracies.
        lossTrain = [lossTrain, loss];
        accTrain = [accTrain, acc];
        % Update the network state.
        slowFast.State = state;
        % Update the gradients and parameters for the video classifier
        % using the SGDM optimizer.
        [slowFast,params.Velocity,learnRate] = ...
            updateLearnables(slowFast,gradients,params,params.Velocity,iteration);
        if ~hasdata(mbq) || iteration == params.NumIterations
            % Current epoch is complete. Do validation and update progress.
            trainTime = toc(trainTime);
            accTrain = mean(accTrain);
            lossTrain = mean(lossTrain);
            % Update the training progress.
            displayVerboseOutputEveryEpoch(params,start,learnRate,epoch,iteration,...
                accTrain,lossTrain,trainTime);
            updateProgressPlot(params,plotters,epoch,iteration,start,lossTrain,accTrain);
            % Save the trained video classifier and the parameters, that gave 
            % the best training loss so far. Use the saveData supporting function,
            % listed at the end of this example.
            bestLoss = saveData(slowFast,bestLoss,iteration,lossTrain,params);
        end
        if ~hasdata(mbq) && iteration < params.NumIterations
            % Current epoch is complete. Initialize the training loss, accuracy
            % values, and minibatchqueue for the next epoch.
            accTrain = [];
            lossTrain = [];
            
            epoch = epoch + 1;
            trainTime = tic;
            shuffled = shuffleTrainDs(dsTrain);
            mbq = createMiniBatchQueue(shuffled, numOutputs, params);            
        end
        iteration = iteration + 1;
    end
    % Display a message when training is complete.
    endVerboseOutput(params);
    disp("Model saved to: " + params.ModelFilename);
end