Removing a character from a table (that's within a struct)

Question

Yonathan Zarkovian 2021-10-7

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1468921-removing-a-character-from-a-table-that-s-within-a-struct

评论： Yonathan Zarkovian 2021-10-13

faceDatasetGroundTruth.mat

Hi,

I have a .mat file with the name faceDatasetGroundTruth.mat (a struct) that contains a table (called 'faceDataset') with two columns.

Each value in the second column starts and ends with a ' character, which I'd like to remove so that I'm only left with [95,71,226,313] in the first row, for example.

Thanks in advance.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

C B 2021-10-7

编辑：C B 2021-10-8

在 MATLAB Online 中打开

Is there any specific reason for doing so ?

because when i run following it shows me that first char is '[' and not '''

>> faceDataset.face{1}(1)
ans =
    '['

>> faceDataset
faceDataset =
  3×2 table
    filename       face    
    ________    ___________
      'ab'      '[1 2 2 4]'
      'bc'      '[1 2 2 4]'
      'cd'      '[1 2 2 4]'
>> faceDataset.face{1}
ans =
    '[1 2 2 4]'
>> faceDataset.face{1}(1)
ans =
    '['
>> faceDataset.face{1}(end)
ans =
    ']'

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Stephen23 2021-10-8

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1468921-removing-a-character-from-a-table-that-s-within-a-struct#answer_803956

编辑：Stephen23 2021-10-8

在 MATLAB Online 中打开

faceDatasetGroundTruth.mat

As far as I understand, you want to convert one column/variable of the table from cell array of character vectors to numeric. This is easy with convertvars:

S = load('faceDatasetGroundTruth.mat')
S = struct with fields:
    faceDataset: [202599×2 table]
T = S.faceDataset
T = 202599×2 table
imageFilenameface______________________________________________

    {'faceImages/000001.jpg'}    {'[95,71,226,313]'  }
    {'faceImages/000002.jpg'}    {'[72,94,221,306]'  }
    {'faceImages/000003.jpg'}    {'[216,59,91,126]'  }
    {'faceImages/000004.jpg'}    {'[622,257,564,781]'}
    {'faceImages/000005.jpg'}    {'[236,109,120,166]'}
    {'faceImages/000006.jpg'}    {'[146,67,182,252]' }
    {'faceImages/000007.jpg'}    {'[64,93,211,292]'  }
    {'faceImages/000008.jpg'}    {'[212,89,218,302]' }
    {'faceImages/000009.jpg'}    {'[600,274,343,475]'}
    {'faceImages/000010.jpg'}    {'[113,110,211,292]'}
    {'faceImages/000011.jpg'}    {'[166,68,125,173]' }
    {'faceImages/000012.jpg'}    {'[102,31,104,144]' }
    {'faceImages/000013.jpg'}    {'[89,132,247,342]' }
    {'faceImages/000014.jpg'}    {'[110,122,234,324]'}
    {'faceImages/000015.jpg'}    {'[93,86,190,263]'  }
    {'faceImages/000016.jpg'}    {'[39,89,283,392]'  }
F = @(c)sscanf([c{:}],'[%f,%f,%f,%f]',[4,Inf]).';
T = convertvars(T,'face',F);
T = convertvars(T,'imageFilename',@string) % optional, a bit slow.
T = 202599×2 table
imageFilenameface_______________________________________________

    "faceImages/000001.jpg"     95     71    226    313
    "faceImages/000002.jpg"     72     94    221    306
    "faceImages/000003.jpg"    216     59     91    126
    "faceImages/000004.jpg"    622    257    564    781
    "faceImages/000005.jpg"    236    109    120    166
    "faceImages/000006.jpg"    146     67    182    252
    "faceImages/000007.jpg"     64     93    211    292
    "faceImages/000008.jpg"    212     89    218    302
    "faceImages/000009.jpg"    600    274    343    475
    "faceImages/000010.jpg"    113    110    211    292
    "faceImages/000011.jpg"    166     68    125    173
    "faceImages/000012.jpg"    102     31    104    144
    "faceImages/000013.jpg"     89    132    247    342
    "faceImages/000014.jpg"    110    122    234    324
    "faceImages/000015.jpg"     93     86    190    263
    "faceImages/000016.jpg"     39     89    283    392

7 个评论
显示 5更早的评论隐藏 5更早的评论

Stephen23 2021-10-8

编辑：Stephen23 2021-10-8

在 MATLAB Online 中打开

@Yonathan Zarkovian: you are confusing how data are stored with how data are displayed.

Character vectors within a cell array within a table are displayed with single quote marks, but those do NOT indicate that there are actually single quote characters at the ends of the character vectors. This is trivial to demonstrate:

C = {'cat';'hat'};
T = table(C)
T = 2×1 table
       C   
    _______

    {'cat'}
    {'hat'}
T.C{1}(1) % the first character is 'c', not a single quote.
ans = 'c'

So your question is basically asking how to remove something that is not there... but such a task is impossible: you cannot remove what is not there. If you want to display tables differently then you could write your own routine to do so (but this would not be a trivial task and not many people would waste their time on that).

Because you asked how to remove non-existent characters, I interpreted your question as asking how to convert those character vectors (in a cell array) into actual numeric data.

Perhaps that is what you really need, who knows.

"Here's what the column should look like"

The important question is: why should it look like that? Your character vectors already DO NOT have single quotes (that you are trying to remove, even though they do not exist), so the actual stored characters are already exactly as you show.

Do you actually need numeric data or character/text data?

https://xyproblem.info

https://en.wikipedia.org/wiki/XY_problem

https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem

Yonathan Zarkovian 2021-10-10

编辑：Yonathan Zarkovian 2021-10-10

在 MATLAB Online 中打开

Hi,

Thank you for the clarification. The example I showed in the screenshot above is of an existing dataset that works with the algorithm, so I'm assuming that's what my dataset should like like as well, for it to work.

The example is taken from this guide:

Object Detection Using YOLO v3 Deep Learning - MATLAB & Simulink (mathworks.com)

The first column should be text and the second numeric.

Here's the full code and the error message with the faceDatasetGroundTruth.mat file:

%% download pretrained network (set to 'true' to train the network)
doTraining = false;
if ~doTraining
    preTrainedDetector = downloadPretrainedYOLOv3Detector();    
end
%% load data (modify to faces dataset)
data = load('faceDatasetGroundTruth.mat')
faceDataset = data.faceDataset;
% Add the full path to the local face data folder.
faceDataset.imageFilename = fullfile(pwd, faceDataset.imageFilename);
% Split the data set into a training set for training the network, and a test set for evaluating the network. Use 60% of the data for training set and the rest for the test set
rng(0);
shuffledIndices = randperm(height(faceDataset));
idx = floor(0.6 * length(shuffledIndices));
trainingDataTbl = faceDataset(shuffledIndices(1:idx), :);
testDataTbl = faceDataset(shuffledIndices(idx+1:end), :);
% Create an image datastore for loading the images
imdsTrain = imageDatastore(trainingDataTbl.imageFilename);
imdsTest = imageDatastore(testDataTbl.imageFilename);
% Create a datastore for the ground truth bounding boxes
bldsTrain = boxLabelDatastore(trainingDataTbl(:, 2:end));
bldsTest = boxLabelDatastore(testDataTbl(:, 2:end));
% Combine the image and box label datastores
trainingData = combine(imdsTrain, bldsTrain);
testData = combine(imdsTest, bldsTest);
% detect invalid images, bounding boxes or labels - discard invalid images
% manually
% validateInputData(trainingData);
% validateInputData(testData);
%% Data Augmentation (using random transformation)
augmentedTrainingData = transform(trainingData, @augmentData);
%% Define YOLO v3 Object Detector (use 227 227 3 for lower computational time. standard size for v3 is 416 416 3)
networkInputSize = [450 450 3];
rng(0)
trainingDataForEstimation = transform(trainingData, @(data)preprocessData(data, networkInputSize));
numAnchors = 6;
[anchors, meanIoU] = estimateAnchorBoxes(trainingDataForEstimation, numAnchors)
area = anchors(:, 1).*anchors(:, 2);
[~, idx] = sort(area, 'descend');
anchors = anchors(idx, :);
anchorBoxes = {anchors(1:3,:)
    anchors(4:6,:)
    };
baseNetwork = squeezenet;
classNames = trainingDataTbl.Properties.VariableNames(2:end);
yolov3Detector = yolov3ObjectDetector(baseNetwork, classNames, anchorBoxes, 'DetectionNetworkSource', {'fire9-concat', 'fire5-concat'});
%% Preprocess Training Data
preprocessedTrainingData = transform(augmentedTrainingData, @(data)preprocess(yolov3Detector, data));
data = read(preprocessedTrainingData);
reset(preprocessedTrainingData);
%% Training options
numEpochs = 80;
miniBatchSize = 8;
learningRate = 0.001;
warmupPeriod = 1000;
l2Regularization = 0.0005;
penaltyThreshold = 0.5;
velocity = [];
%% Train Model
if canUseParallelPool
   dispatchInBackground = true;
else
   dispatchInBackground = false;
end
mbqTrain = minibatchqueue(preprocessedTrainingData, 2,...
        "MiniBatchSize", miniBatchSize,...
        "MiniBatchFcn", @(images, boxes, labels) createBatchData(images, boxes, labels, classNames), ...
        "MiniBatchFormat", ["SSCB", ""],...
        "DispatchInBackground", dispatchInBackground,...
        "OutputCast", ["", "double"]);
if doTraining
    
    % Create subplots for the learning rate and mini-batch loss.
    fig = figure;
    [lossPlotter, learningRatePlotter] = configureTrainingProgressPlotter(fig);
    iteration = 0;
    % Custom training loop.
    for epoch = 1:numEpochs
          
        reset(mbqTrain);
        shuffle(mbqTrain);
        
        while(hasdata(mbqTrain))
            iteration = iteration + 1;
           
            [XTrain, YTrain] = next(mbqTrain);
            
            % Evaluate the model gradients and loss using dlfeval and the
            % modelGradients function.
            [gradients, state, lossInfo] = dlfeval(@modelGradients, yolov3Detector, XTrain, YTrain, penaltyThreshold);
    
            % Apply L2 regularization.
            gradients = dlupdate(@(g,w) g + l2Regularization*w, gradients, yolov3Detector.Learnables);
    
            % Determine the current learning rate value.
            currentLR = piecewiseLearningRateWithWarmup(iteration, epoch, learningRate, warmupPeriod, numEpochs);
    
            % Update the detector learnable parameters using the SGDM optimizer.
            [yolov3Detector.Learnables, velocity] = sgdmupdate(yolov3Detector.Learnables, gradients, velocity, currentLR);
    
            % Update the state parameters of dlnetwork.
            yolov3Detector.State = state;
              
            % Display progress.
            displayLossInfo(epoch, iteration, currentLR, lossInfo);  
                
            % Update training plot with new points.
            updatePlots(lossPlotter, learningRatePlotter, iteration, currentLR, lossInfo.totalLoss);
        end        
    end
else
    yolov3Detector = preTrainedDetector;
end
%% Evaluation
% results = detect(yolov3Detector,testData,'MiniBatchSize',8);
% 
% % Evaluate the object detector using Average Precision metric.
% [ap,recall,precision] = evaluateDetectionPrecision(results,testData);
% 
% % Plot precision-recall curve.
% figure
% plot(recall,precision)
% xlabel('Recall')
% ylabel('Precision')
% grid on
% title(sprintf('Average Precision = %.2f', ap))
%% Detect Objects
% Read the datastore.
% data = read(testData);
% 
% % Get the image.
% I = data{1};
% 
% [bboxes,scores,labels] = detect(yolov3Detector,I);
% 
% % Display the detections on image.
% I = insertObjectAnnotation(I,'rectangle',bboxes,scores);
% 
% figure
% imshow(I)
%% test with single image file
img = imread('test.jpg');
[bboxes,scores,labels] = detect(yolov3Detector,img);
img = insertObjectAnnotation(img,'rectangle',bboxes,scores);
figure
imshow(img)
%% webcam setup
% cam = webcam('EpocCam Camera');
% cam.resolution = '640x480';
% img = snapshot(cam);
% [bboxes,scores,labels] = detect(yolov3Detector,img);
% img = insertObjectAnnotation(img,'rectangle',bboxes,scores);
% figure
% imshow(img)
%% object detection in video
% v = VideoReader('cars.mp4');
% img = read(v,1);
% [bboxes,scores,labels] = detect(yolov3Detector,img);
% img = insertObjectAnnotation(img,'rectangle',bboxes,scores);
% figure
% imshow(img)
%%
% for c = 1:25
%     img = read(v,c);
%     [bboxes,scores,labels] = detect(yolov3Detector,img);
%     img = insertObjectAnnotation(img,'rectangle',bboxes,scores);
%     figure
%     imshow(img)
% end
    
>> yolov3_Copy
data = 
  struct with fields:
    faceDataset: [202599×2 table]
Error using boxLabelDatastore (line 234)
Invalid data in column 1 of input table 1.
Error in yolov3_Copy (line 27)
bldsTrain = boxLabelDatastore(trainingDataTbl(:, 2:end));
Caused by:
    Error using boxLabelDatastore>iAssertValidBBoxFormat (line 870)
    The size of bounding box data must be M-by-4, M-by-5, or M-by-9, where M is the number of boxes in each table element. The column
    in the training data table that contains the bounding boxes must be a cell array.
 
>> 

Stephen23 2021-10-12

在 MATLAB Online 中打开

faceDatasetGroundTruth.mat

"Unfortunately it results in this error"

Yes, because you are not supplying that tool with the data in the format that it requires. You are using the tool, which means that you have its documentation, which means that you can read its documentation and find out what are the specific needs for the data.

The error message also gives some hints, perhaps something like this might be what that tool needs:

S = load('faceDatasetGroundTruth.mat')
S = struct with fields:
    faceDataset: [202599×2 table]
T = S.faceDataset
T = 202599×2 table
imageFilenameface______________________________________________

    {'faceImages/000001.jpg'}    {'[95,71,226,313]'  }
    {'faceImages/000002.jpg'}    {'[72,94,221,306]'  }
    {'faceImages/000003.jpg'}    {'[216,59,91,126]'  }
    {'faceImages/000004.jpg'}    {'[622,257,564,781]'}
    {'faceImages/000005.jpg'}    {'[236,109,120,166]'}
    {'faceImages/000006.jpg'}    {'[146,67,182,252]' }
    {'faceImages/000007.jpg'}    {'[64,93,211,292]'  }
    {'faceImages/000008.jpg'}    {'[212,89,218,302]' }
    {'faceImages/000009.jpg'}    {'[600,274,343,475]'}
    {'faceImages/000010.jpg'}    {'[113,110,211,292]'}
    {'faceImages/000011.jpg'}    {'[166,68,125,173]' }
    {'faceImages/000012.jpg'}    {'[102,31,104,144]' }
    {'faceImages/000013.jpg'}    {'[89,132,247,342]' }
    {'faceImages/000014.jpg'}    {'[110,122,234,324]'}
    {'faceImages/000015.jpg'}    {'[93,86,190,263]'  }
    {'faceImages/000016.jpg'}    {'[39,89,283,392]'  }
F = @(c)num2cell(sscanf([c{:}],'[%f,%f,%f,%f]',[4,Inf]).',2);
T = convertvars(T,'face',F)
T = 202599×2 table
imageFilenameface____________________________________________

    {'faceImages/000001.jpg'}    {[  95 71 226 313]}
    {'faceImages/000002.jpg'}    {[  72 94 221 306]}
    {'faceImages/000003.jpg'}    {[  216 59 91 126]}
    {'faceImages/000004.jpg'}    {[622 257 564 781]}
    {'faceImages/000005.jpg'}    {[236 109 120 166]}
    {'faceImages/000006.jpg'}    {[ 146 67 182 252]}
    {'faceImages/000007.jpg'}    {[  64 93 211 292]}
    {'faceImages/000008.jpg'}    {[ 212 89 218 302]}
    {'faceImages/000009.jpg'}    {[600 274 343 475]}
    {'faceImages/000010.jpg'}    {[113 110 211 292]}
    {'faceImages/000011.jpg'}    {[ 166 68 125 173]}
    {'faceImages/000012.jpg'}    {[ 102 31 104 144]}
    {'faceImages/000013.jpg'}    {[ 89 132 247 342]}
    {'faceImages/000014.jpg'}    {[110 122 234 324]}
    {'faceImages/000015.jpg'}    {[  93 86 190 263]}
    {'faceImages/000016.jpg'}    {[  39 89 283 392]}

Yonathan Zarkovian 2021-10-13

Thank you, this method seems to work.

请先登录，再进行评论。

Removing a character from a table (that's within a struct)

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

7 个评论
显示 5更早的评论隐藏 5更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Removing a character from a table (that's within a struct)

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

7 个评论 显示 5更早的评论隐藏 5更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

7 个评论
显示 5更早的评论隐藏 5更早的评论