How to partition data in cells for validation in machine learning model?

Question

Isabelle Museck 2024-8-1，13:38

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2142106-how-to-partition-data-in-cells-for-validation-in-machine-learning-model

评论： Isabelle Museck 2024-8-13，21:14

Hello there , I have training data for 4 trials stores in a 4x1 cell named "trainingdataX" and "trainingdataY" as whoen here and I am trying to pull out 15 percent of all this data for validation purposes and store it in variables "Xval" and "Yval". How would I be able to do this if the data is stored in a cells corresponding to the trials and ensure the corresponding value is partioned out for validation too? Any help is greatly appreciated!

%Exclude Data for Val
   rng('default')
   n = %im not sure what to put here to have it pull data from each of the 4 trials
   partition = cvpartition(n,'Holdout',0.15);
   idxTrain = training(partition);
    FinalTrainX = trainingdataX(idxTrain,:)
    FinalTrainY = trainingdataY(idxTrain,:)
    
    idxNew = test(partition);
    Xval = trainingdataX(idxNew,:)
    Yval = trainingdataY(idxNew,:)

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

YERRAMADAS 2024-8-1，13:55

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2142106-how-to-partition-data-in-cells-for-validation-in-machine-learning-model#answer_1493401

Use the cross-validation method to maximize the data available for each of these sets

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Isabelle Museck 2024-8-13，21:14

Could you possibly epand more on how to use the cross-validation method in MATLAB?

请先登录，再进行评论。

Answer 2

Aditya 2024-8-1，14:20

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2142106-how-to-partition-data-in-cells-for-validation-in-machine-learning-model#answer_1493421

在 MATLAB Online 中打开

Hi @Isabelle Museck,

To partition data stored in cells for validation, you need to first concatenate the data from all trials into single matrices. After partitioning, you can then split the data back into the training and validation sets.

before moving forward you need to transpose your X and Y data, so that each row of X can correspond to the row of Y.

Here's a sample code for this:

% sample data 
trainingdataX = cell(4, 1);
trainingdataY = cell(4, 1);
for i = 1:4
    trainingdataX{i} = rand(541, 63); 
    trainingdataY{i} = rand(541, 1);  
end
% Concatenate data
allX = vertcat(trainingdataX{:});
allY = vertcat(trainingdataY{:});
% Partition data (15% holdout for validation)
rng('default'); % For reproducibility
partition = cvpartition(size(allX, 1), 'Holdout', 0.15);
idxTrain = training(partition);
idxVal = test(partition);
% Split into training and validation sets
FinalTrainX = allX(idxTrain, :);
FinalTrainY = allY(idxTrain, :);
Xval = allX(idxVal, :);
Yval = allY(idxVal, :);
% Display results
fprintf('Training data X size: %dx%d\n', size(FinalTrainX, 1), size(FinalTrainX, 2));
fprintf('Training data Y size: %dx%d\n', size(FinalTrainY, 1), size(FinalTrainY, 2));
fprintf('Validation data X size: %dx%d\n', size(Xval, 1), size(Xval, 2));
fprintf('Validation data Y size: %dx%d\n', size(Yval, 1), size(Yval, 2));

I hope this helps!

2 个评论
显示无隐藏无

Isabelle Museck 2024-8-1，19:10

Hello Aditya,

Thank you so much for your help. This makes alot of sense however when I try intergrate the code qwith my data is not vertically concatoning the data in the cells properly. I am ending up with "allX" being a 252x541 double and "allY" being a 4x541 double as shown here:

When I run the code you provided, I should be getting a 2163x63 double for "allX" and a 2164x1 double for "allY". Do you know why it may not be concatonating correclty for me and my data?

Aditya 2024-8-1，19:56

编辑：Aditya 2024-8-1，19:58

在 MATLAB Online 中打开

Hi @Isabelle Museck,

As mentioned in my post that your initial data is in shape: 63X541 & 1X541, which is incorrect for vertical concat, for this you need to take the transpose of it and use it:

Inorder to transpose it you can use the below line of code:

% Transpose each cell using cellfun
trainingdataX = cellfun(@transpose, trainingdataX, 'UniformOutput', false);
trainingdataY = cellfun(@transpose, trainingdataY, 'UniformOutput', false);

or you can do it manually using the for loop!

Hope this clarifies your doubt!

请先登录，再进行评论。