Cleaning data for machine learning

Question

FERNANDO CALVO RODRIGUEZ 2023-3-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1928435-cleaning-data-for-machine-learning

评论： FERNANDO CALVO RODRIGUEZ 2023-3-31

Hey!

I am trying to clean up the missing data described as NaN for a regression using the neural network fitnet function. The thing is that these missing values for each observation I have, I don't know them and I can't remove them because I would lose the meaning. I know that in python it can be done with a pandas drop function, but in matlab I don't know how to do it without getting an error in the neural network.

If someone knows something, it would be appreciated.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Luca Ferro 2023-3-14

编辑：Luca Ferro 2023-3-14

It's not quite clear to me if you want to remove the NaNs or replace them with 0s.

In any case, it would be very useful if you could share the data

FERNANDO CALVO RODRIGUEZ 2023-3-15

I don't want to remove them, I just want the network to ignore them. Because if it has many NaN values the network does not work when the data it has is more than enough to find the answers.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Vijeta 2023-3-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1928435-cleaning-data-for-machine-learning#answer_1202679

Hi Fernando,

One way to handle missing data (NaN values) in a regression problem using the fitnet function in MATLAB is to impute the missing values with some reasonable estimate before feeding the data into the neural network. There are several methods for imputing missing values, such as mean imputation, median imputation, and regression imputation.

A graphical user-friendly MATLAB interface is presented here: the Missing Data Imputation (MDI) Toolbox.
MDI Toolbox allows imputing incomplete datasets, following missing completely at random pattern.
Different state-of-the-art methods are included in the toolbox, such as trimmed scores regression and data augmentation

You can refer to the following documentation: https://www.mathworks.com/help/deeplearning/ref/fitnet.html?s_tid=srchtitle_fitnet_1

Thanks.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

FERNANDO CALVO RODRIGUEZ 2023-3-31

I don't exactly want to do that since the response variables follow a certain function that I don't know, since it is the behavior of a new material under compression. But yes, what you tell me would work for somewhat simpler networks.

请先登录，再进行评论。