Hi Ioana,
I understand that the challenge you're encountering with your 1D CNN is primarily due to the imbalanced dataset, which significantly hinders achieving higher training accuracy. A custom weighted classification layer can be useful for addressing the imbalance by applying different weights to the classes during the loss calculation.
To fix this problem, ensuring that the custom layer correctly handles the batch size (N) and applies the class weights appropriately during both the forward and backward passes is important. The modifications should include correcting the loss and gradient calculations which will be responsible for proper application of class weights. In the original “forwardLoss” function, the operation W*(T.*log(Y)) may not apply the class weights properly to each corresponding class prediction since there is a high probability of dimension mismatch issues.
Possible code changes include:
- In the “forwardLoss” function, ensure numerical stability and correct weight application by using loss = -sum(sum(W .* (T .* log(Y + eps)), 1), 2) / N. This adjustment will help in handling the imbalance by applying the weights more accurately.
- For the “backwardLoss” function, implement the gradient calculation with dLdY = -(W' .* (T ./ (Y + eps))) / N , ensuring gradients are correctly computed and scaled for each class, considering the class weights and numerical stability.
After implementing these modifications, try to test the custom layer with a small dataset to ensure it behaves as expected, both in terms of loss calculation and gradient propagation. You can also refer to below MATLAB answer to better understand how the custom weighted classification layer construction works.
Hope it helps!
Thanks
