Optimization code. One line takses 86% of working time.

Question

Jan Kowal 2016-5-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/284157-optimization-code-one-line-takses-86-of-working-time

评论： Ahmet Cecen 2016-5-14

Hello Everyone! I am developing a Clasificator for hand written signs and digits using Logical Regression and Softmax function.

I have a problem with one line in one function because it takes too much time.

function [grad] = logistic_cost_function(xTrain, yTrain, w)
%xTrain = [30134 x 11760]
%xTrain = [30134 x 1]
%w = [36 x 11760]
    N = size(xTrain,1);
    K = 36;
    grad = zeros(size(w));
    ARG = w * xTrain';
    for n = 1 : N
        xRow = xTrain(n,:);
        Probability = softmax(ARG(:,n));
        label_n = yTrain(n);
        for k = 1 : K
            IND = label_n == k;
            prob_k = Probability(k);
            grad(k,:) = grad(k,:) + (IND - prob_k) * xRow;
        end
    end
    grad = -grad/N;
end

It is the time counted on only 2 epochs. My point is to count a few thousands of epchos.

Does anyone have any idea how to improve an algorithm?

Bet Regards

3 个评论
显示 1更早的评论隐藏 1更早的评论

Walter Roberson 2016-5-14

IND is a logical value, a 0 or 1. Are you sure you want to be subtracting prob_k from that? That would be likely to give you a negative value from the subtraction for the cases where the two are not equal.

Ahmet Cecen 2016-5-14

I believe that might be the point. Don't think of it as a negative probability, think of IND as an indicator function. You either subtract expectation by the probability of hitting the case, or add expectation by the probability of not hitting the case. I may be wrong though as I have no idea what he is actually trying to do.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ahmet Cecen 2016-5-14

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/284157-optimization-code-one-line-takses-86-of-working-time#answer_222072

编辑：Ahmet Cecen 2016-5-14

在 MATLAB Online 中打开

        for k = 1 : K
            IND = label_n == k;
            prob_k = Probability(k);
            grad(k,:) = grad(k,:) + (IND - prob_k) * xRow;
        end

Here is a way out of this loop (well hopefully, as I can't actually run the code to verify):

 IND = zeros(K,1); IND(label_n)=1;
 prob_k = Probability;
 % (IND - prob_k) IS A COLUMN VECTOR, SO OUTER PRODUCT.
 grad = grad + (IND - prob_k)*xRow;

Hmm... turns out no need for bsxfun.

4 个评论
显示 2更早的评论隐藏 2更早的评论

John D'Errico 2016-5-14

Ahmet - bsxfun is not needed here because * between a column and row vector is well defined as a matrix product already. That outer product has always worked.

Ahmet Cecen 2016-5-14

Yeah, in a previous version of my answer I didn't see that operation was an outer product for some reason and used a weird combination of bsxfun and repmat. Sometimes when I focus on trying to understand other people's code the simple stuff manages to elude me.

请先登录，再进行评论。