RBM - linear to binary layer

Question

0 个投票

I am working in Matlab and developing a project which is focused on deep learning. Since I have to apply this technique to images, I thought that the usual binary-binary units are not appropriate since the images are gray scale and not binarized. So I tried to follow the code prof. Hinton provided on his website to write my implementation of a linear-to-binary rbm. In my idea this should be the first layer of the autoencoder: the following ones can be binary-to-binary.

So these are some lines of the code I wrote (the variables' names are quite self-explicative):

    poshidprobs = sigmoid(data*vishid + repmat(hidbiases, numcases, 1));
    batchposhidprobs(:,:,batch)=poshidprobs;
    posprods    = data' * poshidprobs;    % a sort of correlation
    poshidact   = sum(poshidprobs);
    posvisact = sum(data);
    %%%%%%%%%END OF POSITIVE PHASE  %%%%%%%%%%%%%%%%
    poshidstates = poshidprobs > rand(numcases,numhid);
    %%%%%%%%%START NEGATIVE PHASE  %%%%%%%%%%%%%%%%%
    % linear reconstruction
    negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);
    % hidden units update
    neghidprobs = sigmoid(negdata*vishid + repmat(hidbiases,numcases,1));
    negprods  = negdata'*neghidprobs;
    neghidact = sum(neghidprobs);
    negvisact = sum(negdata);
    %%%%%%%%%END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%
    err= sum(sum( (data-negdata).^2 ));
    errsum = err + errsum;
    %%%%%%%%%UPDATE WEIGHTS AND BIASES %%%%%%%
        vishidinc = momentum*vishidinc + ...
            epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);
        visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);
        hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);
        vishid = vishid + vishidinc;
        visbiases = visbiases + visbiasinc;
        hidbiases = hidbiases + hidbiasinc;

Does it make sense to you? Can you spot any mistake? The reason why I am asking is because the reconstruction error stays very high, like 500000 or so. So I was asking myself (and the community) whether this is normal or it indicates some problems and mistakes.