I have used the code of Prof. Geoffrey Hinton in training Deep Neural Network using MNIST database (binary Image)

Question

jeffin 2016-1-20

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/264485-i-have-used-the-code-of-prof-geoffrey-hinton-in-training-deep-neural-network-using-mnist-database

编辑： Walter Roberson 2016-1-21

Whenever i googled to find code for Deep Neural Network, i can find it only for MNIST binary image set. I understand MNIST is the standard database for Deep Neural Network. Is it possible to train the DNN using colour images?

I tried Prof. Geoffrey Hinton's Deep Neural Network code for RGB colour images. I got an output, but not clear. I analysed and found that i need to do some modification in the function "restricted Boltzmann machine" (rbm.m) to get the actual output. Could somebody help me here. Or it would be fine if could point out any source code for Deep Neural Network trained using colour images? . Thanks in advance.

code of function rbm.m ; source : http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html

% Version 1.000 
%
% Code provided by Geoff Hinton and Ruslan Salakhutdinov 
%
% Permission is granted for anyone to copy, use, modify, or distribute this
% program and accompanying programs and documents for any purpose, provided
% this copyright notice is retained and prominently displayed, along with
% a note saying that the original programs are available from our
% web page.
% The programs and documents are distributed without any warranty, express or
% implied.  As the programs were written for research purposes only, they have
% not been tested to the degree that would be advisable in any important
% application.  All use of these programs is entirely at the user's own risk.
% This program trains Restricted Boltzmann Machine in which
% visible, binary, stochastic pixels are connected to
% hidden, binary, stochastic feature detectors using symmetrically
% weighted connections. Learning is done with 1-step Contrastive Divergence.   
% The program assumes that the following variables are set externally:
% maxepoch  -- maximum number of epochs
% numhid    -- number of hidden units 
% batchdata -- the data that is divided into batches (numcases numdims numbatches)
% restart   -- set to 1 if learning starts from beginning 
epsilonw      = 0.1;   % Learning rate for weights 
epsilonvb     = 0.1;   % Learning rate for biases of visible units 
epsilonhb     = 0.1;   % Learning rate for biases of hidden units 
weightcost  = 0.0002;   
initialmomentum  = 0.5;
finalmomentum    = 0.9;
% restart =1
% batchdata = testbatchdata;
% batchdata = batchdata11;
[numcases numdims numbatches]=size(batchdata);
if restart ==1,
  restart=0;
  epoch=1;
% Initializing symmetric weights and biases. 
  vishid     = 0.1*randn(numdims, numhid);
%   size(vishid)
  hidbiases = 255*zeros(1,numhid);
  visbiases  = 255*zeros(1,numdims);
    poshidprobs = zeros(numcases,numhid);
    neghidprobs = zeros(numcases,numhid);
    posprods    = zeros(numdims,numhid);
    negprods    = zeros(numdims,numhid);
    vishidinc  = zeros(numdims,numhid);
    hidbiasinc = zeros(1,numhid);
    visbiasinc = zeros(1,numdims);
    batchposhidprobs=zeros(numcases,numhid,numbatches);
  end
for epoch = epoch:maxepoch,
 fprintf(1,'epoch %d\r',epoch); 
 errsum=0;
 for batch = 1:numbatches,
 fprintf(1,'epoch %d batch %d\r',epoch,batch); data = batchdata(:,:,batch);
%%%%%%%%%START POSITIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  poshidprobs = 1./(1 + exp(-data*vishid - repmat(hidbiases,numcases,1)));    
  batchposhidprobs(:,:,batch)=poshidprobs;
  posprods    = data' * poshidprobs;
  poshidact   = sum(poshidprobs);
  posvisact = sum(data);
%%%%%%%%%END OF POSITIVE PHASE  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  poshidstates = poshidprobs > rand(numcases,numhid);
%%%%%%%%%START NEGATIVE PHASE  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  negdata = 1./(1 + exp(-poshidstates*vishid' - repmat(visbiases,numcases,1)));
  neghidprobs = 1./(1 + exp(-negdata*vishid - repmat(hidbiases,numcases,1)));    
  negprods  = negdata'*neghidprobs;
  neghidact = sum(neghidprobs);
  negvisact = sum(negdata); 
%%%%%%%%%END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  err= sum(sum( (data-negdata).^2 ));
  errsum = err + errsum;
     if epoch>5,
       momentum=finalmomentum;
     else
       momentum=initialmomentum;
     end;
%%%%%%%%%UPDATE WEIGHTS AND BIASES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    vishidinc = momentum*vishidinc + ...
                epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);
    visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);
    hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);
      vishid = vishid + vishidinc;
      visbiases = visbiases + visbiasinc;
      hidbiases = hidbiases + hidbiasinc;
      % size(vishid)
  %%%%%%%%%%%%%%%%END OF UPDATES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    end
    fprintf(1, 'epoch %4i error %6.1f  \n', epoch, errsum); 
  end;