That's not huge - it's only about a tenth the size of a typical digital image. Anyway, you will get nan in the data where the Excel cell is empty. So use conv2() and isnan() to replace the missing ones. Assuming there is only one missing cell, not a string of them, do something like this:
%data = rand(18, 24); % Create sample/test data.
%data(5, 2) = nan;
data = xlsread(filename);
% Find missing cells.
nanLocations = isnan(data);
% Assign zeros to nan locations
data(nanLocations) = 0;
% Count non-nans
kernel = [1;1;1];
nonNanCounts = conv2(double(~nanLocations), kernel, 'same');
% Sum up values in 3x1 window.
sums = conv2(data, kernel, 'same');
% Divide the sum by the count to get the mean.
meanMatrix = sums ./ nonNanCounts;
% Replace nan's in original data with mean values.
data(nanLocations) = meanMatrix(nanLocations);
