numRows = 10;
numCols = 12;
gameBoard = randi(2, numRows, numCols) - 1;
You could then use conv2 to determine the sum of the neighbouring live cells (i.e the number of living neighbours) for each cell in your game board like
liveCellCounts = conv2(gameBoard, [1 1 1; 1 0 1; 1 1 1], 'same');
This differs from the function signature that you have provided, but you may be able to adapt either this code for the function, or change the function altogether. The code to update the game board given the number of live neighbouring cells could be simplified to 2-3 lines depending upon your rules to update the population.