One problem you're probably having is that you are overwriting the Overtimesalary variable each loop. I would restructure the code a bit since you've got that same equation in two places, using an anonymous function for calculating pay. I would also eliminate the selection between functions, instead implanting a single function for pay that works whether overtime is worked or not.
Here's what I would do:
S= 12;
PayFunc = @(t,S) S*t + min(((S*0.5)*(t-40)),0);
payTable = zeros(50-19,2);
for t = 20:50
rowPtr = t-19;
payTable(rowPtr ,1) = t;
payTable(rowPtr ,2) = PayFunc(t,S)
end
