Here's a simple example using a linear regression, hopefully you find it helpful:
% random data
X = rand(100,1);
Y = 0.5*X + rand(100,1);
% linear regression
b = [ones(numel(X),1), X]\Y;
Yfit = b(2)*X + b(1); % Y(i) and Yfit(i) are located at same x value
% compare corresponding points in y direction
idx = Y > Yfit; % idx(i) tells whether Y(i) is above Yfit(i)
% plot
scatter(X(idx),Y(idx),'r');
hold on;
scatter(X(~idx),Y(~idx),'b');
plot(X,Yfit,'k');
