Using the built-in MATLAB profiler, the biggest computational load is in the evaluations of integral(). Which is no surprise since it is notoriously slow inside for loops, let alone inside a double for-loop inside a recursive function, when there are much more efficient ways of integration.
Below, I've rewritten your code to exclude for loops and calculate everything as efficiently as I could come up with. Also, I shifted n down by one compared to your code to be more consistent with the equations you provided.
% INITIALIZE MATLAB
clear variables
close all
clc
% DEFINE INPUT ARGUMENTS
n = 1;
t = -2 : 0.01 : 5;
% CALL RECURSIVE STATE FUNCTION
x = state(n,t);
% PLOT X
figure(1);
plot(t,x,'-b','LineWidth',2);
% STATE FUNCTION
function x = state(n,t)
% CHECK INPUT ARGUMENTS
if n < 0 || ~isscalar(n) || fix(n)~=n
error('"n" must be a non-negative integer.');
end
% INITIALIZE X
N = length(t);
x(1,:) = ones(1,N);
% BASE CASE
if n == 0
return
end
% CALCULATE VALID INDICES OF t FOR x
idx1 = t >= 0 & t <= 3;
% ALL OTHER CASES
x(2:n+1,:) = zeros(n,N);
for i = 2 : n+1
% CALCULATE u
u = control(i-1,t-1); % <-- Edit: upper bound t-1 not t
% CALCULATE STATE FUNCTION
x(i,idx1) = 2*cumtrapz(t(idx1),u(idx1));
end
% RETURN FINAL ROW
x = x(n+1,:);
function u = control(n,t)
% ASSUME u IS ZERO
u = zeros(size(t));
% FIND VALID INDICES OF t FOR u
idx2 = t >= 0 & t <= 2;
% CALCULATE CONTROL FUNCTION
u(idx2) = cumtrapz(t(idx2), x(n,idx2));
end
end
Edit: My apologies, I forgot the 
upper bound on the integral of 
. It is highlighted in the comments above.
upper bound on the integral of . It is highlighted in the comments above.
