Yes, it is possible to use "too much memory" without getting an error message. When the memory requirements get very large, MATLAB will start to use virtual memory, and all the processing will slow down.
Try searching "matlab virtual memory" for hints here about memory issues. Here is one I found: