S = dir( pathName );
N = { S( [ S.isdir ] ).name };
N = N( ~cellfun( @(x) strcmp( x, '.' ) || strcmp( x, '..' ), N ) );
set(handles.listbox4,'String',N)
should work fine. Your method would also pick up files as well as directories. I assume from your title that you don't want that.