The problem most is most likely with your library not with MATLAB. When a library is loaded the operating system increases the load count for the library by one and unloadlibrary decrements it by one. If the count goes to 0 the operating system has the option of unloading it. Doing so is not mandatory... Any other library loading the library you are using or the library itself may be locking itself into memory by calling loadlibrary on itself.
Many libraries do not work correctly if unloaded and reloaded. Calling unloadlibrary should be considered optional.
With windows com dlls the unload is deliberately delayed to prevent thrashing when the dll is unloaded and reloaded in rapid succession.