There is no limit for xls files except the number of columns. So please explain the prioblem with the 340MB file with any details. Mentioning only, that there is any problem does not allow to suggest an improvement. Perhaps using the more powerful XLSX file instead helps already.
The file format .dat is not unique or standardized. Therefore we cannot guess how the conversion should happend. Creating an .m file would be strange, because these files are thought for Matlab code, not for storing data.
