str2double --> double
Did you know that function double with string vector input significantly outperforms str2double with the same input:
x = rand(1,50000);
t = string(x);
tic; str2double(t); toc
tic; I1 = str2double(t); toc
tic; I2 = double(t); toc
isequal(I1,I2)
Recently I needed to parse numbers from text. I automatically tried to use str2double. However, profiling revealed that str2double was the main bottleneck in my code. Than I realized that there is a new note (since R2024a) in the documentation of str2double:
"Calling string and then double is recommended over str2double because it provides greater flexibility and allows vectorization. For additional information, see Alternative Functionality."
37 个评论
时间降序
Thanks so much for sharing. I always used sscanf, from a recommandation found here: https://stackoverflow.com/questions/8363132/why-is-str2double-so-slow-in-matlab-as-compared-to-a-mex-function
x = rand(1,50000);
t = string(x);
s1=str2double(t);
s2=double(t);
isequal(s1,s2)
s3 = reshape(sscanf(sprintf('%s#', t), '%g#'), size(t));
isequal(s1,s3)
timeit(@() str2double(t))
timeit(@() double(t))
timeit(@() reshape(sscanf(sprintf('%s#', t), '%g#'), size(t)))
ans =
0.1206
ans =
0.0015
ans =
0.0109
Cool! Thanks for sharing!
This is interesting @Jan Studnicka, thanks for posting. The double conversion method of string is a relatively new, highly optimised and multithreaded C++ built-in. str2double is a much older .m file and has some um......legacy behaviour! Hence all the notes about not using in the doc.
With that said, the performance gap between the two here is more than I'd like and so I've brought this to the attention of the relvant peple in development to see what can be done.
Nice tip! Thanks for sharing.
@Mike Croucher, I'm not seeing a Code Analyzer performance message for str2double. Maybe that could be another enhancement request?
This is being discussed internally as we speak. I believe that one of the issues is that the two approaches are not drop-in replacements for each other which makes it tricky to come up with robust automatic advice. All the right people are thinking about this now though thanks to @Jan Studnicka's observation.
@Mike Croucher as you're looking into possible improvements to str2double, take a look at some of the ideas I've implemented in my drop-in replacement str2number function on the File Exchange (FEX:106265). It provides significant speedup for most common use-cases.
Thanks Yair, I have passed that infomation on.
A bit of history -- the original function that people used for this purpose was str2num, which I think dates back to MATLAB 4 around 1990. The function str2num was implemented using eval so that it used exactly the same mechanism that the MATLAB interpreter used to parse MATLAB code. That's how it could handle all the variations in how floating-point, complex constants could be written, as well as the variations in how constant vectors and matrices could be written.
However, the use of eval turned out to be problematic, for both performance and behavior reasons. Calling eval on user-provided input could open a backdoor to potential mischief. Here is what the doc for str2num says about this today: "Security Considerations: str2num is implemented using eval which might lead to undesired side effects. When calling str2num with untrusted user input, use Evaluation='restricted' or str2double to avoid unexpected code execution."
As I recall, eliminating the hidden use of eval was the primary motivation for the introduction of str2double, which I think was around MATLAB 5, 1997. It was faster than str2num and didn't have the potential for bad eval behavior, and MathWorks encouraged people to use it instead.
As @Mike Croucher pointed out, though, str2double supports a lot of "legacy" behavior, as Mike called it. Essentially, str2double was implemented in a way that tries to do nearly the same job as str2num was doing, but without direct access to the parser conversion mechanism. Take a look at str2double.m to get an idea of all the different cases involved just for handling all the ways that complex numbers can be written. Reading it gives me a headache.
I doubt that it is worth the development and testing time (and the risk of introducing incompabilities) to try to optimize all of str2double, but it's possible that there are straightforward ways to optimize certain commonly-used code paths.
Thanks so much for the historical perspective Steve. When discussing a language that has been in continuous use for over 40 years, and now has over 5 million users, such perspectives are useful. They help us understand why things are the way they are.
Something that might not make sense in a modern context, made perfect sense 30 years ago and then evolved over time, fixing bugs and satisying user demands. At some point we may have to change lanes and switch to another approach but users (rightly!) expect that all their old code will continue working.
Everyone (including me!) wants the very best that modern tech can allow while never breaking anything old. These are sometimes mutually exclusive requirements. I am somehow reminded of the 'spacebar heating' XKCD comic right now xkcd: Workflow
I can think of three core language features, introduced in the late 1990s and early 2000s, whose priority and functional design were influenced by the desire to reduce the need for calling eval. Oooh, I can feel a new blog post coming.
Sorry for drifting completely off topic. 🙂
I am still waiting for your blog post.
I just replaced str2double(x) with double(string(x)) and got 20x faster.
@Duy Hai - thanks for the nudge about blog posts. It's good to hear that there are readers who might be interested.
That is a funny comic.
I think that MathWorks has struck a good balance in this respect. As I understand it, most of the time older functions, rather than being fully deprecated, are given the "not recommended" status in documentation and Editor suggestions, with a brief description of a newer function to use instead. This way, MathWorks can stop developing dead-end functions (that many users may still rely upon) while focusing on the newer ones.
Of course, the suggested newer function may not be a one-for-one replacement, and sometimes I find that I do have to stick with the older function. But, it is very helpful to know that (1) current functionality should work for the foreseeable future and (2) there is an improved option available.
I vote for MathWorks to convert all legacy functions to be highly optimized, multithreaded C++ built-ins!
That is certainly an understandable request! Sometimes, though, "legacy" functions are legacy because they turned out to be a design dead-end, possibly for the reason that they are difficult or impossible to optimize well. The best example of this that I can think is the concept of cell arrays of strings, commonly called cellstrs. The difficulty of handling cellstrs efficiently at the language execution level (because of the general nature of a cell as a container), as well as the unfortunate behavioral discontinuity between code written for multiple strings and code written for one string, were big motivations for the very challenging, multiyear project of introducing true string arrays into MATLAB.
That said, the various MATLAB development teams involved really, really do want to make any MATLAB code that people write go as fast as possible. Since that can't be accomplished all at once, though, the work must be identified and prioritized. This very thread will feed into that prioritization process when @Mike Croucher creates a str2double enhancement request for development to consider.
I wondered about "why" -- care to explain a little more? At least superficially, a string looks awfully similar to a cell-like container for a char() variable, just like cellstr.
How much effort does MATLAB need to do to determine if the C variable defined below is a cellstr?
C = repmat({'a'}, 1, 1e6);
iscellstr(C)
How about now?
C{end} = 1;
iscellstr(C)
The iscellstr function not only had to look at the class of C, in the worst case scenarios (if it is a cellstr, or if the last element is the one that disqualifies it) it had to check to make sure the contents of each and every cell was a char array.
What if we had a string array?
S = string(C);
isstring(S)
If the variable is of class string, we know it contains text data. Any other type of data that would be assigned into it is converted (if possible) into a string. That uniformity can simply argument validation.
@dpb - Why what? Can you be more specific about what you'd like to know? This thread has covered a lot of ground. 🙂
My first foray into string was to write a MID$ function as
mid=@(s,i1,i2)s(i1:i2);
s="Freddy Flintstone";
%mid(s,2,4)
Oh. Then must be
mid=@(s,i1,i2)s{i1:i2};
%mid(s,2,4)
Well, then
mid=@(s,i1,i2)s{:}(i1:i2);
mid(s,2,4)
just was/is so similar in syntax to the cellstr() that didn't see just what one really gained other than the overloaded plus and numeric conversion were useful on occasion. Of course, hadn't discovered anything about the speedup then.
My use cases wouldn't ever have had something like Steven's example, either; would be far more careful with what did with or to a cellstr() just because knew what were and intended use.
Agree the mismatch in functions like uigetfile are a pain to deal with alternate returns, that the escape route is then numeric as well is a pain indeed. On that score, however, I thought it and friends should have been fixed up internally instead of being released as were with that mismatch....although I suppose there may have been timing issues regarding introduction as to which was on the block first?
I've come to graduating to using them more although old habits are hard to break and my first inclination is always to start with an opening curly bracket, not a double quote.
I do think the class is missing some functions like mid() and so on, but I suppose again, having to add in the array index besides the positions sorta' defeats it...
But, I grok Steven's efficiency point above and that have to be able to handle all possibilities however badly designed/used by end users in not keeping tabs on their cellstr() variables.
The string class has mid() functionality. We just spell it extractBetween, to fit into the pattern of extractBefore and extractAfter.
s="Freddy Flintstone";
[extractBefore(s, 2);
extractBetween(s, 2, 4);
extractAfter(s, 4)]
This can handle non-scalar string arrays too.
sb = [s, "Barney Rubble"]
[extractBefore(sb, 2);
extractBetween(sb, 2, 4);
extractAfter(sb, 4)]
This works with char vectors and cellstr as well. Though what they return (char or cellstr) can get confusing in the char case.
c = char(s);
extractBefore(c, 2)
extractBetween(c, 2, 4)
extractAfter(c, 4)
cs = cellstr(s);
extractBefore(cs, 2)
extractBetween(cs, 2, 4)
extractAfter(cs, 4)
Yeah, I discovered that somewhat later or maybe while, I don't remember now...but were all the functions introduce initially with the class itself? Seems like I recall things didn't all arrive initially. Not that it is of import any more and it may be I just wasn't diligent enough in digging through the available functions at the time...other than being somewhat wordy, it does work nicely and the family including the pattern matching has been quite useful in recent work, indeed. I only wish split() wouldn't die on different count of delimiters so could wrap things up together... <grin>
I only wish split() wouldn't die on different count of delimiters so could wrap things up together.
I'm just interested in what you are trying to accomplish that split won't do for you.
The particular use case was needing a way to encode a date, amount, name and optional comment into an existing spreadsheet that didn't have flexibility to alter layout so the data was supposed to be entered with a semi-colon as a separator. Issue was, couldn't ever train the data entry ladies still needed a trailing semi-colon even there was no comment field. split barfs instead or returning empty field(s) in such a case - and I understand the tendency of the ladies to not add a trailing ";", it certainly doesn't appear it is needed on entry. Special case, yeah, but painful at the time. I've discovered a few other instances it has bit as well although I don't know that I can reproduce the exact use of top of head at the moment.
BTW, not knowing to whom to address it for Discussions, the interface seems quite unfriendly -- I kept seeing there was a response only an hour old, but sorting "Descending" only shows something from Nov 2025; had to page through the whole thing and find the "Load More" and then scroll to the bottom to find the reply....somehow it ought to be able to find all the replies and sort them all. It seems difficult to use interace for more than a very few replies in a thread.
What alternative behavior for split would you propose? One possibility would be to automatically add trailing delimiters to any line containing fewer delimiters than the maximum number found in the string array. I suspect that would slow down the execution speed of split because the function would not be able to determine the exact number of delimiters to use until all the lines have been scanned once. For this reason, I doubt that it would be an acceptable change to the default behavior. But perhaps it could be an optional behavior.
Yeah, that or similar was what ended up doing; just seemed like an unecessary fixup -- and, of course, don't think about it the next time it comes up until get bitten.
I'll forward your feedback about the Discussions interface to someone at MathWorks.
If you ever run into a similar issue again with using split, @dpb, try an initial step to fix up the string array before passing it to split. It might look something like this.
s = [ "a;b;c" ; "d;e"]
Now, calling split(s,";") gives this error message:
Error using split
Element 2 of the text contains 1 delimiters while the previous elements have 2. All
elements must contain the same number of delimiters.
Preprocess s to add trailing semicolons where needed.
num_fields = 3;
fix = count(s,";") < (num_fields - 1);
s(fix) = s(fix) + ";"
split(s,";")
+1 to what @Steve Eddins just said! A great way to get some attention to a function that matters to you is to do what @Jan Studnicka did here and provide a self-contained demonstration that current behaviour is sub-optimal.
tic and toc is certainly easier to remember than the syntax for using timeit. If I were to use timeit, I'd probably have to look up the documentation every time.
Really? I don't believe I've even opened the documentation once in the past 10 years. I just assume it needs an anonymous function (or handle) that can run without any further input.
So I must admit that it isn't as easy on the eyes as tic-toc, but is this really so much more convoluted than I think? I'm not trying to put you down, just genuinely curious.
My biggest hesitation with using timeit is that it requires a function handle as input. Often, what I am timing is ad hoc and does not align clearly with function or class boundaries.
I also fully understand the (typically longer) startup time for the first run and potential lack of precision when the runtime is small. So, I always make sure to run the tic/toc cycle at least once before recording the time and run it many times in a loop (if necessary) to generate an overall execution time that is on the order of at least 1–10 seconds from which to calculate the average runtime.
That is a well-motivated and sound approach.
Another factor to be aware of is that MATLAB can often execute code in functions faster than it can execute the same code entered interactively in the command window.
In this case the difference is pretty small, but you shouldn't use tic toc for comparing functions. timeit considers things like the slower first use of a function and statistical variation.
x = rand(1,50000);
t = string(x);
s1=str2double(t);
s2=double(t);
isequal(s1,s2)
timeit(@() str2double(t))
timeit(@() double(t))
Thanks for the comment Rik. I usually prefer running tic/toc several times over the timeit from the same reasons @Mike Croucher mentioned in his blog post Should we ban tic/toc? However, in this case, timeit would be better, because the reader can see only one run.
I encourage anyone interested in tic/toc vs. timeit to check out my 15-Jan-2025 blog post, "The Story of TIMEIT." It describes the original motivations for creating it, how it works, its move into MATLAB, and some discussion about why people may prefer one or the other solution. There is also some speculative discussion about possible enhancements to timeit.
interesting...
登录以参与
标签
