How to calculate number of floating-point operations (FLOPs) of a CNN model in MatLab
97 次查看(过去 30 天)
显示 更早的评论
How to calculate number of floating-point operations (FLOPs) of a CNN model in MatLab?
2 个评论
Walter Roberson
2023-2-13
The number of FLOPs needed to train the model, or the number of FLOPs needed to classify or predict ?
FLOPs is not used much anymore except for academic discussions as an approximation to algorithm costs similar to 'Big-O notation". There are a lot of things that FLOPs do not take into account. For example if you are computing on a GPU, then the common operation A*x + B might well be converted into a single hardware instruction to do Fused Muitiply And Add; https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation replacing two floating point operations with a single hardware instruction.
采纳的回答
Walter Roberson
2023-2-14
Time complexity cannot be measured in FLOPs. Time complexity needs to be measured in "Big-O" notation. For a thesis you would typically measure average case and worst case (unless those are the same.)
Suppose that your program starts with a loop that initializes 1000 one-byte locations to 0 (for example for some kind of histogram, so independent of data size.) How many FLOPs is that? Is it 1000? Is it 1000 for the zeroing, plus 1000 increments for the for loop, plus 1000 comparisons? Is it 125 of each of those because the compiler unrolls the loop and then combines 8 adjacent byte zeros into a single floating point zero assignment? Is it "zero" because of Demand Page Zero hardware?
Whatever it is, for a given compiler it is constant, and that means that the complexity is constant, O(1) in big-O notation. But FLOPs cares about that.
If the algorithm involves a value to a positive integer power, then is the compiler going to use binary decomposition of the power to determine the optimal sequence of multiplicationqq? Is the compiler going with substitute log(), multiply, exponential and if so does that count as 3 FLOPs? Does the compiler emit some kind of hardware exponential instruction and so only count as one FLOP?
None of these are important to time complexity, so no-one measures FLOPs for that purpose.
FLOPs are a hold over from the days when instructions were performed entirely sequentially, with no pipelining and when there was only one ALU (Arithmetic Logic Unit), and were used to calculate actual runtime. FLOPs divided by MIPS (millions of instructions per second) to get seconds of run time. That is not time complexity.
2 个评论
Walter Roberson
2023-2-18
No, there is not.
You could experiment with https://www.mathworks.com/matlabcentral/fileexchange/50608-counting-the-floating-point-operations-flops
There used to be a different File Exchange contribution that allowed you to annotate your code to get approximate flops counts, but I am having difficultly locating it at the moment.
I suggest you also read https://www.mathworks.com/matlabcentral/answers/839375-flops-of-dag-neural-network#answer_709465
In theory it is possible to read the CPU Cycle Counters from the processors -- but those are not necessarily meaningful. See https://devblogs.microsoft.com/oldnewthing/20160429-00/?p=93385
On Windows (not sure about Linux, but not on MacOS) you might be able to use .NET to access performance counters; see https://michaelscodingspot.com/performance-counters/
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!