How to calculate number of floating-point operations (FLOPs) of a CNN model in MatLab

97 次查看(过去 30 天)
How to calculate number of floating-point operations (FLOPs) of a CNN model in MatLab?
  2 个评论
Walter Roberson
Walter Roberson 2023-2-13
The number of FLOPs needed to train the model, or the number of FLOPs needed to classify or predict ?
FLOPs is not used much anymore except for academic discussions as an approximation to algorithm costs similar to 'Big-O notation". There are a lot of things that FLOPs do not take into account. For example if you are computing on a GPU, then the common operation A*x + B might well be converted into a single hardware instruction to do Fused Muitiply And Add; https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation replacing two floating point operations with a single hardware instruction.
debojit sharma
debojit sharma 2023-2-14
Sir, actually I need to calculate FLOPS of my DL model to measure time complexity. This is required in my thesis. So, is there any MATLAB function with which I can calculate this? or please suggest any other method

请先登录,再进行评论。

采纳的回答

Walter Roberson
Walter Roberson 2023-2-14
Time complexity cannot be measured in FLOPs. Time complexity needs to be measured in "Big-O" notation. For a thesis you would typically measure average case and worst case (unless those are the same.)
Suppose that your program starts with a loop that initializes 1000 one-byte locations to 0 (for example for some kind of histogram, so independent of data size.) How many FLOPs is that? Is it 1000? Is it 1000 for the zeroing, plus 1000 increments for the for loop, plus 1000 comparisons? Is it 125 of each of those because the compiler unrolls the loop and then combines 8 adjacent byte zeros into a single floating point zero assignment? Is it "zero" because of Demand Page Zero hardware?
Whatever it is, for a given compiler it is constant, and that means that the complexity is constant, O(1) in big-O notation. But FLOPs cares about that.
If the algorithm involves a value to a positive integer power, then is the compiler going to use binary decomposition of the power to determine the optimal sequence of multiplicationqq? Is the compiler going with substitute log(), multiply, exponential and if so does that count as 3 FLOPs? Does the compiler emit some kind of hardware exponential instruction and so only count as one FLOP?
None of these are important to time complexity, so no-one measures FLOPs for that purpose.
FLOPs are a hold over from the days when instructions were performed entirely sequentially, with no pipelining and when there was only one ALU (Arithmetic Logic Unit), and were used to calculate actual runtime. FLOPs divided by MIPS (millions of instructions per second) to get seconds of run time. That is not time complexity.
  2 个评论
debojit sharma
debojit sharma 2023-2-18
Is there any equivalent function/code in matlab that can perform the following operation as shown in the following python code:
from keras.applications.resnet50 import ResNet50
from keras.applications.vgg16 import VGG16
from keras.applications.mobilenet import MobileNet
from net_flops import net_flops
model = VGG16(weights=None, include_top=True, pooling=None,input_shape=(224,224,3))
model.summary()
#Prints a table with the FLOPS at each layer and total FLOPs
net_flops(model,table=True)
Walter Roberson
Walter Roberson 2023-2-18
No, there is not.
There used to be a different File Exchange contribution that allowed you to annotate your code to get approximate flops counts, but I am having difficultly locating it at the moment.
In theory it is possible to read the CPU Cycle Counters from the processors -- but those are not necessarily meaningful. See https://devblogs.microsoft.com/oldnewthing/20160429-00/?p=93385
On Windows (not sure about Linux, but not on MacOS) you might be able to use .NET to access performance counters; see https://michaelscodingspot.com/performance-counters/

请先登录,再进行评论。

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Deep Learning Toolbox 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by