清空筛选条件
清空筛选条件

How does one apply max(0,z) element-wise in a vectorized way (apply ReLu linear rectifier element-wise)

25 次查看（过去 30 天）
Brando Miranda
Brando Miranda 2016-3-25
回答： Joss Knight 2016-3-29
I wanted to implement something similar to applying a mathematical function element-wise to a matrix. For example something like:
kernel_matrix = max(0, W' * X + b) %linear rectifier element-wise vectorized code
in a manner that the code is vectorized. Is this possible in matlab? So far the max documentation doesn't seem super helpful.
Ideally, it would be important that this element-wise way of doing this is supported by GPU arrays too.
So for example if our matrix we want to pass through the ReLu is:
>> A
A =
8 1 6
3 5 7
4 9 2
and the following is the threshold the neurons/ReLu should activate:
B =
1 7 9
3 1 1
8 4 4
then the ReLu should only activate when A(i,j) - B(i,j) >= 0 to produce:
activation =
8 0 0
3 5 7
0 9 0
  2 个评论
Roger Stafford
Roger Stafford 2016-3-25
It isn't clear what matrix the term "element-wise" is to apply to. Please give information about the sizes of the various variables here and what "element-wise" is to apply to.
Brando Miranda
Brando Miranda 2016-3-25
it should just work just like
exp(A)
on any matrix, the dimensions or size, elements don't matter.
I think figured out how to do it though. Will answer now.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（2 个）

Brando Miranda
Brando Miranda 2016-3-25
编辑：Brando Miranda 2016-3-25
The only way that occurred to me how to do it was to simply compare the matrix with the number 0 (or the individual thresholds if you are dealing with a ReLu) and then multiply the original matrix with the logical matrix. In particular:
Z = rand(D,N); %pre-activation matrix
logical_matrix = (Z >= 0); %which entries should activate neurons
A = Z .* logial_matrix;
if you are dealing with a threshold though and want to activate for individual threshold, then one could do instead:
X = rand(D,N); %X original data matrix
logical_matrix = ((W' * Z - B) >= 0); %which entries should activate neurons based on threshold B
A = (W' * Z + B) .* logial_matrix;
the idea is to implement ReLu(w' * x + b) for element wise matrices which the above does.
I have not checked if its fast in GPUs but it should be better than doing a for loop if one has access to GPU. Will check shortly.
Joss Knight
Joss Knight 2016-3-29
Not quite sure whether your point is clear because this is exactly how the two-argument form of max works - element-wise.
>> max(0, magic(3)-5)
ans =
3 0 1
0 0 2
0 4 0

类别

在 Help Center 和 File Exchange 中查找有关 Get Started with MATLAB 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by