How does one apply max(0,z) element-wise in a vectorized way (apply ReLu linear rectifier element-wise)

26 次查看（过去 30 天）
Brando Miranda
Brando Miranda 2016-3-25
回答： Joss Knight 2016-3-29
I wanted to implement something similar to applying a mathematical function element-wise to a matrix. For example something like:
kernel_matrix = max(0, W' * X + b) %linear rectifier element-wise vectorized code
in a manner that the code is vectorized. Is this possible in matlab? So far the max documentation doesn't seem super helpful.
Ideally, it would be important that this element-wise way of doing this is supported by GPU arrays too.
So for example if our matrix we want to pass through the ReLu is:
>> A
A =
8 1 6
3 5 7
4 9 2
and the following is the threshold the neurons/ReLu should activate:
B =
1 7 9
3 1 1
8 4 4
then the ReLu should only activate when A(i,j) - B(i,j) >= 0 to produce:
activation =
8 0 0
3 5 7
0 9 0
  2 个评论
Roger Stafford
Roger Stafford 2016-3-25
It isn't clear what matrix the term "element-wise" is to apply to. Please give information about the sizes of the various variables here and what "element-wise" is to apply to.
Brando Miranda
Brando Miranda 2016-3-25
it should just work just like
exp(A)
on any matrix, the dimensions or size, elements don't matter.
I think figured out how to do it though. Will answer now.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（2 个）

Brando Miranda
Brando Miranda 2016-3-25
编辑：Brando Miranda 2016-3-25
The only way that occurred to me how to do it was to simply compare the matrix with the number 0 (or the individual thresholds if you are dealing with a ReLu) and then multiply the original matrix with the logical matrix. In particular:
Z = rand(D,N); %pre-activation matrix
logical_matrix = (Z >= 0); %which entries should activate neurons
A = Z .* logial_matrix;
if you are dealing with a threshold though and want to activate for individual threshold, then one could do instead:
X = rand(D,N); %X original data matrix
logical_matrix = ((W' * Z - B) >= 0); %which entries should activate neurons based on threshold B
A = (W' * Z + B) .* logial_matrix;
the idea is to implement ReLu(w' * x + b) for element wise matrices which the above does.
I have not checked if its fast in GPUs but it should be better than doing a for loop if one has access to GPU. Will check shortly.
Joss Knight
Joss Knight 2016-3-29
Not quite sure whether your point is clear because this is exactly how the two-argument form of max works - element-wise.
>> max(0, magic(3)-5)
ans =
3 0 1
0 0 2
0 4 0

类别

在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by