Step function backprop

Article from www.avaye.com


Step function backprop

In order to apply backpropagation, the activation function needs to be derivable. Unfortunately, when the activation function is a step function, it cannot be derived. A workaround is to use a sigmoid activation function with a steep curve.

1/(1+exp(-ax))

Make a as large as you want for the curve to be steep enough.


< Back

All content copyrighted by Avaye.com