theano7_activation_function.py

# View more python tutorials on my Youtube and Youku channel!!!

# Youtube video tutorial: https://www.youtube.com/channel/UCdyjiB5H8Pu7aDTNVXTTpcg
# Youku video tutorial: http://i.youku.com/pythontutorial

# 7 - Activation function

"""
The available activation functions in theano can be found in this link:
http://deeplearning.net/software/theano/library/tensor/nnet/nnet.html

The activation functions include but not limited to softplus, sigmoid, relu, softmax, elu, tanh...

For the hidden layer, we could use relu, tanh, softplus...
For classification problems, we could use sigmoid or softmax for the output layer.
For regression problems, we could use a linear function for the output layer.

"""