import matplotlib.pylab as plt
import numpy as np
x= np.arange(-10,10,0.1)
f= 1 / (1 + np.exp(-x))
plt.plot(x,f)
plt.xlabel('x')
plt.ylabel('f(x)')
plt.show()
Taking a look at what np.arange does.
print (np.arange(-10,10,1))
[-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9]
Returns an ndarray with values of -10 to 10 in 1 step intervals. We can test that it’s an array with the following.
https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.arange.html
myArray = (np.arange(-10,10,1))
print (myArray[3])
-7
In the above we recall the 4th element of the array (because all arrays are indexed from 0.
Taking a look at the line
f = 1 / (1 + np.exp(-x))
This is expressed in maths as:
$latex f(z) = \frac{1}{1+ exp(-z)}&s=4 $
The following lines use the matplotlibrary to display the data and draw labels on the x and y axis.
Playing with the code a little bit and changing the step size, we can easily see how the step size alters the sigmoid.
import matplotlib.pylab as plt
import numpy as np
x= np.arange(-10,10,4)
f= 1 / (1 + np.exp(-x))
plt.plot(x,f)
plt.xlabel('x')
plt.ylabel('f(x)')
plt.show()
Effect of different weights on sigmoid
import matplotlib.pylab as plt
import numpy as np
weight1 = 0.5
weight2 = 1.0
weight3 = 2.0
weight4 = 3.0
weight5 = 4.0
label1 = 'weight1 = 0.5'
label2 = 'weight2 = 1.0'
label3 = 'weight3 = 2.0'
label4 = 'weight4 = 3.0'
label5 = 'weight5 = 4.0'
x= np.arange(-10,10,0.1)
for w, l in [(weight1, label1),(weight2, label2),(weight3, label3),(weight4, label4),(weight5, label5)]:
f = 1 / (1 + np.exp(-x*w))
plt.plot(x, f, label=l)
plt.xlabel('x')
plt.ylabel('h_w(x)')
plt.legend(loc=2)
plt.show()
Different weight adjust the shape of the sigmoid, i.e. change the relationship between the input values and the output values.
Adding bias to a sigmoid
In the image above we can see that gradual increases in x gradually increase the output, although higher weight values cause a sharper increase. We can see that the central pivot of activation as around 0x.
If we want to shift these curves to the left or right, we have to use a bias.
import matplotlib.pylab as plt
import numpy as np
weight = 3
b1 = -10
b2 = 0
b3 = 10
biaslabel1 = 'bias = -10'
biaslabel2 = 'bias = 0'
biaslabel3 = 'bias = 10'
x= np.arange(-10,10,0.1)
for b, l in [(b1, biaslabel1),(b2, biaslabel2),(b3, biaslabel3)]:
f = 1 / (1 + np.exp(-(x*weight+b)))
plt.plot(x, f, label=l)
plt.xlabel('x')
plt.ylabel('h_wb(x)')
plt.legend(loc=2)
plt.show()
