## Sigmoids using matplot lib

import matplotlib.pylab as plt
import numpy as np
x= np.arange(-10,10,0.1)
f= 1 / (1 + np.exp(-x))
plt.plot(x,f)
plt.xlabel('x')
plt.ylabel('f(x)')
plt.show()


Taking a look at what np.arange does.

print (np.arange(-10,10,1))

[-10  -9  -8  -7  -6  -5  -4  -3  -2  -1   0   1   2   3   4   5   6   7    8   9]

Returns an ndarray with values of -10 to 10 in 1 step intervals. We can test that it’s an array with the following.

https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.arange.html

myArray = (np.arange(-10,10,1))
print (myArray[3])

 -7

In the above we recall the 4th element of the array (because all arrays are indexed from 0.

Taking a look at the line

f = 1 / (1 + np.exp(-x))

This is expressed in maths as:

$latex f(z) = \frac{1}{1+ exp(-z)}&s=4$

The following lines use the matplotlibrary to display the data and draw labels on the x and y axis.

Playing with the code a little bit and changing the step size, we can easily see how the step size alters the sigmoid.

import matplotlib.pylab as plt
import numpy as np
x= np.arange(-10,10,4)
f= 1 / (1 + np.exp(-x))
plt.plot(x,f)
plt.xlabel('x')
plt.ylabel('f(x)')
plt.show()


### Effect of different weights on sigmoid

import matplotlib.pylab as plt
import numpy as np
weight1 = 0.5
weight2 = 1.0
weight3 = 2.0
weight4 = 3.0
weight5 = 4.0
label1 = 'weight1 = 0.5'
label2 = 'weight2 = 1.0'
label3 = 'weight3 = 2.0'
label4 = 'weight4 = 3.0'
label5 = 'weight5 = 4.0'
x= np.arange(-10,10,0.1)
for w, l in [(weight1, label1),(weight2, label2),(weight3, label3),(weight4, label4),(weight5, label5)]:
f = 1 / (1 + np.exp(-x*w))
plt.plot(x, f, label=l)
plt.xlabel('x')
plt.ylabel('h_w(x)')
plt.legend(loc=2)
plt.show()


Different weight adjust the shape of the sigmoid, i.e. change the relationship between the input values and the output values.

### Adding bias to a sigmoid

In the image above we can see that gradual increases in x gradually increase the output, although higher weight values cause a sharper increase. We can see that the central pivot of activation as around 0x.

If we want to shift these curves to the left or right, we have to use a bias.

import matplotlib.pylab as plt
import numpy as np

weight = 3
b1 = -10
b2 = 0
b3 = 10
biaslabel1 = 'bias = -10'
biaslabel2 = 'bias = 0'
biaslabel3 = 'bias = 10'

x= np.arange(-10,10,0.1)
for b, l in [(b1, biaslabel1),(b2, biaslabel2),(b3, biaslabel3)]:
f = 1 / (1 + np.exp(-(x*weight+b)))
plt.plot(x, f, label=l)
plt.xlabel('x')
plt.ylabel('h_wb(x)')
plt.legend(loc=2)
plt.show()