The input to every LSTM network layer must be 3D. But what does this mean?
Let’s consider a sequence of multiple time steps with 1 feature. This could be a sequence of 10 values, each value representing the close price of candlestick charts for example (therefore 1 feature). I know these values are far from close prices from a chart but this is just to keep things simple
1, 2, 3, 4, 5, 6, 7, 8, 9, 10
This sequence can be defined as a numpy array.
from numpy import array
data = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
The reshape() function in numpy can be used to reshape the above 1 dimentional array into a 3 dimenstional array, with 1 sample, 10 time steps, and 1 feature.
This function takes 1 argument which is a tuple that defines the new shape of the array.
data = data.reshape((1, 10, 1))
Let’s look at the output of the full code.
from numpy import array
data = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
data = data.reshape((1, 10, 1))
print (data)
print (data.shape)
[[[ 1]
[ 2]
[ 3]
[ 4]
[ 5]
[ 6]
[ 7]
[ 8]
[ 9]
[10]]]
(1, 10, 1)
The data above is ready to be used in an LSTM network (note: normalization would be required as well but that’s not covered here)
model = Sequential()
model.add(LSTM(32, input_shape=(10, 1)))
model.add(Dense(1))
Let’s consider data that has 2 features. For example data that includes close prices and open prices from chart data.
from numpy import array
data = array([[1,1.4], [2,2.1], [3,21], [4,41], [5,5.1], [6,6.4], [7,7], [8,8.3], [9,6.7], [10,8]])
data = data.reshape((1, 10, 2))
print (data)
print (data.shape)
[[[ 1. 1.4]
[ 2. 2.1]
[ 3. 21. ]
[ 4. 41. ]
[ 5. 5.1]
[ 6. 6.4]
[ 7. 7. ]
[ 8. 8.3]
[ 9. 6.7]
[10. 8. ]]]
(1, 10, 2)
The code above represents a 10 step time sequence with 2 features (open and close prices). To reshape this into a 3D array (code highlighted) we would use 1, 10, 2 in the as arguments in the reshape() function.
This data would then be passed to the LSTM network as follows.
model = Sequential()
model.add(LSTM(32, input_shape=(10, 2)))
model.add(Dense(1))
It’s worth noting, when we output with:
print (data.shape)
A returned value of (1, 10, 2) signifies:
1 Sample
10 timesteps
2 features
More information in this reshape() function can be found here.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.reshape.html