The input to every LSTM network layer must be 3D. But what does this mean?

Let’s consider a sequence of multiple time steps with 1 feature. This could be a sequence of 10 values, each value representing the close price of candlestick charts for example (therefore 1 feature). I know these values are far from close prices from a chart but this is just to keep things simple

1, 2, 3, 4, 5, 6, 7, 8, 9, 10

This sequence can be defined as a numpy array.

from numpy import array data = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

The *reshape() *function in numpy can be used to reshape the above 1 dimentional array into a 3 dimenstional array, with 1 sample, 10 time steps, and 1 feature.

This function takes 1 argument which is a tuple that defines the new shape of the array.

data = data.reshape((1, 10, 1))

Let’s look at the output of the full code.

from numpy import array data = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) data = data.reshape((1, 10, 1)) print (data) print (data.shape)

[[[ 1]

[ 2]

[ 3]

[ 4]

[ 5]

[ 6]

[ 7]

[ 8]

[ 9]

[10]]]

(1, 10, 1)

The data above is ready to be used in an LSTM network (note: normalization would be required as well but that’s not covered here)

model = Sequential() model.add(LSTM(32, input_shape=(10, 1))) model.add(Dense(1))

Let’s consider data that has 2 features. For example data that includes close prices and open prices from chart data.

from numpy import array data = array([[1,1.4], [2,2.1], [3,21], [4,41], [5,5.1], [6,6.4], [7,7], [8,8.3], [9,6.7], [10,8]]) data = data.reshape((1, 10, 2)) print (data) print (data.shape)

[[[ 1. 1.4]

[ 2. 2.1]

[ 3. 21. ]

[ 4. 41. ]

[ 5. 5.1]

[ 6. 6.4]

[ 7. 7. ]

[ 8. 8.3]

[ 9. 6.7]

[10. 8. ]]]

(1, 10, 2)

The code above represents a 10 step time sequence with 2 features (open and close prices). To reshape this into a 3D array (code highlighted) we would use 1, 10, 2 in the as arguments in the reshape() function.

This data would then be passed to the LSTM network as follows.

model = Sequential() model.add(LSTM(32, input_shape=(10, 2))) model.add(Dense(1))

It’s worth noting, when we output with:

print (data.shape)

A returned value of (1, 10, 2) signifies:

1 Sample

10 timesteps

2 features

More information in this reshape() function can be found here.

https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.reshape.html