- Sequences are padded to the 'longest' in a mini-batch. More on that here: Sequence Options
- The same initial value for Hidden state is used across all sequences and mini-batches. The column dimension is 1 because the hidden state is same for all sequences. It is not related to the min-batch size being 1. Refer docs here. This is also related to your follow-up comment from another question. The column dimension for 'HiddenState' property would never be 2.
You mention this:
I'm trying to have a stateless lstm since each time series is independent from one another in my case so I use a minibatch size= 1.
and this:
It does not make sense to me to use the same initial hidden state [...] for all the batches because then It seems to me that it loses its individual properties [...]
An LSTM network trains on a dataset of one particular kind. It's learnable parameters are 'InputWeights', 'RecurrentWeights' and 'Bias' as seen in this example here. Every LSTM has a fixed initial 'HiddenState' property. On training a network, the input and recurrent weights are learnt and adapt to the required targets / error which needs to be minimized. The initial hidden state would only be a small influence when you have relatively long sequences.
So, to clear some confusion, the individual properties of sequences from a dataset are captured in the input and recurrent weights. Every LSTM has to be trained on some dataset to learn from it. I'm not sure what you mean by stateless LSTM or about each time series (sequence) being independent from each other.
Take the example of the Japanese Vowel Dataset. All the sequences are separate from each other but they also share similarities in the sense that they are all about vowels. Training an LSTM on such a dataset would mean that the network captures the individual properties in its weights.
