I trained an LSTM for next-word prediction on text data.The training loss decreases normally.But when I generate text, it repeats the same token again and again.It feels like the model is ignoring the sentence.
Decode Trail Latest Questions
My RNN works fine on short sequences.When I give it longer inputs, predictions become random.Loss increases with sequence length.It feels like the model forgets earlier information.