Vanilla recurrent neural networks (RNNs) form the basis of more sophisticated models, such as LSTMs and GRUs. There are lots of great articles, books, and videos that describe the functionality, mathematics, and behavior of RNNs so, don't worry, this isn't yet another rehash. (See below for a list of resources.) My goal is to present an explanation that avoids the neural network metaphor, stripping it down to its essence—a series of vector transformations that result in embeddings for variable-length input vectors.
Thank you for this. As someone who who understands linear algebra well, I feel the neural network formalism obfuscates the core meaning of these processes. It's good for describing composition, but not good for understanding the parts in-between.
Looking forward to your take on LSTMs and transforms ;).
