We can see that LSTMs can emulate this kind of logic, but they also make mistakes.
Humans can also make mistakes when processing language. We're still better than LSTM's, but I'm not sure we can claim a qualitative difference.
Furthermore, even though we can process sentences with very deep embedding like "The rat the cat the dog bit chased escaped", my intuition is that we are not using our normal language processing systems for that. When I read that sentence, I just fail to process it and then I invoke my logic systems to try to determine the structure and decode it, in a way that feels totally different from processing a normal sentence (I'm not understanding it in real time, in a natural way, but rather solving a small puzzle). So I personally don't find the Chomskyan arguments based on that kind of corner cases very convincing.
That is indeed a contrived example of recursion, but recursion (in the sense of embedding) can be much simpler and easier to parse. For example:
"John, my friend from high school, who married your cousin, Mary, is coming over for dinner".
This sort of embedding is what makes human language infinite in scope- you can keep embedding sub-sentences for ever, and so you can produce new utterances forever.
This ability to infinitely extend and recombine the meaning of utterances is what gives human language its expressive power, and what is absent from animal languages, so far as we know.
Examples like that are parsable because they are similar to what we would call, in programming, tail recursion (i.e., recursion that doesn't really need recursion). It's true that you can embed an infinite number of subsentences ("John, my friend from high school, who married your cousin, Mary, who had an affair with the bartender, Jack, who hated his sister, Lisa, who was a fan of Lady Gaga, is coming over for dinner") but you only need two "stack frames", one to remember John and the other for the rest.
The middle part is basically equivalent to saying "Mary had an affair with the bartender, Jack. Jack hated his sister, Lisa. Lisa was a fan of Lady Gaga". My intuition is that it's parsed basically as separate sentences. Once you finish one of them you can just forget it, you only need to remember John (as there is more information about him in the end). Sentences where you need to remember more elements (i.e., you actually need unbounded recursion) become unparsable in real time as my previous example.
Of course, I don't have scientific evidence to back the things I'm saying, it's just intuition, but the same can be said of the Chomskyan theories.
Humans can also make mistakes when processing language. We're still better than LSTM's, but I'm not sure we can claim a qualitative difference.
Furthermore, even though we can process sentences with very deep embedding like "The rat the cat the dog bit chased escaped", my intuition is that we are not using our normal language processing systems for that. When I read that sentence, I just fail to process it and then I invoke my logic systems to try to determine the structure and decode it, in a way that feels totally different from processing a normal sentence (I'm not understanding it in real time, in a natural way, but rather solving a small puzzle). So I personally don't find the Chomskyan arguments based on that kind of corner cases very convincing.