"Note that, even if Grover fails to detect a given piece as fake, our findings suggest that releasing many such articles taken together would be relatively easy to spot. Thus, if a source of Neural Fake News disseminates a large number of articles, Grover will be increasingly capable of spotting these articles as malicious."

That doesn't solve the exploit above

Of course, to retrain Grover, you'd have to already know that they were machine-generated.

The generated news also needs to look like it was written by a human and be readable by the average human.

So you need a step that filters out obvious junk, maybe by eyeballing or with another net.

It is possible that the only way to beat grover is rubbish such as contorted, but valid, grammer or overuse of synonyms.