You're right: I wish OpenAI could find a way to "donate" GPT-2 or GPT-3 to the CHM, or some open archive.
I feel like that generation of models was around the point where we were getting pleasantly surprised by the behaviors of models. (I think people were having fun translating things into sonnets back then?)