If you're wondering why OpenAI is bothering to fight the never ending war to align their models here it is. Misguided people are already using it for tasks like this and the blame falls on the model provider when it reflects our own biases back at us.
It would be fascinating to explore perhaps the greatest mirror that has ever existed pointed back at humanity and show near indisputable proof of the many many unconscious biases that folks constantly deny. You could even have models trained on different time periods to see how those biases evolve.
But these things are designed to be tools and nobody expects a drill to be ableist so you have a weird amount of responsibility foisted upon by your own existence to do something. Lest you knowingly amplify the very worst parts of ourselves when it's deployed.
And this isn't theoretical, folks in CPS are right now deploying this to synthesize and make recommendations on cases. It's going to be catastrophic all the while every agency fights to be on the waitlists because it's the first thing that can take work off their plate.
I don't know what you or they are expecting to get out of it. The only good answers you're likely to get about what people are like (on average) were copied from Wikipedia or some other random website.
As a source of hints or fancy autocomplete, an LLM can be pretty great, but it's not a way of doing statistics on people or even on texts, and you can't trust it blindly.
It would be fascinating to explore perhaps the greatest mirror that has ever existed pointed back at humanity and show near indisputable proof of the many many unconscious biases that folks constantly deny. You could even have models trained on different time periods to see how those biases evolve.
But these things are designed to be tools and nobody expects a drill to be ableist so you have a weird amount of responsibility foisted upon by your own existence to do something. Lest you knowingly amplify the very worst parts of ourselves when it's deployed.
And this isn't theoretical, folks in CPS are right now deploying this to synthesize and make recommendations on cases. It's going to be catastrophic all the while every agency fights to be on the waitlists because it's the first thing that can take work off their plate.