Surely an LLM could parse the chat and keep a public FAQ up to date? Somebody must already have implemented this, but last time I looked I still came empty handed...
Yeah, if you want to have an inconsistent documentation with lots of subtle and non-subtle errors, using a non-deterministic black box to generate it is a great way to achieve that.
From some experiments with a discord bot, the non-deterministic responses you get from an LLM are the big killer for that use case. Ask the model to summarize 100 lines of chat 10 times and you will get 10 different outputs, all worded subtly differently, with different headlines missing each time. (Even on state-of-the-art GPT-4)
