A SO post usually has an answer that works and is a reasonably good idea. ChatGPT lacks the ability to see if its responses pass even a basic sanity test. Sometimes it's answers are good and sometimes they are totally wrong. Hard for a person asking the question to determine which answer they got.
this line of code has at least two gotchas. if you run it to test it out, there is a good chance you will destroy your data. It is otherwise a valid line of code.
> With code interpreter, ChatGPT can check that it's code at least runs without errors
is not a solution to gotchas/caveats that build up over time that ChatGPT won't know about. ChatGPT can warn about caveats from that line of code because it's old code. It does not have a magical defense that will allow it to warn about caveats, errors, security risks, or deprecations that are introduced after its knowledge cutoff.
The best solution that we've been able to find so far is for ChatGPT to do a search and feed the results into its context. But that has its own problems, and importantly it only works if the underlying search engines are still returning up-to-date relevant results and documentation from sites like StackOverflow.
Yes, but ultimately it's still a "statistically likely" answer and not the "right" one. Often the two overlap and maybe one day they'll make it an overlapping venn diagram. But that day is not today.
