Bryan Caplan:

It seems like you and other AIs are bad Bayesians. When presented with counterevidence, you should move toward or against the presenter with roughly equal probability. It depends on whether the argument was better or worse than expected. In fact, you seem to either move toward the speaker or just not change.

ChatGPT:

I think you’ve identified a genuine and important failure mode in AI reasoning: LLMs tend to treat a plausible counterargument as evidence in favor of its conclusion, rather than comparing the counterargument to what they should have expected to hear.

Your Bayesian point is correct, with one qualification: there’s no general theorem that updating toward and away from a speaker should happen with equal probability. But under appropriate calibration assumptions, both directions should occur, and an argument that’s weaker than expected should cause a negative update—even if the argument is perfectly valid as far as it goes.

Here is the full dialogue.  But will they try to fix this?  Should they try to fix this?  Can you undo this with good enough prompting?  The scope and limits of Bayesian are truly at stake here…

The post Are LLMs good or bad Bayesians? appeared first on Marginal REVOLUTION.

#LLMs #good #bad #Bayesians