9 points alexmolas 1 hour ago 9 comments
kantahayashi 59 minutes ago | parent
I tested Jev with a fair die 400 times without telling it the die result. The true probability of face 1 is 1/6, but Jev always chose face 1 and the probability it returned was about 83%. I also tested with a fair coin 200 times and got 0.92 probability.
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
Write-up: "Jev Does Not Play Dice" https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
alexmolas 49 minutes ago | parent
But "problems about actual probabilities whose answers can't be known at all" are exactly the problems where calibration is important. Since calibration is one of the big claims about Jev I'd expect it to perform well in these problems.
kantahayashi 20 minutes ago | parent
I agree. I think it's odd behavior too. Jev should be good at actual probability problems given the phrase "calibrated probabilities" TypeSafe uses for Jev. Maybe the reason is the data used in their training method (RLCD). If all the data consists of problems with a correct answer, I think this kind of odd behavior could happen.
orbital-decay 8 minutes ago | parent
The reason and the word you're looking for is mode collapse. It happens during RL, it's not a dataset problem, and it's the reason for many issues with LLM writing and all sorts of issues in all types of models. The result you got is 100% predictable and unsurprising, as it's a neglected problem nobody really rushes to solve as current big model training is laser focused on one thing and one tool.
tomrod 15 minutes ago | parent
Echoes a bit of a philosophical distinction with a long history: "Knightian Uncertainty" versus "Probability".
edot 7 minutes ago | parent
Hah! I did the exact same tests as you! I found that if you give it the choice to say "not sure", it picks that 100% of the time. But if you pin it in a corner, then yes it does these weird things. Also yes, the continuous options were much more accurate than the choices. Not sure why that is.
conmod278 11 minutes ago | parent
In future, we will see intiatives similar to OpenStreetMap for Textual data or Web similar to high quality non-contaminated steel.
daft_pink 7 minutes ago | parent
I'm really looking for a multi-modal image capable version of Jev.
If we could get machine learning type results on images without training, that would be fantastic.
simonw 2 minutes ago | parent
Fine tuning LLMs has turned out to be mostly not worth the effort, but I wonder if fine tuning Jev-style models will turn out to be a whole lot more useful.