40 points hn-aarvin 1 hour ago 20 comments
quantisan 29 minutes ago | parent
ummonk 25 minutes ago | parent
js4ever 24 minutes ago | parent
copperx 22 minutes ago | parent
mjburgess 21 minutes ago | parent
Very high hallucination rates too.
SwellJoe 20 minutes ago | parent
azan_ 19 minutes ago | parent
andunieee 10 minutes ago | parent
hadlock 17 minutes ago | parent
Yahivin 15 minutes ago | parent
asadm 8 minutes ago | parent
danpalmer 4 minutes ago | parent
YuechenLi 17 minutes ago | parent
But my question is, what does this benchmark actually measure about the LLM's engineering abilities? It just looks like a page of random bar graphs to me.
andunieee 11 minutes ago | parent
Yahivin 6 minutes ago | parent
https://one.google.com/about/google-ai-plans/ had a 75% off on the Pro plan ($5/mo for 3 months) and limits are decent.
dudus 4 minutes ago | parent