17 points surprisetalk 1 hour ago 8 comments
iugtmkbdfil834 1 hour ago | parent
Well, it was nice to have a functional models for a bit. Shame it didn't last long.
stratos123 49 minutes ago | parent
Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed:
- they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use.
- they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine.
- they are going to get a separate dedicated evaluator specifically for bioweapon risks