6 points speckx 1 hour ago 1 comments
gaal_dornick 3 minutes ago | parent
Huge +1. We used a custom decision model to improve recall of our retrieval pipeline by ~1.5x while reducing cost by ~70x compared to just Opus. We used the custom decision model to quickly and cheaply drop a larger number of egregious results and have the LLM only look at the cases the smaller model was not sure about.
We believe in this pattern so much that we are building Princi (https://princi.ai/) to help teams train and host their own custom decision models. It is great to see this approach gaining traction.