12 points koljab 8 hours ago 1 comments
There had to be a method to get this signal closer to what I felt is optimal. My idea was to use a 10 ms energy detector with a threshold that adapts to environment or room noise. Which worked amazingly well. It created more false positives but in my speculative generation pipeline that would not matter. If it's wrong and the user continues speaking I can abort the pipeline fast.
I benchmarked my solution over ~3200 turns from my assistant logs and it came out ~110 ms faster initially. Still 100 ms now after some a fix which was needed for quiet speech.
I made it public if you want to have a look. If you measure it on your data I really appreciate any feedback. There's a benchmark script in the repo.
Thx for reading, have a good day folks.
com2kid 20 minutes ago | parent
Run a tiny shitty llm, one that is super fast. Or better yet try out a decision model now days.
Feed it the previous conversation turn and all the STT (ASR output) you have so far and just ask it for a probability that the user is done speaking.
Use that probability to shorten your VAD time.
It isn't as good as a true speech to speech model but this will cut a few hundred ms compared to pure VAD. Oh as a bonus it also lets you wait longer when the user isn't done talking yet.