YHK - yet another hacker news clone

Voxtral 3

165 points ilreb 1 hour ago 41 comments

drum55 1 hour ago | parent

Closed model unfortunately, continues the theme of TTS / STT being really not great for local deployments.

causal 1 hour ago | parent

> Available via Mistral’s EU-hosted API or self-deployment

What does this mean then?

verdverm 1 hour ago | parent

You have to call sales to get access for on-prem deployment and the models

If it were open weights, it would be on Huggingface and they would have put a link in the post

j-pb 1 hour ago | parent

Breeze is pretty good imho.

aliljet 1 hour ago | parent

What options would you run locally at this stage? I'm finding a ton of success with local models in this space ..

7777777phil 1 hour ago | parent

I have been using different Parakeet models for STT over the past months and been pretty impressed with the local performance.

deepsquirrelnet 58 minutes ago | parent

You could try ours. They're very small and english only right now (training Spanish now). But they're a CPU optimized architecture and even the largest size will work on a single thread with realtime streaming.

https://huggingface.co/collections/futo-org/asr4all

You can run them client-side using wasm-simd. I setup a space for the small sized model. There's a big step up in accuracy going to the medium size though:

https://huggingface.co/spaces/futo-org/asr4all-client-side

Bolwin 17 minutes ago | parent

That's fascinating. Are these for your keyboard?

You may have gone too fast honestly. Something like parakeet runs fast enough and seems to have much lower wer

verdverm 57 minutes ago | parent

I'm looking for a local model that can do diarization, have any suggestions for this?

I have https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize open in a tab to try out

A leader board on HF: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

xnx 32 minutes ago | parent

Parakeet. For my use it seems about equal to Whisper in WER, but is >10x faster.

Mashimo 1 hour ago | parent

Dang, this could have been fun for home assistent.

IsaacZerh 42 minutes ago | parent

Funny because Gradium just released a local TTS that runs on many devices

https://x.com/GradiumAI/status/2112563128487514512?s=20

dr_hooo 40 minutes ago | parent

Too bad it's not open, unlike the previous version.

However, I found Qwen ASR and QwenTTS to be quite good, and not too large. Also https://github.com/kyutai-labs/pocket-tts was also great (but only a few languages)

sorenjan 16 minutes ago | parent

New version of PocketTTS is apparently censored: https://github.com/kyutai-labs/pocket-tts/issues/390

dr_hooo 9 minutes ago | parent

Oh, wow, I wasn't aware. Weird choice...

Lucasoato 1 hour ago | parent

Wait, did they release any new model or just changed the voxtral page? Voxal Mini 3B rocks hard in Handy!

aliljet 1 hour ago | parent

One more hard question. How real time is this conversation? They're showing something with zero lag between responses.

satvikpendem 1 hour ago | parent

Closed model means I'm immediately not interested. There are lots of closed STT/TTS models that are insanely good, better than any open model, so unless this is open then not sure I'd ever use this.

dymk 55 minutes ago | parent

I think the special sauce of this isn't the STT/TSS (although it does appear to be quite good) - it's the integration of a real-time reasoning model to actually do something with the raw text of the conversation, and whatever they're doing to hide tool call latency if the model needs to search for something.

kalabrium 56 minutes ago | parent

Hopefully smaller models soon that can be deployed locally for personal use, definitely don't want the speech data sent outside. But having the speech ability is going to improve usefulness of the already great gen AI models drastically!

sarkarghya 55 minutes ago | parent

I like that a lot of ASR models now offer native context biasing capabilities. If I have enough volume, I would probably just self-host my model and do my own biasing but good enough to roll out custom solutions.

mistic92 53 minutes ago | parent

My main issue with STT models are - WER - price - speed - language support

ilaksh 52 minutes ago | parent

How did they integrate Voxtral Large exactly? Does it output a special token to indicate it is asking for help, or is it running a command?

Anyway, if it is trained to decide when to think/ask for help that's good.

Too bad it's not open source though. But for large companies in Europe or elsewhere that want to control their data, this might be one of the best options now.

Although the frontier is true full duplex. Anyone know if there is anything that is open source for that other than MoshiRag or PersonaPlex?

Also did the CSM update some months back fix it? I never got a chance to test it.

arjie 48 minutes ago | parent

Interesting. I've found the Voxtral series pretty nice actually. Good transcription and diarization which is the primary personal use-case where I want to turn podcast content into text since I read faster than I listen. The older ones were pretty cheap too and had the nice property that they don't hallucinate text when there's no speech.

scosman 42 minutes ago | parent

Closed :(

MisterMunchkin 26 minutes ago | parent

The examples they give of how they “improved” the voices aren’t great. Version 1 sounded like actual people, while this version sounds like generic AI slop voice.

CodesInChaos 20 minutes ago | parent

Slightly off-topic, but can anybody recommend a local model/tool that fixes timestamps of existing srt subtitles? That should be easier than generating them from scratch.

When I last looked for this a few years ago, I only found a really primitive one that did a linear transformation of all timestamps in the video, and none to correct each line individually.

edit: Claude recommended alass. Any experiences with that one, or better options?

joshstrange 10 minutes ago | parent

I don't plan to release it as it part of an overall "media health" tool I built but you should be able to vibe this pretty easily. I have mine using Whisper to check episodes at intervals throughout to compare against this subtitles and then shift them as needed.

One thing to watch for, make sure it never tries to 2 subtitles at the same time or they will stack in a weird way. It's easy to avoid that but my first pass on a few episodes screwed up the subtitles.

Also, you almost certainly want to extract the subtitles for this work, or at least that's what I did. A combo of pulling out the subtitles in the video file and using OpenSubtitles (VIP) to clean up my media.

CodesInChaos 18 minutes ago | parent

Considering the progress on speech recognition, why do YouTube subtitles have little in common with what's spoken in the video? Shouldn't Google be able to do a good job at auch a task?

Jabrov 9 minutes ago | parent

Scale is a big part of it.

Every minute, 500 hours of video are uploaded to YouTube.

Even Google can't afford to run the biggest baddest model on every single one of those hours.

zuInnp 12 minutes ago | parent

Quite funny that the agent starts speaking with a French accent after reading French :D

wiether 6 minutes ago | parent

You mean it speaks French with an English accent?

envoked 9 minutes ago | parent

Excited to give this a try in my local Hermes setup. A poor conversation modality is the only thing that’s been bringing me back to the Claude app.