53 points antipaul 2 hours ago 17 comments
tony_cannistra 44 minutes ago | parent
In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.bensyverson 34 minutes ago | parent
Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter.
hirako2000 18 minutes ago | parent
janderson215 44 minutes ago | parent
browningstreet 23 minutes ago | parent
enos_feedler 18 minutes ago | parent
Someone 8 minutes ago | parent
Someone 9 minutes ago | parent
The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.
It’s more like an advertorial for their tech.
peri-cl 10 minutes ago | parent
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
butterlesstoast 8 minutes ago | parent
isodev 7 minutes ago | parent