PcChip 52 minutes ago | parent
I didn't see any benchmarks against vllm, sglang, exllama, etc
rancor 43 minutes ago | parent
Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.