Board:   [DoveNet] Internet Discussion
Subject: AI LLM Artificial intelli
From:    phigan
Date:    Wed Mar 11, 2026 18:13

Subject: AI LLM Artificial intelligence infrastructure
@VIA: TACOPRON
  Re: AI LLM Artificial intelligence infrastructure
  By:bbsing to phigan on Tue Mar 10 2026 10:31 pm

 > What models are you liking, and what sizes are you getting good TPS?
 > What do you think is good TPS?
 > Have you tried Deepseek-v3?
 > vllm?

I'm not liking any of them. I haven't tried qwen2.5 out yet, Ijust downloaded
it, but the rest of them aren't any good. At least for what I've been trying,
which is code.

devstral:24b
codellama:34b
qwen3-coder:30b

I had the most success using 'Zed' as a front end to the qwen3-coder model.
Codellama actually refused to write code at all. Devstral says it's going to
write code but never does and gets stuck in a loop. I also tried installing
claude and launching that with those same models. Codellama wouldn't open at
all, saying it didn't support tools or whatever. Devstral again got stuck in a
loop telling me it was going to do things but not actually doing them.
qwen3-coder kept going back and forth on fixing one thing, breaking another,
then breaking the first thing when it fixed the second thing, etc.

My GPU was second hand off Craigslist after scouring that and eBay quite a bit.

Still haven't found a good front end to use, but I have nottried Open WebUI
just yet.

---
 ■ Synchronet ■ TIRED of waiting 2 hours for a taco? GO TO TACOPRONTO.bbs.io
.