Board:   [DoveNet] Internet Discussion
Subject: AI LLM Artificial intelli
From:    Lonewolf
Date:    Sat Mar 14, 2026 05:37

Subject: AI LLM Artificial intelligence infrastructure
@VIA: BINARYDR
  Re: AI LLM Artificial intelligence infrastructure
  By:phigan to bbsing on Wed Mar 11 2026 05:13 pm

 >   Re: AI LLM Artificial intelligence infrastructure
 >  > What models are you liking, and what sizes are you getting good TPS?
 >  > What do you think is good TPS?
 >  > Have you tried Deepseek-v3?
 >  > vllm?
 >
 > I'm not liking any of them. I haven't tried qwen2.5 out yet, I just
 > downloaded it, but the rest of them aren't any good. At least for what I've
 > been trying, which is code.
 >
 > devstral:24b
 > codellama:34b
 > qwen3-coder:30b
 >
 > My GPU was second hand off Craigslist after scouring that and eBay quite a
 > bit.
 > Still haven't found a good front end to use, but I have not tried Open WebUI
 > just yet.
 >
How you tried llama.cpp? I've got Ollama, LM Studio and llama.cpp on my 24gb vram system. llama.cpp is definetly better in all areas and it has a nice web UI too.

Lonewolf
---
 ■ Synchronet ■ Fireside BBS - AI-WX - firesidebbs.com:23231
.