[HN Gopher] Trinity large: An open 400B sparse MoE model
___________________________________________________________________
Trinity large: An open 400B sparse MoE model
Author : linolevan
Score : 50 points
Date : 2026-01-28 00:57 UTC (22 hours ago)
(HTM) web link (www.arcee.ai)
(TXT) w3m dump (www.arcee.ai)
| linolevan wrote:
| I'm particularly excited to see a "true base" model to do
| research off of (https://huggingface.co/arcee-ai/Trinity-Large-
| TrueBase).
| mynti wrote:
| They trained it in 33 days for ~20m (that includes apparently not
| only the infrastructure but also the salaries over a 6 month
| period). And the model is coming close to QWEN and Deepseek.
| Pretty impressive
| zamadatix wrote:
| The price/scaling of training another same class model always
| seems to be dropping through the floor but training models
| which score much better seems to be hitting a brick wall.
|
| E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs
| 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50%
| is equal chance of winning) over 1.5 years. Meanwhile, even the
| open weights stuff OpenAI gave away last summer scores between
| the two.
|
| The exception seems to be net new benchmarks/benchmark
| versions. These start out low and then either quickly get
| saturated or hit a similar wall after a while.
| observationist wrote:
| This is a wonderful release.
| frogperson wrote:
| What exactly does "open" mean in this case? Is it weights and
| data or just weights?
| someotherperson wrote:
| It's always open weights.
| jetpackjoe wrote:
| It's never open data
| jacquesm wrote:
| Well, it is, it's your data to begin with after all but
| admitting that would create some problems.
| mwcampbell wrote:
| Given that it's a 400B-parameter model, but it's a sparse MoE
| model with 13B active parameters per token, would it run well on
| an NVIDIA DGX Spark with 128 GB of unified RAM, or do you
| practically need to hold the full model in RAM even with sparse
| MoE?
| timschmidt wrote:
| Even with MoE, holding the model in RAM while individual
| experts are evaluated in VRAM is a bit of a compromise. Experts
| can be swapped in and out of VRAM for each token. So RAM <->
| VRAM bandwidth becomes important. With a model larger than RAM,
| that bandwidth bottleneck gets pushed to the SSD interface. At
| least it's read-only, and not read-write, but even the fastest
| of SSDs will be significantly slower than RAM.
|
| That said, there are folks out there doing it.
| https://github.com/lyogavin/airllm is one example.
| antirez wrote:
| Can run with mmap() but it is slower. 4-bit quantized there is
| a decent ratio between the model size and the RAM, with a fast
| SSD one could try to see how it works. However when a model is
| 4-bit quantized there is often the doubt that it is not better
| than an 8-bit quantized model of 200B parameters, it depends on
| the model, on the use case, ... Unfortunately the street for
| local inference of SOTA model is being stopped by the RAM
| prices and the GPU request of the companies, leaving us with
| little. Probably today the best bet is to buy Mac Studio
| systems and then run distributed inference (MLX supports this
| for instance), or a 512 GB Mac Studio M4 that costs, like 13k$.
| greggh wrote:
| The only thing I question is the use of Maverick in their
| comparison charts. That's like comparing a pile of rocks to an
| LLM.
___________________________________________________________________
(page generated 2026-01-28 23:00 UTC)