[HN Gopher] Show HN: Timber - Ollama for classical ML models, 33...
___________________________________________________________________
Show HN: Timber - Ollama for classical ML models, 336x faster than
Python
Author : kossisoroyce
Score : 180 points
Date : 2026-03-02 00:57 UTC (22 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jnstrdm05 wrote:
| I have been waiting for this! Nice
| kossisoroyce wrote:
| Glad you got it just in time!
| Dansvidania wrote:
| Can't check it out yet, but the concept alone sounds great. Thank
| you for sharing.
| kossisoroyce wrote:
| You're welcome!
| mehdibl wrote:
| Ollama is quite a bad example here. Despite popular, it's a
| simple wrapper and more and more pushed by the app it wraps
| llama.cpp.
|
| Don't understand here the parallel.
| ekianjo wrote:
| I guess the parallel is "Ollama serve" which provides you with
| a direct REST API to interact with a LLM.
| sieve wrote:
| llama-cpp provides an API server as well via llama-server
| (and a competent webgui too).
| eleventyseven wrote:
| This is the vLLM of classic ML, not Ollama.
| kossisoroyce wrote:
| TBVH I didn't think about naming it too much. I defaulted to
| Ollama because of the perceive simplicity and I wanted that
| same perceived simplicity to help adoption.
| tl2do wrote:
| Since generative AI exploded, it's all anyone talks about. But
| traditional ML still covers a vast space in real-world production
| systems. I don't need this tool right now, but glad to see work
| in this area.
| visarga wrote:
| A nice way to use traditional ML models today is to do feature
| extraction with a LLM and classification on top with trad ML
| model. Why? because this way you can tune your own decision
| boundary, and piggy back on features from a generic LLM to
| power the classifier.
|
| For example CV triage, you use a LLM with a rubric to extract
| features, choosing the features you are going to rely on does a
| lot of work here. Then collect a few hundred examples, label
| them (accept/reject) and train your trad ML model on top, it
| will not have the LLM biases.
|
| You can probably use any LLM for feature preparation, and
| retrain the small model in seconds as new data is added. A
| coding agent can write its own small-model-as-a-tool on the fly
| and use it in the same session.
| benrutter wrote:
| What do you mean by "feature extraction with an LLM?". I can
| get this for text based data, but would you do that on
| numeric data? Seems like there are better tools you could use
| for auto-ML in that sphere?
|
| Unless by LLM feature extraction you mean something like
| "have claude code write some preprocessing pipeline"?
| mirsadm wrote:
| Isn't the whole point for it to learn what features to
| extract?
| brokensegue wrote:
| "classical ML" models typically have a more narrow range of
| applicability. in my mind the value of ollama is that you can
| easily download and swap-out different models with the same API.
| many of the models will be roughly interchangeable with tradeoffs
| you can compute.
|
| if you're working on a fraud problem an open-source fraud model
| will probably be useless (if it even could exist). and if you own
| the entire training to inference pipeline i'm not sure what this
| offers? i guess you can easily swap the backends? maybe for
| ensembling?
| eleventyseven wrote:
| > if you own the entire training to inference pipeline i'm not
| sure what this offers
|
| 336x faster than Python, and swapping backends in a production
| environment can be far from trivial
| rudhdb773b wrote:
| If the focus is performance, why use a separate process and have
| to deal with data serialization overhead?
|
| Why not a typical shared library that can be loaded in python, R,
| Julia, etc., and run on large data sets without even a memory
| copy?
| sriram_malhar wrote:
| Perhaps because the performance is good enough and this
| approach is much simpler and portable than shared libraries
| across platforms.
| kossisoroyce wrote:
| Exactly. The objective is to abstract away completely. Shared
| libraries just add too much overhead.
| bob001 wrote:
| This lets you not even need Python, r, Julia, etc but directly
| connect to your backend systems that are presumably in a fast
| language. If Python is in your call stack then you already
| don't care about absolute performance.
| kossisoroyce wrote:
| I owe you a beer!
| o10449366 wrote:
| Can you tell us more about the motivation for this project? I'm
| very curious if it was driven by a specific use case.
|
| I know there are specialized trading firms that have implemented
| projects like this, but most industry workflows I know of still
| involve data pipelines with scientists doing intermediate data
| transformations before they feed them into these models. Even the
| c-backed libraries like numpy/pandas still explicitly depend on
| the cpython API and can't be compiled away, and this data feed
| step tends to be the bottleneck in my experience.
|
| That isn't to say this isn't a worthy project - I've explored
| similar initiatives myself - but my conclusion was that unless
| your data source is pre-configured to feed directly into your
| specific model without any intermediate transformation steps,
| optimizing the inference time has marginal benefit in the overall
| pipeline. I lament this as an engineer that loves making things
| go fast but has to work with scientists that love the convenience
| of jupyter notebooks and the APIs of numpy/pandas.
| kossisoroyce wrote:
| The motivation was edge and latency-critical use cases on a
| product I consulted on. Feature vectors arrived pre-formed and
| a Python runtime in the hot path wass a non-starter. You're
| right that for most pipelines the transformation step is the
| bottleneck, not inference, and Timber doesn't solve that
| (though the Pipeline Fusion pass compiles sklearn scalers away
| entirely if your preprocessing is that simple). Timber is
| explicitly a tool for deployments where you've already solved
| the data plumbing and the model call itself is what's left to
| optimize.
| palashkulsh wrote:
| Nice idea, i needed something like it
| OutOfHere wrote:
| It would be safer to use a Zig or Rust or Nim target. C risks
| memory-unsafe behavior. The risk profile is even bigger for vibe-
| coded implementations.
| kossisoroyce wrote:
| Fair point in general, but the risk profile here is actually
| quite low. The generated C is purely computational, with no
| heap allocation, no pointer arithmetic, no user-controlled
| memory, no IO. It's essentially a deeply nested if/else tree
| over a fixed-size float array. The "unsafe" surface in C is
| largely a non-issue when the code is statically shaped at
| compile time from a deterministic compiler pass.
|
| Rust/Zig/Nim would add toolchain complexity with minimal safety
| gain for this specific output shape. Those were my
| considerations.
| OutOfHere wrote:
| > Rust/Zig/Nim would add toolchain complexity
|
| Fair response in general, but Zig is well known to lower
| toolchain complexity, not add it.
| theanonymousone wrote:
| Wouldn't it be much more useful if the request received raw input
| (i.e. before feature extraction), and not the feature vector?
| marcyb5st wrote:
| You can do that with Onnx. You can graft the preprocessing
| layers to the actual model [1] and then serve that. Honestly, I
| already thought that ONNX (CPU at least) was already low level
| code and already very optimized.
|
| @Author - if you see this is it possible to add comparisons (ie
| "vanilla" inference latencies vs timber)?
|
| [1] https://gist.github.com/msteiner-
| google/5f03534b0df58d32abcc... <-- A gist I put together in the
| past that goes from PyTorch to ONNX and grafts the
| preprocessing layers to the model, so you can pass the raw
| input.
| kossisoroyce wrote:
| I'll check this out as soon as I am at my desk.
| deepsquirrelnet wrote:
| Does this use something like xnnpack under the hood?
___________________________________________________________________
(page generated 2026-03-02 23:02 UTC)