[HN Gopher] Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Sil...
       ___________________________________________________________________
        
       Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon
        
       About six months ago, I started working on a project to fine-tune
       Whisper locally on my M2 Ultra Mac Studio with a limited compute
       budget. I got into it. The problem I had at the time was I had
       15,000 hours of audio data in Google Cloud Storage, and there was
       no way I could fit all the audio onto my local machine, so I built
       a system to stream data from my GCS to my machine during training.
       Gemma 3n came out, so I added that. Kinda went nuts, tbh.  Then I
       put it on the shelf.  When Gemma 4 came out a few days ago, I
       dusted it off, cleaned it up, broke out the Gemma part from the
       Whisper fine-tuning and added support for Gemma 4.  I'm presenting
       it for you here today to play with, fork and improve upon.  One
       thing I have learned so far: It's very easy to OOM when you fine-
       tune on longer sequences! My local Mac Studio has 64GB RAM, so I
       run out of memory constantly.  Anywho, given how much interest
       there is in Gemma 4, and frankly, the fact that you can't really do
       audio fine-tuning with MLX, that's really the reason this exists
       (in addition to my personal interest). I would have preferred to
       use MLX and not have had to make this, but here we are. Welcome to
       my little side quest.  And so I made this. I hope you have as much
       fun using it as I had fun making it.  -Matt
        
       Author : MediaSquirrel
       Score  : 95 points
       Date   : 2026-04-07 19:37 UTC (3 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | dsabanin wrote:
       | Thanks for doing this. Looks interesting, I'm going to check it
       | out soon.
        
         | MediaSquirrel wrote:
         | you are welcome! It was a fun side quest
        
       | craze3 wrote:
       | Nice! I've been wanting to try local audio fine-tuning. Hopefully
       | it works with music vocals too
        
       | LuxBennu wrote:
       | I run whisper large-v3 on an m2 max 96gb and even with just
       | inference the memory gets tight on longer audio, can only imagine
       | what fine-tuning looks like. Does the 64gb vs 96gb make a
       | meaningful difference for gemma 4 fine-tuning or does it just
       | push the oom wall back a bit? Been wanting to try local fine-
       | tuning on apple silicon but the tooling gap has kept me on
       | inference only so far.
        
         | MediaSquirrel wrote:
         | Memory usage increases quadratically with sequence length.
         | Therefore, using shorter sequences during fine-tuning can
         | prevent memory explosions. On my 64GB RAM machine, I'm limited
         | to input sequences of about 2,000 tokens, considering my
         | average output for the fine-tuning task is around 1,000 tokens
         | (~3k tokens total).
        
           | LuxBennu wrote:
           | Ah that makes sense, quadratic scaling is brutal. So with
           | 96gb i'd probably get somewhere around 4-5k total sequence
           | length before hitting the wall, which is still pretty
           | limiting for anything multimodal. Do you do any gradient
           | checkpointing or is that not worth the speed tradeoff at
           | these sizes?
        
       | yousifa wrote:
       | This is super cool, will definitely try it out! Nice work
        
       | pivoshenko wrote:
       | nice!
        
       ___________________________________________________________________
       (page generated 2026-04-07 23:00 UTC)