[HN Gopher] Vicuna on iPhone
       ___________________________________________________________________
        
       Vicuna on iPhone
        
       Author : tosh
       Score  : 79 points
       Date   : 2023-04-30 10:49 UTC (12 hours ago)
        
 (HTM) web link (mlc.ai)
 (TXT) w3m dump (mlc.ai)
        
       | simonw wrote:
       | This is pretty fun. It's pretty competent for a model that runs
       | on a phone - I did get a great hallucination out of it when I
       | asked it about myself:
       | https://twitter.com/simonw/status/1652363265979318272
        
       | homarp wrote:
       | see https://news.ycombinator.com/item?id=35763483 also
        
       | aledalgrande wrote:
       | Encode 20.5 tok/s and decode 6.6 tok/s on my 14 Pro (non-Max)
       | 
       | Did pretty good on an okonomiyaki recipe
       | 
       | Edit: the coding capabilities are pretty hilarious. I asked about
       | partials in Turbo streams and it correctly answered, but then
       | when I asked about a code sample it gave me a PHP+MySQL query?
       | Who knows what happened there.
        
       | batrat wrote:
       | Are there any Android alternatives?
        
       | simooooo wrote:
       | What the heck is it?
        
         | esafak wrote:
         | A large language model; machine learning artifact.
        
         | rasengan wrote:
         | A specifically trained Llama
        
       | carterschonwald wrote:
       | Bummer it crashes on my iPhone 11pro.
        
         | rock_artist wrote:
         | It also crashes on my SE. This is most likely due to memory as
         | it suggests.
         | 
         | Update : Also other users with beefier hardware comment about
         | it.
        
       | Operyl wrote:
       | Hmm. I just get it to consistently crash on my 14 Pro Max, latest
       | stable iOS build. At least TestFlight will send those reports on
       | over.
        
         | isoprophlex wrote:
         | Me too. I thought it was my crap ass iphone 11, but if you're
         | also suffering on a 14 pro... maybe it's a bug, not a hw
         | constraint.
        
         | InvaderFizz wrote:
         | 13 Pro Max, no crashes but definitely went OOM and slowed to a
         | crawl. My phone also got very warm.
         | 
         | I wasn't able to get much useful output for the things I
         | normally use ChatGPT to help with. It insisted on giving
         | general steps instead of code for basic tasks I threw at it.
         | (Certificate generation using openssl or python).
        
         | skykooler wrote:
         | Same thing on 12 mini. I think 6gb of ram is a little low for
         | this.
        
       | isoprophlex wrote:
       | Fantastic effort, I'm on holidays but I'm going to burn through a
       | significant chunk of my mobile internet traffic allotment to test
       | this... I seriously thought it'd take at least until summer for
       | someone to make this happen (LLMs on mobile devices)
       | 
       | Edit: too bad, it crashes all the time without generating output.
        
       ___________________________________________________________________
       (page generated 2023-04-30 23:01 UTC)