Post AYm69D6iHftmHUtzto by ahyatt@urbanists.social
 (DIR) More posts by ahyatt@urbanists.social
 (DIR) Post #AYhkNWPlcI6VS4HG7M by simon@fedi.simonwillison.net
       2023-08-14T00:24:04Z
       
       0 likes, 0 repeats
       
       New release of my llm-mlc plugin for https://llm.datasette.io/ - this plugin allows you to run large language models like Llama 2 using the MLC Python library, which means you get GPU acceleration on M1/M2 Macs! https://github.com/simonw/llm-mlc/releases/tag/0.4
       
 (DIR) Post #AYm69D6iHftmHUtzto by ahyatt@urbanists.social
       2023-08-16T02:46:53Z
       
       0 likes, 0 repeats
       
       @simon This is great, the 13b model went from taking about an hour to being almost instantaneous on my M1 macbook.  The 70b model crashed my machine though, lol.