[HN Gopher] Run LLaMA and Alpaca on your computer
___________________________________________________________________
Run LLaMA and Alpaca on your computer
Author : kristianpaul
Score : 136 points
Date : 2023-04-05 17:03 UTC (5 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| ConanRus wrote:
| [dead]
| nunobrito wrote:
| Nice, thanks for making this available.
| kevviiinn wrote:
| Has anyone been able to get this to download the 65B model? It
| keeps failing for Mr but 30B downloads just fine
| toastal wrote:
| Linked library has a DCMA takedown. Host this stuff outside of US
| corporate control.
| shaky-carrousel wrote:
| If the model weights are able to be copyrighted, then they are
| also able to be a derivative work of something copyrighted, as
| the model is based on copyrighted works. So either the weights
| are infringing copyright, which means they don't belong to
| Meta, or the weights can't be copyrighted.
|
| Either way, that DMCA is worthless.
| hbn wrote:
| I'm not sure the logic follows.
|
| Music can be copyrighted, and it can also be made from
| samples from other music that's copyrighted. But sampling
| still happens without infringing copyright.
|
| I'd say the small handful of bits flipped in a model from
| training on some text, or piece of code, or an image is even
| less copyrighted information brought over than a music sample
| or borrowed/referenced melody.
| demoride wrote:
| Your philosophical argument is interesting, but what the op
| was saying was one of the linked repos used in this repo is
| inaccessible due to DMCA: https://github.com/shawwn/llama-dl
|
| So while what you say may be true the DMCA seems to have
| worth for these orgs because they can get code removed by the
| host, who is uninterested in litigating, and the repo owner
| likely is even less capable of litigating the DMCA.
|
| Unfortunately as a tool of fear and legal gridlock DMCA has
| shown itself to be very useful to those with ill intent.
| nl wrote:
| I'd suggest all the internet lawyers on this thread read
| "Foundation models and Fair Use":
| https://arxiv.org/abs/2303.15715
|
| TL;DR: There's a good fair use argument but it isn't a given
| that fair use always applies.
| halfeatenscone wrote:
| I don't think this follows. Calling it a derivative work
| already feels like a stretch, but even granting that framing,
| the use is clearly transformative and therefore likely to be
| considered fair use in the US.
|
| I actually wrote a Wikipedia article on the intersection of
| copyright law and deep learning models the other day (https:/
| /en.wikipedia.org/wiki/Artificial_intelligence_and_co...). I
| was hoping to include a section on the copyrightability of
| model weights, but was sadly able to find 0 coverage in
| reliable sources.
| nl wrote:
| This should include https://arxiv.org/abs/2303.15715
| sliken wrote:
| Or just post a magnet link to a torrent.
| knodi123 wrote:
| Downloading alpaca weights actually does use a torrent now!
| But it seems a little.... leechy... to use a fire-and-forget
| CLI torrent client to grab a dozens of gigabytes file and
| then quit.
| Dwedit wrote:
| llama.cpp is simpler to get working than this is.
| mastazi wrote:
| How so?
|
| how is this: npx dalai llama 7B npx
| dalai serve
|
| harder than: # build this repo git
| clone https://github.com/ggerganov/llama.cpp cd
| llama.cpp make #For Windows and CMake, use
| the following command instead: cd
| <path_to_llama_folder> mkdir build cd build
| cmake .. cmake --build . --config Release
| # obtain the original LLaMA model weights and place them in
| ./models ls ./models 65B 30B 13B 7B
| tokenizer_checklist.chk tokenizer.model # install
| Python dependencies python3 -m pip install torch numpy
| sentencepiece # convert the 7B model to ggml FP16
| format python3 convert-pth-to-ggml.py models/7B/ 1
| # quantize the model to 4-bits (using method 2 = q4_0)
| ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-
| model-q4_0.bin 2 # run the inference
| ./main -m ./models/7B/ggml-model-q4_0.bin -n 128
| seydor wrote:
| I hope we will see a revival of native desktop development with
| AI. There is really no reason for bloated JS containers, we can
| ask AI to rewrite it in C++.
| dist-epoch wrote:
| There will be a single app in the future: a generative AI app
| which will synthesize pixels on the screen depending on what
| you are doing.
| knodi123 wrote:
| That will be absurdly wasteful of power. My calculator runs
| using the power of ambient light in a comfortably lit room.
| There are some things that an LLM is well-suited for, but
| c'mon. Unless electricity and hardware becomes free, I just
| don't see your prediction coming true.
| brucethemoose2 wrote:
| One might call C++ bloat. Why not C, or maybe even a stricter C
| subset more amenable to compiler optimization.
| suby wrote:
| I hope so too. Plus, in my experience with using GPT4 to
| generate code, it's more effective at writing C++ than it is at
| writing Python. Something about annotating the types helps it
| with getting things right.
| hbn wrote:
| > Something about annotating the types helps it with getting
| things right.
|
| Same rule holds true for human programmers!
| nickthegreek wrote:
| For some reason my install does not have the model selector at
| the top. Real shame, thats the feature I wanted!
| tmaly wrote:
| The windows install says: When installing Visual Studio, make
| sure to check the 3 options as highlighted below:
|
| Python development Node.js development Desktop development with
| C++
|
| What if you already have visual studio installed and you did not
| check all of these?
| muricula wrote:
| It's been a little while but iirc, you can rerun the visual
| studio installer and check those boxes and it will update your
| install.
| henry_viii wrote:
| Previous discussion:
|
| https://news.ycombinator.com/item?id=35127020
___________________________________________________________________
(page generated 2023-04-05 23:01 UTC)