[HN Gopher] Llama 2 Everywhere (L2E): Standalone, Binary Portabl...
___________________________________________________________________
Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable
Llama 2
Author : jjwiseman
Score : 274 points
Date : 2023-10-05 23:18 UTC (23 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| hospitalJail wrote:
| I didn't quite understand this. Why is llama offered as a .cpp
| file instead of a compiled binary?
|
| I mean, obviously there are benefits from having the code, but
| most of the time people will give you an executable.
|
| Why isnt this the same?
| hirako2000 wrote:
| Binaries are published in the release section, in the form of a
| bootable ISO.
|
| Instructions also are there too
| pmarreck wrote:
| Sweet, now someone do Mistral-7B-Everywhere and make it also take
| advantage of any available Nvidia GPU's or M1 Mac GPU's
| eole666 wrote:
| Cool project ! It would be nice if the README displayed the ram
| (and vram for GPUs) needed to run it on a device.
| AMICABoard wrote:
| Right now this is CPU only. GPU support coming soon. It needs
| only 512mb and any x86_64 CPU now.
| AMICABoard wrote:
| Creator here. Thank you for sharing!
| bcjordan wrote:
| Can one write installable apps for this OS that Llama can call
| out to?
|
| We need to go deeper!
| AMICABoard wrote:
| We will go the depth. In the coming versions.
| xcdzvyn wrote:
| What does cat /dev/llama
|
| do? Would I get a kind of LLM stream of consciousness? That's
| incredible :-)
| AMICABoard wrote:
| cat /dev/llama is not yet implemented. However we have a
| module to which you can give a prompt as parameter. It's
| buggy now.
|
| Our goal is to write a proper kernel module to implement
| three things:
|
| 1. a character device 2. 1st backend is a LLMZip ie you write
| to say /dev/l2ezip, you get a compressed stream out 3. 2nd
| backend is a LLM, ie you write a prompt to say /dev/llama2,
| you get a completion back
|
| So the 1st backend could be useful for compressed telemetry
| The second backend could be useful for IoT LLM, or our
| ambitious plan of responding to telemetry, ie take action,
| such as control motor speed etc.
| jamal-kumar wrote:
| Easily the first practical use I've seen after hearing about
| cosmo libc. Very cool!
| leonheld wrote:
| This is hilarious (in a good way), I love it. Thanks for
| creating it.
| AMICABoard wrote:
| :)
| ingenieroariel wrote:
| Pretty cool!
|
| > My research goal is to train models using various hardware
| telemetry data with the hope that the models learn to interpret
| sensor inputs and control actuators based on the insights they
| glean from the sensor inputs. This research direction may open
| up exciting possibilities in fields such as automation, space,
| robotics and IoT, where L2E can play a pivotal role in bridging
| the gap between AI and physical systems.
|
| This part was easy to miss but quite interesting, could you
| expand a bit here? What does L2E stand for?
| pmarreck wrote:
| > What does L2E stand for?
|
| um...
| AMICABoard wrote:
| Thank you. So L2E stands for Llama 2 Everywhere. Thanks again
| for checking it out.
| throwaway10965 wrote:
| It's the name of the project - Llama2Everywhere.
| ingenieroariel wrote:
| thanks! I am still curious about the sensor inputs part.
|
| I am trying to replace the 2.4ghz controller on my electric
| skateboard to make 0 to 5kmh and braking more pleasant and
| maybe use gyroscopes to do away with the controller
| altogether. What would tokens be in that case? Do you
| create a CAN style representation and feed that to the llm?
| What kind of throughput do you foresee being possible on
| which hardware?
| ilc wrote:
| Basic control theory will work better than AI here. The
| mathematical models used in control theory have been used
| in computing since at least the 50's (Kalman Filters). I
| suspect you won't have issues with computational power.
|
| Figuring out exactly which model to use and how, may take
| some work. Also understanding control theory will allow
| you to do things like traction control, etc.
| AMICABoard wrote:
| In addition. I Agree.
| AMICABoard wrote:
| I am still thinking how I'll pull this off. But basically
| it is collect tons of Telemetry converted to in
| ascii/text.
|
| Telemetry in sense sensor streams, both command and
| responses.
|
| Then we'll just train a small model for long enough. Then
| we will see how it would respond.
|
| That's the plan sort of.
| datadrivenangel wrote:
| In this case, I would ask the LLM to suggest an algorithm
| to minimize acceleration, jerk and snap based on the
| expected sensor input data, and then just implement that.
| Probably in memory on whatever runs the board.
|
| Straightforward control problem of bringing the board
| from 5-0kmh smoothly?
| AMICABoard wrote:
| Let's see.
| AMICABoard wrote:
| Correct :)
| LouisvilleGeek wrote:
| Is this somehow based on Temple OS?
| AMICABoard wrote:
| No but each version of L2E OS has a name. v0.1 is the first
| version and we wanted to pay our tribute to Terry A Davis, so
| called it TempleDOS and added a bit of TempleOS references.
| LouisvilleGeek wrote:
| Got it! Like how you snuck the Amiga floppy disk in as well!
| AMICABoard wrote:
| :) I love my Amiga 500 and I miss her :)
| LouisvilleGeek wrote:
| Me too! Guru meditations and all.
| AMICABoard wrote:
| FAKIR HAS NAILS ON BOOT SCREEN :)
| jamal-kumar wrote:
| man those kids are gonna look that up and go down a rabbit
| hole aren't they
| m3kw9 wrote:
| No specified of memory or processing requirements? How do I know
| if it will work on my machine?
| AMICABoard wrote:
| Sorry there. It's 512MB RAM & x86_64 CPU. This just an alpha
| version so not very usable :)
| tibbydudeza wrote:
| https://en.opensuse.org/AMD_OpenCL
|
| I gave up - so confusing.
| AMICABoard wrote:
| This OS doesn't use opencl yet. The non OS program version can
| take advantage of opencl.
| lawlessone wrote:
| very weird, i like it.
| AMICABoard wrote:
| Thank you :)
| skybrian wrote:
| This could be a lot clearer about which install methods are
| offered. It seems to be an OS that you have to boot into a VM?
| What are the system requirements for the container?
| AMICABoard wrote:
| You are right. Clearer instructions will be added.
|
| This v0.1 and consider it alpha. A lot of stuff is broken.
|
| The current build just needs 512MB RAM and an x86_64 CPU.
|
| The ISO can be downloaded from the releases here:https://github
| .com/trholding/llama2.c/releases/tag/L2E_OS_v0...
|
| Currently it doesn't do much useful stuff except for stories.
|
| You could etch the ISO to pendrive and boot on real system or
| you could run it in qemu/VM too:
|
| qemu-system-x86_64 -display gtk,zoom-to-fit=off -m 512 -accel
| kvm -vga virtio -cdrom l2eos.iso
|
| There is a known issue that for some it does not work on
| Virtual Box. All that will be fixed in future versions.
|
| There are also easter eggs and a hidden game of DOOM (Freedom)
|
| Pics:
| https://twitter.com/VulcanIgnis/status/1708851772435968017
|
| https://www.reddit.com/r/LocalLLaMA/comments/16zklam/i_creat...
|
| User posted Video of Doom Play:
| https://www.reddit.com/user/multiverse_fan/comments/170orkz/...
| A4ET8a8uTh0 wrote:
| It is a great start for alpha. While I agree with poster
| above you, I absolutely do not want you to get discouraged.
| This is awesome and can only get better from here.
| AMICABoard wrote:
| I agree. This week, I'll take time to polish it a bit. And
| better docs for sure.
| explosion-s wrote:
| Are there pre-compiled binaries or do we have to build them
| ourselves?
| AMICABoard wrote:
| In the releases you find the ISO which is pre compiles.
|
| If you want to build it yourself, you can do make l2e_os_iso if
| you clone the repo and you are on a linux machine.
| PoignardAzur wrote:
| > _How do we make sure that the output is factual and not
| hallucinated?_
|
| One method the readme doesn't mention: ask the same question
| multiple times. Apparently research suggests that when LLMs
| hallucinate answers, their hallucination is likely to be a
| different one every time, where as factual answers will tend to
| be consistent.
|
| https://arxiv.org/abs/2305.18248
| Davidzheng wrote:
| Purely anecdotally this is not a very successful method in my
| usage
|
| To reproduce: I've had it be consistent on hallucinations on
| the stable homotopy groups of spheres; on the quote "who speaks
| of victory to endure is all"; and many other fact based
| questions. Model: gpt4
| [deleted]
| [deleted]
| AMICABoard wrote:
| Asking repeatedly would be impractical. I have some vague idea
| in my mind called the fact engine. It's like a wiki where
| people could key in facts. Also facts based on "timeframes", ie
| 1800's 1900's etc (eg Moon Landing). The fact engine feeds a
| model that checks for hallucinations, and also rule based
| engines. Then a final score would be assigned based on
| consensus. Something like that.
___________________________________________________________________
(page generated 2023-10-06 23:01 UTC)