[HN Gopher] StableLM Zephyr 3B
___________________________________________________________________
StableLM Zephyr 3B
Author : roborovskis
Score : 96 points
Date : 2023-12-07 18:10 UTC (4 hours ago)
(HTM) web link (stability.ai)
(TXT) w3m dump (stability.ai)
| josh-sematic wrote:
| Note that it uses a non-commercial license. Still pretty cool
| though!
| Reubend wrote:
| Yeah, I think this is a great release, but I also suspect that
| most people won't end up using it just because of the license.
| It's actually a lot more restrictive than what I would
| personally consider "commercial" usage:
|
| > Non-Commercial Uses does not include any production use of
| the Software Products or any Derivative Works.
|
| So even if you want to launch a free service using this, that's
| not allowed.
| m3kw9 wrote:
| How would one go about making a .llamafile for this?
| simonw wrote:
| Convert it to GGML and use a zip tool to add that to a
| llamafile package.
|
| https://huggingface.co/TheBloke?search_models=Zephyr doesn't
| have a GGML for it yet but I wouldn't be surprised to see one
| by the end of the day.
| pleks wrote:
| And it's been uploaded
| https://huggingface.co/TheBloke/stablelm-
| zephyr-3b-GGUF/tree...
|
| Getting 20toks/s on M1 mba where as LLaVa I ground to a halt.
| Very impressed
| adamkochanowicz wrote:
| Can't wait for someone smarter than me to make this compatible
| with MLC on iPhone.
| haltist wrote:
| You can do this by just following a tutorial:
| https://huggingface.co/docs/diffusers/main/en/using-
| diffuser.... ML/AI models are just function graphs and most of
| the frameworks support saving and loading safetensor serialized
| graphs.
| simonw wrote:
| "This model is being released under a non-commercial license that
| permits non-commercial use."
|
| I'm very interested in high quality 3B models, but it's hard to
| get excited about this given the increasing array of commercially
| usable models.
| brianjking wrote:
| Yeah, Replit is likely the best option out there for a 3B model
| size, right?
| filterfiber wrote:
| Refact has a decent 1.6B model that I think is better
|
| https://huggingface.co/smallcloudai/Refact-1_6B-fim
| anigbrowl wrote:
| They have more fully open stuff in the pipeline. IMHO it's good
| that they put out stuff for hobbyists to play around with so
| that they're not immediately overtaken by people ready to
| deploy things at commercial scale.
| emadm wrote:
| It will be included under our membership next week which starts
| at $1 a month after grant ($20 base)
| makin wrote:
| The parent comment was referring to free ($0 a month) models.
| With those, companies don't need to plan around the
| possibility that Stability AI hikes up the price afterwards.
| emadm wrote:
| It'll be static with cpi max, flat membership for all core
| models
|
| Just released video, sdxl turbo and 3d, code and more
| coming
|
| Very positive reaction so far and we will still do our
| grants for OSS and do OSS collaborations, done over 10m
| A100 hours over last year
|
| Launches next few days
| https://x.com/emostaque/status/1732197072290353455?s=46
| filterfiber wrote:
| After the Unity re-licensing fiasco, and OpenAI yanking old
| models,
|
| is there any protections in place to allow the use of
| StableLM-Zephyr-3B indefinitely or will you be able to just
| deny any continuing access to models?
| emadm wrote:
| We looked at that, will be self service for commercial with
| flat pricing including all base models, weights are all
| downloadable by anyone.
|
| Models are very interesting
| ilaksh wrote:
| Can you link to an explanation of how membership and
| licensing works for commercial use?
| simlevesque wrote:
| How fast are these small models on a 4090, is it like 100ms ?
| 500ms ?
| pulse7 wrote:
| Mistral-7B gives you 80 tokens/second on 4090. So this one will
| be faster...
| emadm wrote:
| It's about twice the speed
| stavros wrote:
| Am I reading it right that performance was roughly comparable
| with GPT-3.5? How is this even possible?
| filterfiber wrote:
| No it's not (according to their benchmarks).
|
| Zephyr-7B-B still beats it in most benchmarks but it's close.
|
| This model is almost Zephyr-7B-B performance at 3B size which
| is a lot better for inference requirements.
| emadm wrote:
| Yeah got a way to beat 3.5 but it beats most of the first
| generation llama tunes even guacano 65b
|
| Lots of improvements to go
| alsodumb wrote:
| By comparing on benchmarks that are either limited, or have
| data leaks, or in most cases just don't make sense in terms of
| usability - I've personally stopped looking at benchmarks to
| compare models. Personally, if I want to try a new model I hear
| a lot of chatter about, I use it for a few hours in my daily
| workflow. My baseline is GPT3.5 and GPT4, and I compare the
| models with them in terms of my day to day usage.
| kouteiheika wrote:
| So in your experience which open model is currently the best?
| Version467 wrote:
| Not really. They already chose to show the benchmark where it
| does best and even then it's still quite a bit worse (though
| definitely impressive for its size). If you take a look at
| other benchmarks, for example MMLU@5-shot then this does 46.3,
| while gpt-3.5 does 70.
|
| But there might be some use cases where this one is close
| enough in performance and the difference in cost and speed make
| it a better choice.
| filterfiber wrote:
| > Hardware: StableLM Zephyr 3B was trained on the Stability AI
| cluster across 8 nodes with 8 A100 80GBs GPUs for each nodes.
|
| I might be missing it but do they say the number of training
| tokens that was used to train this?
|
| This would help with efforts like TinyLlama in trying to figure
| out how well the scaling works with training tokens vs parameter
| size and challenging the chinchilla model.
| emadm wrote:
| We included full training details for the base model on 4
| trillion tokens including wandb etc
|
| https://stability.wandb.io/stability-llm/stable-lm/reports/S...
| nextworddev wrote:
| Stability is apparently up for sale, hence the recent steady
| stream of releases
| supermatt wrote:
| This space is so confusing when it comes to licenses.
|
| "Zephyr" is MIT, but "Stability Zephyr" is non commercial. They
| could have at least used a different name.
|
| "Inspired" in all but license it would seem
| mirekrusin wrote:
| How those licenses work for generated content? If it's non-
| commercial does it mean I can still use it for work to generate
| stuff? In other words - is it similar to ie. using GIMP, which is
| open source, but I can still use created content in commercial
| product without attribution?
___________________________________________________________________
(page generated 2023-12-07 23:01 UTC)