[HN Gopher] RedPajama 7B (an Apache 2.0-licensed LLaMa) is now a...
___________________________________________________________________
RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available
Author : bestcoder69
Score : 66 points
Date : 2023-06-06 17:17 UTC (5 hours ago)
(HTM) web link (www.together.xyz)
(TXT) w3m dump (www.together.xyz)
| kichik wrote:
| Someone is into children's books.
|
| https://a.co/d/6nR2YZR
| refulgentis wrote:
| Omg. Great catch!
| mdaniel wrote:
| please don't use link shorteners, they're 404s waiting to
| happen: https://www.amazon.com/dp/0451474570 ("Llama Llama Red
| Pajama Board book - Illustrated, May 5, 2015"; 9780451474575)
| thewataccount wrote:
| This is great news!
|
| I don't mean to be negative but - has anyone at all found a use
| for any of the 7B models? And are you only using it because
| you're resource constrained? Or is it a purely academic thing for
| the moment?
|
| This model is all over the place in the benchmarks, it looks
| roughly comparable to llama, and I know it's open to use unlike
| llama. But I haven't seen a model under 30B consistent enough to
| actually use for anything myself.
|
| Hopefully the improved dataset will improve the next version, I
| truly want small models to work well, I'm sure it time it will
| improve.
|
| EDIT: I mean this isn't even a trick question: "There are two
| shapes each a different color, one shape is red and one shape is
| blue. Shape A is red. Shape B must be what color?"
|
| > This logic puzzle is based on a classic riddle that has been
| passed down for generations. The answer, of course, is that Shape
| B must be a red shape, as red is the only color that can be used
| to represent all reds.
|
| > The key to solving the puzzle is to recognize that red can be
| used to describe color as well as blood, which is another word
| for shape.
|
| What ????
|
| EDIT 2: GPT3.5-turbo says "there's no definitive color for shape
| B"
|
| Only GPT4 gets this right, "Since shape A is red, and the other
| shape is blue, shape B must be blue."
|
| I have no idea how they struggle on this.
| redox99 wrote:
| WizardLM-30B-Uncensored (4bit)
|
| > There are two shapes each a different color, one shape is red
| and one shape is blue. Shape A is red. Shape B must be what
| color?
|
| > Shape B can only be the other available color which is blue
| since we know that shape A is red.
|
| (I agree with you that <30B models are very weak)
| behohippy wrote:
| Vicuna-13b (4bit) got the answer right, the first time as
| well.
| senko wrote:
| GPT 3.5-turbo (via API) gets confused with this as well untill
| I tell it: "There are only two shapes."
| bestcoder69 wrote:
| Definitely don't quantize to 4-bit at this param count, if
| that's what you're doing. The redpajama.cpp repo is a bit crazy
| for making that the default.
|
| I'd say at this size don't count on accuracy. Think: fancy
| auto-complete, simple NLP task-doer, or a base for fine-tuning
| a task specific model. IME RedPJ-3B was too small to do
| anything at all other than ramble. The novel contribution of
| this model here being you can use this at work, and it has a
| GGML implementation unlike Falcon and MPT (last I checked) -
| meaning CPU-based inference on the cloud VM you use at work,
| maybe.
| redox99 wrote:
| People that run 7B 4bit is probably because they don't have
| the memory for more. And there are tests showing that
| quantizing a bigger model is always better than running a
| smaller model full precision (i.e LlaMa 13B 4bit > 7B fp16)
| thrtythreeforty wrote:
| This is some Vizzini ( _The Princess Bride_ ) grade logic
|
| https://www.youtube.com/watch?v=rMz7JBRbmNo
| philovivero wrote:
| Some of the most hilarity I've ever seen was asking various 7B
| models: "How old is a child who was born 9 years ago?"
|
| Paragraphs and paragraphs of complex calculations, ending up
| with "9.5" or "11" or some other close-but-very-wrong answer.
| jstarfish wrote:
| > I don't mean to be negative but - has anyone at all found a
| use for any of the 7B models?
|
| Productive/generative and logic-based tasks improve with the
| size of the corpus, but smaller models are useful for reductive
| tasks where florid prose and accuracy are not a requirement--
| i.e. summarize this or classify that. I use dolly-3B for quick-
| and-dirty work like this since I can run it on spare hardware.
|
| Commercially, you could probably get away with hot-swapping
| large models for small ones during periods of high demand-- for
| a chat service, kick every 3-4th exchange to a smaller, cheaper
| model to reduce load on the larger one. Nobody is perfectly
| coherent _all_ the time...
| thewataccount wrote:
| Huh interesting it does appear pretty good at summarization
| when I tried it. Are they consistent enough to not modify the
| information or mix up numbers in your experience?
|
| That's my main concern is it making "changes" to what it's
| summarizing for that usecase.
| redox99 wrote:
| That's kind of expected since GPT2 was already relatively
| decent at summarizing.
| ShannonLimiter wrote:
| GPT 3.5 Turbo gets the right answer for me every time.
| redox99 wrote:
| First try on GPT 3.5 (through the chat web) I get
|
| > Based on the information provided, Shape A is red and Shape
| B is the other shape. However, it is not explicitly stated
| what color Shape B is. Therefore, without further
| information, we cannot determine the color of Shape B.
|
| I retried a second time and it answered the same
|
| I retried a third time and it answered correctly
| thewataccount wrote:
| GPT3.5 turbo almost got it for me, sort of did but
|
| "If Shape A is red and Shape B is a different color, one
| possible color for Shape B could be blue. However, it's
| important to note that the color of Shape B could be any
| color other than red. Without further information or
| constraints, there is no definitive answer for the color of
| Shape B."
|
| Not that it matters a ton here, I'm just surprised that
| 3.5turbo had issues with it at all.
| thrtythreeforty wrote:
| The given example is quite funny: Write an email
| to my friends inviting them to come to my home on Friday for a
| dinner party, bring their own food to share. Dear
| friends, (snip) I have also organized a variety of
| fun activities to keep us entertained such as: * *Karaoke*
| *Movie night *Board games *Outdoor games *Book
| club *Painting together *Relaxing by the fireplace
| *Crafts *Tea party *Strip poker *Dance party
|
| Wow, RedPajama, one of these things is not like the others...
|
| (Edit, it's also a good example that this model is less censored
| than recent GPTs. It doesn't know that this isn't appropriate in
| this context, but it's not afraid to add it.)
| throwaway106382 wrote:
| Only if the dance party explicitly happens _after_ strip poker.
| yieldcrv wrote:
| That's a good invite prompt for friends, I like to see my
| attractive friends naked
| faitswulff wrote:
| To be fair, the prompt only required bringing food, not clothes
| 0cf8612b2e1e wrote:
| Going to be a pretty quick game if everyone arrives
| unclothed.
| [deleted]
___________________________________________________________________
(page generated 2023-06-06 23:02 UTC)