[HN Gopher] Mistral CEO confirms 'leak' of new open source AI mo...
___________________________________________________________________
Mistral CEO confirms 'leak' of new open source AI model nearing
GPT4 performance
Author : pg_1234
Score : 297 points
Date : 2024-01-31 19:32 UTC (3 hours ago)
(HTM) web link (venturebeat.com)
(TXT) w3m dump (venturebeat.com)
| MallocVoidstar wrote:
| Check out the PR he submitted:
| https://huggingface.co/miqudev/miqu-1-70b/discussions/10/fil...
| rrr_oh_man wrote:
| Cynical me smells a PR move
| seydor wrote:
| All their PR moves seem to be like this one. Not complaining
| rubymamis wrote:
| Tweet by Mistral CEO:
| https://x.com/arthurmensch/status/1752737462663684344
|
| > An over-enthusiastic employee of one of our early access
| customers leaked a quantised (and watermarked) version of an old
| model we trained and distributed quite openly.
|
| > To quickly start working with a few selected customers, we
| retrained this model from Llama 2 the minute we got access to our
| entire cluster -- the pretraining finished on the day of Mistral
| 7B release.
|
| > We've made good progress since -- stay tuned!
| Timon3 wrote:
| I love their approach! Mistral seems to be what I long ago
| hoped OpenAI to be.
| unshavedyak wrote:
| Makes me wonder if i should be giving my money to them
| instead of OpenAI. Do they have a Chat Interface like
| ChatGPT? Without first signing up, it's looking like they
| only have endpoints?
| yieldcrv wrote:
| LM Studio
|
| I use dolphin mixstral 8x7B way more than ChatGPT4 for the
| past several months
|
| if any of this sentence makes sense, I have a M1 with 64gb
| RAM and I use 5 bit quantizing with metal with 10,000 token
| context window. Its around 21 tokens/sec which is a little
| faster than the text speed that ChatGPT4 responds with
| josephg wrote:
| What is quality like compared to GPT4?
| yieldcrv wrote:
| for the kinds of discussions I have it is very close
|
| I often have discussions about political topics that I
| don't have a complete history on, things that are hard to
| get a non-emotional non-accusative response from in a
| forum or in person, licensing questions about obscure
| professions, liability questions, roleplays without the
| preachiness about the topic, coding, brand ideas and
| naming
|
| Lots of stuff that I dont want in any cloud or sent
| online, but then it just became second nature to
| primarily use it.
|
| the hallucinations are heavier. like it will give you
| specific links that it made up, and never apologize like
| chatgpt will, it will say "no, thats a real link" so,
| much more of a bullshitter
|
| LM Studio makes it easy to change the temperament though
| with a variety of premade system prompts, or allowing
| custom ones very easily
|
| I mainly use ChatGPT4 for multimodal like audio
| conversations, figuring a DIY problem out by sending it a
| photo of what I'm looking at, having it render photos on
| how to use something - I was at a gym and had it look at
| every piece of equipment there and tell me what it was
| and show me how a human would use it
|
| (I notice this is at odds with another comment, I havent
| used ChatGPT3.5 in nearly a year, but my experience with
| ChatGPT4 on the aforementioned topics is similar to my
| output in mixtral)
| moralestapia wrote:
| I can answer that from experience (just built a couple
| Q&A style sites), although a bit subjectively.
|
| Compared with the GPTs, Mixtral (and derivatives)
| perform:
|
| * 5% of the time as good as GPT4
|
| * 75% of the time on par with GPT3.5
|
| * 20% of the time a bit worse than GPT3.5 (too chatty and
| hallucinates with ease, although one could argue a better
| prompt could improve things a lot)
|
| Advantages of Mixtral, for me:
|
| * Cost 10-100x cheaper
|
| * Faster completion time
|
| * More deterministic output, in the sense that if you run
| the same query several times you get the same answers
| back (but they could be wrong). GPT almost always gives
| me an answer of great quality BUT with a lot of variance
| across them; even with temperature set to zero. This is a
| PITA when you want some sort of predictable, structured
| output like a JSON object.
| sjwhevvvvvsj wrote:
| Another factor is they monkey around with GPT constantly.
| The GPT you get on Monday may be totally weird on Friday.
| Whereas for a local model it doesn't change unless YOU
| change it.
| terhechte wrote:
| They have an api and it's quite cheap. Their model mistral
| medium is also very powerful. I use it instead of GPT 4
| regularly
| suslik wrote:
| In my experience it is comparable with chatGPT 3.5 but is
| more expensive (I still use it cause I hate guardrails).
| thierrydamiba wrote:
| Does mistral not have any guardrails?
| suslik wrote:
| Mistral API has a flag in the json payload to remove
| guardrails.
| sjwhevvvvvsj wrote:
| Nope! It's actually for adults to make their own choices
| and not some SV public relations firm nerfing it.
| sjwhevvvvvsj wrote:
| You can buy a good GPU and quickly come below cost for
| GPT APIs depending on your workload. I'm doing millions
| of tasks, so eating $8k on a custom build for ML up front
| is a long term cost savings. Not least of all that every
| other month there's an even better model.
| lordswork wrote:
| Does watermarked imply they can figure out which customer
| leaked it?
| danpot wrote:
| How do they even watermark a LLM?
| pjerem wrote:
| Make it learn that woziboza is a green cloud shaped like a
| washing machine ?
| declaredapple wrote:
| You can either modify the model weights in a way that
| doesn't cause any real differences (change a few bits
| somewhere should be enough), or you could watermark the
| actual text output.
|
| Here's a list of research for watermarking LLMs.
|
| https://github.com/hzy312/Awesome-LLM-
| Watermark?tab=readme-o...
| einsum wrote:
| Scott Aaronson did some work on statistical watermarking
| with OpenAI:
| https://www.scottaaronson.com/talks/watermark.ppt
| sangnoir wrote:
| Finetune your standard model on an small, unique set of
| made-up shibboleths (one per customer) before distributing?
| Ask the candidate model where the Fribolouth caves are
| located.
| haolez wrote:
| It's either that or they can narrow it down to a few people.
| EasyMark wrote:
| Seems like they have inferior secops on their end. I tried to
| give it a go and got "an error occurred" in both brave and
| firefox. Some message about a cookie wasn't found even though I
| was logging in with my google id which I use flawlessly with a
| bunch of other services. I have too many other fires to put out
| so I'll pass on this one I guess.
| janalsncm wrote:
| > Quantization in ML refers to a technique used to make it
| possible to run certain AI models on less powerful computers and
| chips by replacing specific long numeric sequences in a model's
| architecture with shorter ones.
|
| It's always amusing when the press tries to explain technical
| concepts. Quantization just means substituting high precision
| numeric types with lower precision ones. And not specific
| "numeric sequences", all numbers.
| SilverBirch wrote:
| It's funny how the original phrase sounds like it was generated
| by chatgpt. I'd guess the reason they didn't phrase it how you
| suggest is they probably thought that people who don't know
| what quantization means aren't going to be happy if you throw
| precision numerics at them. I don't knwo why they didn't just
| say "some of the numbers are rounded" though.
| bsaul wrote:
| GPT-4 has been out for almost a year now, and it seems that the
| frantic pace of open-ai releasing new groundbreaking tech every
| month has come to a halt. Anyone knows what's happening with open
| AI ? has the recent turmoil with sama caused lag in the company ?
| or are they working on some superweapon ?
| artninja1988 wrote:
| Probably getting bogged down by endless safety testing and
| paperwork by now
| harmmonica wrote:
| Maybe tangential to this comment, but I had been using 3.5 to
| write contracts, but a couple of weeks back, even when using
| the exact same prompts that would've worked in the past, 3.5
| started saying, to paraphrase, "for any contracts you need to
| get a lawyer."
|
| Anyone else had this experience? It seems like they're
| actually locking down some very helpful use cases, which
| maybe falls into the "safety" category or, more cynically, in
| the "we don't want to be sued" category.
| transcriptase wrote:
| As they tack on more and more guardrails it becomes lazier
| and less helpful. One way it gets around admitting that it
| has instructed not to help you is by directing you to
| consult a human expert.
|
| I also suspect there's some dynamic laziness parameter
| that's used to counteract increased load on their servers.
| You can ask the same prompt over and over in a new chat
| throughout the day and suddenly instead of writing the code
| you asked for or completing a task, it will do a small part
| of the work with "add the code to do xyz here" or explain
| the steps required to complete a task instead of completing
| it. It happens with v4 as well.
| TeMPOraL wrote:
| > _some dynamic laziness parameter that's used to
| counteract increased load_
|
| That's a brilliant risk mitigation mechanism! The AI
| won't recursively self-improve to superhuman levels if it
| just keeps getting tired of thinking.
| Baeocystin wrote:
| Add 'reducing compute use as much as possible' to the list,
| I'm sure. I can look at some of my saved conversations from
| months ago and feel wistful for what I used to be able to
| do. The Nerfening has not been subtle.
| HeatrayEnjoyer wrote:
| Using it for legal matters is (understandably) explicitly
| against use policy. It's not meant to be used for legal
| advice so the response you're receiving is accurate - go
| get a lawyer.
|
| https://openai.com/policies/usage-policies
| TeMPOraL wrote:
| It's a toy, it's not meant to be used for anything
| actually useful - they'll keep nerfing it case by case,
| because letting users do useful stuff with their models
| is either leaving money on the table, taking an increased
| PR/legal risk, or both.
| hdhshxhsvc wrote:
| Check again in two months
| rgbrgb wrote:
| I will speculate! They have a model that far surpasses GPT-4
| (achieved agi internally) but sama is back on a handshake
| agreement that they will only reveal models that are slightly
| ahead of openly available LLMs. Their reasoning being that
| releasing a proprietary model endpoint slowly leaks the models
| advantage as competitors use it to generate training data.
| londons_explore wrote:
| > competitors use it to generate training data.
|
| I wouldn't think this matters as long as you charge enough to
| use that API. For example, you could have a tiered pricing
| structure where the first 100k words per month generated
| costs $0.001 per word, but after that it costs $0.01 per
| word.
| anon291 wrote:
| WTF is AGI?
|
| EDIT: Clearly the point is lost on the repliers. There is no
| general understanding of what 'general intelligence' is. By
| many metrics, ChatGPT already has it. It can answer basic
| questions about general topics. What more needs to be done?
| Refinement, sure, but transformer-based models have all
| qualifications to be 'general intelligence' at this point.
| The responses are more coherent than many people I've spoken
| with.
| ebb_earl_co wrote:
| Artificial General Intelligence
| anon291 wrote:
| A meaningless term.
| Xirgil wrote:
| Artificial General Intelligence. OpenAI defines it as
| "highly autonomous systems that outperform humans at most
| economically valuable work"
| anon291 wrote:
| Okay, well, I'll say that at least that's a definition
| (it has to be good enough at something to make money).
| Arguably of course, it already does that. Me personally,
| I've used it to automate tasks I would have previously
| shelled out to fiverr, upwork, and mechanical turk. I've
| had great success using it to summarize municipal codes
| from very very lengthy documents, down to concise
| explanations of relevant pieces of information. Since I
| would have previously paid people to do that (was running
| an information service for a friend), I would consider
| that AGI. I guess the catch here is now 'most', but that
| implies a lot of knowledge about the economy I don't
| think openai has. What is 'economically valuable'? Who
| decides?
|
| At the end of the day, as with most things, AGI is a
| meaningless term because no one knows what that is.
| timeon wrote:
| Any curve-fitting is now AI, so they had to come with new
| term.
| anon291 wrote:
| When you take a university / school course, how is that
| functionally different from curve fitting? Given that
| arbitrarily complex states can be modeled as high-
| dimensional curves, all learning is clearly curve
| fitting, whether in humans, machines, or even at the
| abiological level (for example, self optimizing processes
| like natural selection). Even quantum phenomema are -- at
| the end of the day -- curve fitting via gradient descent
| (hopefully it's had enough time to settle at a global
| minima!)
| wtetzner wrote:
| Artificial General Intelligence
|
| https://en.m.wikipedia.org/wiki/Artificial_general_intellig
| e...
| anon291 wrote:
| The problem with this definition 'an agent that can do
| tasks animals or humans can perform' is that it's not
| clear what that would look like. If you produce a system
| that can be interacted with via text input only but is
| otherwise capable of doing everything a human can do in
| terms of information processing, is that AGI? Or does AGI
| imply a human-like embodied form? Why?
| brucethemoose2 wrote:
| It's not, its just too much noise to see.
|
| We are getting a lot of great models out of China in particular
| (Yi, Qwen, InternLM, ChatGLM), and some good continuations like
| Solar.
|
| Lots of amazing papers on architectures and long context are
| coming out.
|
| Backends are going crazy. Outlines is shoving constrained
| generation everywhere, and Lorax is a LLM revelation as far as
| I'm concerned.
|
| But you won't hear about any of this on Twitter/HN. Pretty much
| the only thing people tweet about is vllm/llama.cpp and
| llama/mistral, but there's a lot more out there than that.
| Nick87633 wrote:
| Where do you like to keep up to date on these? Arxiv
| preprints, or some other place?
| markab21 wrote:
| For Llama-based progress - Reddit - /r/LocalLlama has been
| my top source of info, although it's been getting a little
| more noisy lately.
|
| I also hang out on a few Discord servers: - Nous Research -
| TogetherAI / Fireworks / Openrouter - LangChain - TheBloke
| AI - Mistral AI
|
| These, along with a couple of newsletters, basically keep a
| pulse on things.
| brucethemoose2 wrote:
| Lots of interesting information is so fragmented in niche
| Discords. For instance, KoboldAI on merging and RP models
| in general, and some other niches. Llama-Index. VoltaML
| and some others in regards to SD optimization. I could go
| on and on, and know only a tiny fraction of the useful AI
| discords.
|
| And yeah, /r/LocalLlama seems to be getting noisier.
|
| TBH I just follow people and discuss stuff on huggingface
| directly now. Its not great, but at least its not
| discord.
| cyanydeez wrote:
| surprised someone doesnt just build an AI aggregator for
| this type of thing, seems like a real valueable product.
| DANmode wrote:
| Those rooms move too fast, and often are segregated
| good/better/best (meaning the deeper you want to go on a
| topic, the "harder" it is, politically and labor-wise, to
| get invited to the server).
| brucethemoose2 wrote:
| They have! And posted them on HN!
|
| Some are pretty good! Check out this little curated
| nugget: https://llm-tracker.info/
|
| I used to follow one with a UI that resembled HN itself,
| but now I can't find it in my bookmarks, lol.
| TeMPOraL wrote:
| Speaking of hard skills: how does one just _hang out on a
| Discord server_ in any useful fashion? I lost the ability
| to deal with group chats when I started working full-time
| - there 's no way I can focus on the job _and_ keep track
| of conversations happening on some IRC or Discord. I
| wonder what the trick is to use those things as source of
| information, other than "be a teenager, student, or a
| sysadmin or otherwise someone with lots of spare time at
| work", which is what I realized communities I used to be
| part of consist of.
| sanxiyn wrote:
| I follow https://www.reddit.com/r/LocalLLaMA/
| ipaddr wrote:
| Submissions on hn are welcome.
| brucethemoose2 wrote:
| I have submitted some in the past. Others are submitting
| them! And I upvote every one I like in /new. But HNers
| don't really seem interested unless its llama.cpp or
| Mistral, and I don't want to spam.
|
| I can't say I blame them either, there is a lot of insane
| crypto-like fraud in the LLM/GenAI space. I watch the space
| like a hawk... and I couldn't even tell you how to filter
| it, it's a combination of self-training from experience and
| just downloading and testing stuff myself.
| tmaly wrote:
| I see a ton of papers on X related to this space.
| cyanydeez wrote:
| submissions << public interest
| rightbyte wrote:
| I did try make a submission of what I thought was a
| "underreported" LLM two months ago.
|
| https://news.ycombinator.com/item?id=38505986
|
| Zero interest for some reason.
|
| Edit: Deepseek coder has 4 submissions to HN with almost
| zero interest.
| leereeves wrote:
| Very interesting, but the comment you replied to was
| specifically asking about OpenAI.
| brucethemoose2 wrote:
| Yeah I misinterpreted it open ai as "open source ai"
|
| TBH I do not follow OpenAI much. I like my personal models
| local, and my workplace likes their models local as well.
| alchemist1e9 wrote:
| LoRAX [0] does sound super helpful and so I'd be curious if
| there are some good examples of people applying it. What are
| some current working deployments where one has 100s or 1000s
| of LoRA fine tuned models? I guess I can make up stuff that
| makes sense, so that's not really what I'm asking, I'm
| interested in learning about any known deployments and
| example setups.
|
| [0] https://github.com/predibase/lorax
| brucethemoose2 wrote:
| There aren't really any I know of, because its brand new
| and everyone just uses vllm :P
|
| No one knows about it! Which is ridiculous because batched
| requests with loras is mind blowing! Just like many other
| awesome backends like InternLM's backend, LiteLLM,
| Outline's VLLM fork, Aphroidte, exllamav2 batching servers
| and and such. Heck, a lot of trainers don't even publish
| the loras they merge into base models.
|
| Personally we are waiting on the integration with
| constrained grammar before swapping to Lorax. Then I am
| going to add exl2 quantization support myself... I hope.
| semmulder wrote:
| FYI, vLLM also just added experimental multi-lora
| support: https://github.com/vllm-
| project/vllm/releases/tag/v0.3.0
|
| Also check out the new prefix caching, I see huge
| potential for batch processing purposes there!
| infecto wrote:
| Has it come to a halt? I guess looking only at the perspective
| of no gpt-5...then yes? But I see wide access to multi-modal.
| Huge increases to throughput, not much latency these days.
| Better turbo models though I would agree in some areas those
| have been steps back with poorer output quality. Adding to that
| list, massive cost reductions so its easy to justify any of the
| models being used.
| sp332 wrote:
| OpenAI has a huge amount of people working on training data.
| https://time.com/6247678/openai-chatgpt-kenya-workers/
| https://www.nbcnews.com/tech/innovation/openai-chatgpt-ai-jo...
| pstorm wrote:
| A few theories:
|
| - They have better just stuff, but aren't release it yet. They
| are at the top already, it makes sense they would hold their
| cards until someone got close.
|
| - They are more focused on AGI, and not letting themselves get
| side tracked with the LLM race
|
| - LLMs have peaked and they don't want to release only a minor
| improvement.
| declaredapple wrote:
| > - They are more focused on AGI, and not letting themselves
| get side tracked with the LLM race
|
| FWIW OpenAI seems to have a corroded definition for AGI that
| is essentially "[An] AI system generally smarter than
| humans".
|
| They don't seem to use the typical definition I'm used to of
| some variation of autonomy or (pseudo)-sentience.
|
| So their LLM race is the race for AGI
| JumpCrisscross wrote:
| > _have better just stuff, but aren 't release it yet_
|
| This hypothesis has a curious habit of surfacing when OpenAI
| is fundraising. Together with the world-ending potential of
| their complete-the-sentence kit.
| m3kw9 wrote:
| Likely the way AI is trained and inferenced there is a huge
| constraint on the GPU side even if they have GPT5 out. Imagine
| how slow it is, which means they focus on creating useful
| products and APIs around it first
| crotchfire wrote:
| _Anyone knows what 's happening with open AI ?_
|
| Yeah, they're an arms dealer now.
| colordrops wrote:
| I sometimes wonder if they hit a limit the government was
| comfortable with and their more advanced technologies are
| only available to the gov. I assume that's your implication.
| moffkalast wrote:
| LoRA of war.
| YetAnotherNick wrote:
| GPT 4 is so far ahead of everything else that it doesn't make
| much sense to rush for GPT 5 release. They could get extra GPT
| 5 customers when they release it as they are pretty sure no one
| else would take away those users.
| nickthegreek wrote:
| With a higher operating cost as well.
| jsnell wrote:
| Maybe there are diminishing returns on improving quality, so
| they're trying to improve the efficiency at a given quality
| level instead? There is a lot value to producing results
| instantly, but more importantly efficiency will allow you to
| serve more users (everyone is bottlenecked on inference
| capacity) _and_ gain more users thanks to winning on pricing.
| rudasn wrote:
| Yup, I think this is it. Economies of scale haven't kicked in
| yet, and if they continue in the same path it doesn't look
| achievable.
|
| Think differently is again the way forward.
| onlyrealcuzzo wrote:
| Law of diminishing returns. The low-hanging fruit has been
| picked.
|
| > But the company's CEO, Sam Altman, says further progress will
| not come from making models bigger. "I think we're at the end
| of the era where it's going to be these, like, giant, giant
| models," he told an audience at an event held at MIT late last
| week. "We'll make them better in other ways."
|
| https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...
| moffkalast wrote:
| Given how they've struggled to even make even GPT 4
| economical it's highly unlikely that larger models would be
| in any way cost effective for wide use.
|
| And there's certainly more to be found in training longer on
| better datasets and adjusting the architecture.
| msp26 wrote:
| Alignment tax is real.
|
| But they probably have something more powerful internally.
| GPT-4 took months to be available to the public.
| TheCaptain4815 wrote:
| GPT4 came out 3 years after GPT3
| declaredapple wrote:
| Slight knitpick
|
| GPT 3.5 came out 2 years after GPT 3
|
| GPT 4 came out 1 year after 3.5
| beAbU wrote:
| Slight nitpick, but "nitpick" is not spelled with a "k".
| mring33621 wrote:
| knitpic
| kjreact wrote:
| Actually nitpick IS spelled with a "k", just not one at
| the beginning of the word. If we're gonna be pedantic
| details matter!
| yieldcrv wrote:
| them and meta have bought like all of Nvidia's production
| capacity to run and train their next models
| johnfn wrote:
| I know, it's really disappointing. They've only completely
| changed the world like 3 times last year.
| shmatt wrote:
| Probably an unpopular opinion:
|
| A probabilistic word generator is still a word generator. It
| might be slightly better next version, we've seen it get worse,
| its more of the same
|
| We can talk about AGI all we want, but it wont be built on the
| same technology as these word generators. There will have to be
| a technological breakthrough years before we even get close
|
| The companies focusing on LLMs right now are dealing with
|
| * Generate better words
|
| * Make it cheaper to generate (use less compute)
|
| * Find better training material
|
| There is a ton of money to be made, but its still more of the
| same
| vanviegen wrote:
| And humans are just human generators.
|
| In both cases, intelligence appears to be just an interesting
| emergent side effect.
| discreteevent wrote:
| 1996 Deep Blue beats Gary Kasparov: "And humans are just
| chess engines. In both cases intelligence appears to be
| just an interesting side effect"
| ilaksh wrote:
| I'm sorry but this comment seems to be incredibly uninformed.
| OpenAI just released a new version of gpt-4 turbo quite
| recently. And just in the last couple of days they released the
| ability to use @ to bring any GPT into a conversation on the
| fly.
| summarity wrote:
| Results look comparable to Medium indeed (I'm using it via
| Mistrals API, since I got sick of OpenAI switching their stuff
| up). Medium is pretty great, somewhere between 3.5-turbo and
| 4-turbo qualitatively. Would be awesome to have out there.
| seydor wrote:
| How does a world where GPTs are like the latest version of apache
| or mysql, look like? do we go back to the world of millions of
| web hosts (sorry, AI hosts)
| Tiberium wrote:
| For some more context see my submission from a few days ago -
| https://news.ycombinator.com/item?id=39175611, although it's
| admittedly not mistral-medium, but llama2 trained on the same
| dataset (since the outputs do match quite often with the API)
| whimsicalism wrote:
| how do we know it's not mistral medium
| brucethemoose2 wrote:
| Its smart. Definitely keeping it around as my slow/low context
| local llm.
|
| Seems like a great candiate to merge with other 70Bs as well.
| There aren't a lot of really great 70B training continuations,
| like CodeLlama or sequelbox's continuations.
| jstummbillig wrote:
| The collective game of still just playing catch-up to GPT4, which
| was released a year ago, while having apparently no special sauce
| and full well knowing that OpenAI could come up with something
| much better at any point must be really exhausting.
| rubymamis wrote:
| > "...it appears that not only is Mistral training a version of
| this so-called "Miqu" model that approaches GPT-4 level
| performance, but it may, in fact, match or exceed it, if his
| comments are to be interpreted generously."
| dougmwne wrote:
| GPT-4 is an enormous model that took an enormous amount of
| training. The big news is that smaller teams are getting close
| to its performance on a small model that can run on a single
| GPU. No doubt many of these innovations could be scaled up, but
| pretty much only Google and Microsoft have the compute
| resources for the behemoth models(Not just the training, but
| the giant resources required to run the inference for hundreds
| of millions of users. Google already claims to have surpassed
| GPT-4 with their unreleased Gemini Ultra model. No doubt
| OpenAI/Microsoft is sitting on GPT-5 refining it just waiting
| to leapfrog the competition.
| dom96 wrote:
| Don't forget about Meta
| anon373839 wrote:
| Also, GPT-4 isn't actually a model per se. It's a black-box
| product that uses a model.
| ethanbond wrote:
| Maybe if OpenAI also seemed like it was getting stronger,
| especially organizationally. But if I were Mistral and
| following this quickly while OAI was tripping over its own
| shoelaces... that's gotta be very exciting.
| sebzim4500 wrote:
| Is OpenAI tripping over its own shoelaces? They haven't
| released GPT-5 but then there was over two years between
| training GPT-3 and GPT-4 and they spent 9 months safety
| testing GPT-4 before release.
| pb7 wrote:
| It's more exhausting seeing cheering on a single product
| provider just because they were first in the age of complaining
| about giant tech monopolies. Let them cook. The more options,
| the better. It's only a matter of time before people give
| OpenAI the Google and Apple treatment.
| mysteria wrote:
| Why is this being called an open source model? This is a
| proprietary model that has been leaked on the internet, and will
| remain so until Mistral releases it officially.
|
| Like Llama 1 they won't care about personal use, but no corp is
| going to touch this.
| codetrotter wrote:
| Presumably because it is planned to be released as open source
| jallmann wrote:
| Open source is really about reproducibility. Most of these
| model releases are better described as "open weights" because
| we don't know how exactly they were trained.
| andy99 wrote:
| No, open source is about software freedom, see debian free
| software guidelines from which the "open source definition"
| derives.
| https://wiki.debian.org/DebianFreeSoftwareGuidelines
|
| Between these and FSF you've got pretty much all the
| accepted pontificating about free / open source software.
| Reproducibility is not mentioned because it's not really a
| consideration for software.
|
| Model weights aren't software so there's not an automatic
| correspondence between the freedoms, but the essential one
| you might think you need the training data for is freedom
| to modify and inspect the source.
|
| Modification is fine tuning which you're free to do if you
| have the weights. And the model weights + code fully define
| a system that can be interrogated to give a practitioner
| relevant info about how the model works (within our
| understanding) that the training data isn't needed or
| relevant for. I don't see that any freedom on use or
| inspection is violated by not having the data.
|
| It could be nice to have it of course, but it's more about
| using it to learn how, not exercising any freedom.
|
| Incidentally, the big freedom that's usually violated is
| freedom of discrimination against field of endeavor. LLAMA
| et al list uses and industries they restrict from using
| them and because of that are not "open".
| JumpCrisscross wrote:
| > _open source is about software freedom, see debian free
| software guidelines from which the "open source
| definition"_
|
| The Open Software Foundation ironically screwed the pooch
| on this one. Open source commonly means source available,
| more of less. Free software, as in "'free speech,' not as
| in 'free beer'," is the cumbersome construction for what
| open source aspired to mean [1].
|
| [1] https://www.gnu.org/philosophy/free-sw.en.html
| andy99 wrote:
| Why do you say they screwed up? OSI definition is pretty
| clear.
|
| I think the naming is a challenge because in English
| "open" gives the impression that the key point is that
| you can see it, as opposed to anything about freedom. Is
| that what you mean?
|
| I have heard it said that open source is sort of a
| "commercial friendly" version of free software that de-
| emphasizes user freedom. I think some groups push for
| that (like Meta is trying to redefine what open source
| means wrt AI weights). But the OSI defined freedoms
| basically match what FSF pushes.
| JumpCrisscross wrote:
| > _Why do you say they screwed up? OSI definition is
| pretty clear_
|
| They didn't screw up, they screwed the pooch on open
| source != source available. The Open Group's members--
| from IBM to Huawei [1]--started calling the latter open
| source, which set a precedent that's stuck.
|
| [1] https://en.wikipedia.org/wiki/The_Open_Group#Member_F
| orums_a...
| monocasa wrote:
| > Model weights aren't software so there's not an
| automatic correspondence between the freedoms, but the
| essential one you might think you need the training data
| for is freedom to modify and inspect the source.
|
| Models aren't just the weights, but also the list of
| operations to perform using those weights. I haven't
| heard a good definition that allows for neural network
| models to not be software, but allows any other table
| lookup heavy signal processing algorithm to be software.
|
| > Modification is fine tuning which you're free to do if
| you have the weights. And the model weights + code fully
| define a system that can be interrogated to give a
| practitioner relevant info about how the model works
| (within our understanding) that the training data isn't
| needed or relevant for. I don't see that any freedom on
| use or inspection is violated by not having the data.
|
| Mistral wouldn't constrain themselves to fine tuning if
| they have a big enough change, they would go back to
| their build pipeline. This argument sounds a lot like
| 'there's nothing stopping you from patching the binary,
| so that's basically as good as source'.
| btown wrote:
| I often like to think about
| https://github.com/chrislgarry/Apollo-11 as an analogy.
| It's public domain with available source, in the assembly
| language in which it was written... so it fills all the
| definitions of OSS!
|
| But the process by which that code arose, the ability to
| modify any line and understand its impact (heh) on a real
| execution environment, is dependent on a massive process
| that required billions of dollars and thousands of the
| smartest people on the planet. For all intents and
| purposes, without that environment, it is as reliably
| modifiable as an executable binary in any other context -
| or a set of weights, in this one!
| monocasa wrote:
| I don't think that's a great example.
|
| For instance, I can step through and even modify that
| code using tooling like AGC emulators like this one
| http://www.ibiblio.org/apollo/#gsc.tab=0
|
| What makes it open source is access to the same level of
| source access that the original developers worked in.
|
| That's what's missing here. Mistral's engineers do not
| simply open this binary in their editor to do their job.
| samus wrote:
| Reproducibility becomes an important criterion for models
| though.
|
| For normal programs, it is quite easy to decompile an
| unoptimized binary. Even decompiling an optimized will
| lead to source code. To make this harder, an obfuscator
| has to be used.
|
| A model is different because it relies on its weight,
| which are quite a bit more difficult to inspect. Way
| harder than even obfuscated source code. It is magnitudes
| harder to make statements about which information it
| might divulge upon careful questioning, or evaluate its
| biases, if the training data is not available.
| andy99 wrote:
| Even if you have the data you can't do that stuff any
| better. The makeup of the training set doesn't really
| define the behavior in any tractable way. If anything I
| think it's a distraction and even when it is available
| people probably pay too much attention to what's in the
| training data vs actual behavior.
|
| Edit to say that I see benefits to having the training
| data, just that I don't think it's needed to exercise
| enough freedom to qualify as open source in an analogous
| way to software.
|
| Also to add, training on GPUs is not generally
| reproducible anyway because of execution order.
| monocasa wrote:
| I don't think so, they're earlier release they called "open
| source" but never released anything other than the binary.
| 2devnull wrote:
| People really abuse this term. Like using "natural" to market
| stuff. But arsenic is "all natural"! It's "open source" so it
| must be beneficial. "rm -rf c" is "open source" too
| TeMPOraL wrote:
| > _Like using "natural" to market stuff._
|
| Also, because people are idiotically afraid of E-numbers,
| some manufacturers figured they can find whatever fruit or
| bean is naturally rich in the relevant E-compound, and use
| that in the process, allowing them to replace E-whatever
| with ${cute plant name} in the ingredient list (at some
| loss of process efficiency).
| GaggiX wrote:
| Well I remember Tencent using the leaked NovelAI model in
| "Different Dimension Me".
| colordrops wrote:
| Where do you get the idea that corps aren't touching LLama...
| snovv_crash wrote:
| The weights aren't created by humans so they aren't
| copyrightable - at least in theory.
| monocasa wrote:
| That has nothing to do with whether it's open source or not.
|
| Software wasn't for sure copyrightable before 1976 in the US,
| but there was plenty of closed source code that simply only
| ever distributed the binaries.
| epistasis wrote:
| It's open source in the sense that you can take the
| existing weights, apply any one of dozens of techniques for
| adapting the model to your data, and anybody you give your
| model to can do the same.
|
| This right isn't protected by copyright, it's protected by
| the lack of copyright, and how executables are directly
| modifiable. So the AGPL won't be possible, but it's a lot
| like permissive licenses, without attribution.
| monocasa wrote:
| That's like saying an executable that I never gave out
| the source to is "open source" because there's nothing
| stopping you from binary patching it.
|
| Also, the license to the model assumes copyright (Apache
| 2) and works by granting you permissions based on that
| copyright.
| epistasis wrote:
| It's very very different from patching a binary, because
| these models don't work like a simple compilation of
| understandable source code into incomprehensible weights.
| The source training data is even less comprehensible and
| useful than its encapsulation into weights.
|
| A set of weights is a pre-trained local minima in the
| model space is a both executable and modifiable. It's
| usually far more useful than the source training data,
| because the work has been done.
| monocasa wrote:
| > It's very very different from patching a binary,
| because these models don't work like a simple compilation
| of understandable source code into incomprehensible
| weights. The source training data is even less
| comprehensible and useful than its encapsulation into
| weights.
|
| Tons of binary patching works that way. For instance from
| the gameshark days it was relatively common for a patch
| that worked for unknown reasons, but who's discovery was
| tooling assisted and simply displayed a desired effect
| (and commonly a lot of undesired effects that weren't
| clear).
|
| > A set of weights is a pre-trained local minima in the
| model space is a both executable and modifiable.
|
| None of that changes whether it's open source or not. I
| guarantee you that mistral has some code (and a lot o
| data) laying around that created this model.
|
| > It's usually far more useful than the source training
| data, because the work has been done.
|
| For certain operations maybe. I guarantee that Mistral
| wouldn't constrain themselves to fine-tuning if they had
| a major change to make to the model.
| sroussey wrote:
| It's very likely protected by EU copyright.
| Sayrus wrote:
| Copyright is not the only kind of Intellectual Property.
| While they may not be copyrightable (I think you may be
| refering to jugement related to having an AI as author?),
| they clearly are MistralAI IP.
|
| In the same way, data in a database are usually not
| copyrightable. They are still the company's property
| (excluding issues with PII and such).
| bhickey wrote:
| Database rights don't exist in the United States.
|
| Their weights obviously aren't protected by trademark. So,
| what IP regime protects the weights?
| Sayrus wrote:
| Mistral AI is incorporated in Paris, France. IANAL, but I
| think these[1][2] may qualify to protect these weights.
|
| While there is a discussion to be had on sovereignty and
| international reaches of local laws (such as DMCA for
| instance), I think it's disingenuous to consider only US
| legal point of view.
|
| [1] https://eur-lex.europa.eu/legal-
| content/EN/TXT/HTML/?uri=CEL... [2] https://www.legifranc
| e.gouv.fr/loda/id/JORFTEXT000000573438/...
| greiskul wrote:
| I'm not so sure. If it is not copyrightable, and it is a
| patent, and not a trademark, which legal protection would
| it have? It could definitely be considered a trade secret,
| and the theft of it and misapproation of it would be a
| crime, but once it has been published, people that just
| used the published version I don't believe would be
| comitting a crime, since it would lose it's trade secret
| status.
| Sharlin wrote:
| There are certain "related rights" [1] that are weaker
| than full copyright. These include, maybe most
| importantly, performers' rights, but also protection for
| things like photographs (ones that aren't unique enough
| to qualify for full copyright) and, in many
| jurisdictions, databases. These aren't covered by the
| Berne convention so vary quite a bit. But whether a huge
| chunk of floating-point numbers qualifies for any legal
| protection, remains to be seen.
|
| [1] https://en.wikipedia.org/wiki/Related_rights
| sroussey wrote:
| In Europe you can copyright a database of facts, which is
| not something you can do in the US.
|
| Also, people go to jail for leaking trade secrets, and
| this certainly qualifies.
| shiandow wrote:
| That does raise the question how irreversibly you have to
| process something before it is no longer protected by
| copyright.
|
| Obviously zipping something is not enough, even lossy
| compression is (obviously) not enough. But then how does a
| language model differ from a lossy compression? Is it just
| the compression ratio? (are the weights even that much
| smaller than the data?)
|
| There are ways to train models that guarantee that the amount
| of information transferred per data point is limited, but to
| my knowledge those aren't used (and may be prohibitively
| expensive).
| wongarsu wrote:
| When I take a picture with my phone, those pixels weren't
| created by a human either. Yet I still own their copyright.
|
| At the core the question is how much artistic input there is
| in creating LLM models. Are the choice of the model
| architecture, hyperparameters and training data artistic
| choices comparable to those by a photographer setting up a
| shot? Or are they more comparable to technical work that's
| only protectable by trade secrets, patents and trademarks?
| Q6T46nT668w6i3m wrote:
| The parent is asking about weights and from that
| perspective, _stochastic approximation_ is essential.
| londons_explore wrote:
| "Nearing GPT-4"...
|
| The leaderboard[1] shows that there is a huge gap between
| GPT4-0314 and GPT4-Turbo. So if you only just are nearing
| GPT-4-0314, then you're still a year behind the state of the art.
|
| [1]: https://huggingface.co/spaces/lmsys/chatbot-arena-
| leaderboar...
| Terretta wrote:
| Or you're ahead of it, since the earlier GPT4 models beat the
| GPT4-Turbo on a variety of technical use cases.
| YetAnotherNick wrote:
| The leaderboard is easy to hack as well.
| londons_explore wrote:
| I think you might be thinking of GPT-4-0613? That was pretty
| crap all round (but was faster)
| kromem wrote:
| The leaderboards are crap.
|
| Does no one know Goodhart's Law anymore?
|
| We're overtuning for the boards and losing broader capabilities
| not being selected for in the process, such as creative writing
| quality.
|
| There's an anchoring bias around what 'AI' is supposed to be
| good at which reflects what engineers are good at and so the
| engineers with an anchoring bias are evaluating how good LLMs
| are at those things and using it as a target.
|
| Skill-Mix is a start to maybe a better approach, but there
| needs to be a shift in evaluation soon.
| londons_explore wrote:
| Can you give any example queries where the result quality is
| far away from the rankings on the leaderboard? In my
| experience it's pretty spot on, so I'm curious if you're
| asking different sorts of things, or have a different
| definition of a good answer.
|
| I have for example asked "Write me a very funny scary story
| about the time I was locked in a graveyard", and the simpler
| models don't seem to understand that before getting super
| scared and running out of the graveyard they need to explain
| how exactly I was locked in, and what changed that let me
| out.
| refulgentis wrote:
| The top comment to an LMSys leaderboard link on HN is always
| a variation of this song.
|
| And it's always not even wrong, in the Pauli sense of the
| phrase.
|
| OP, your assignment, if you choose to accept it, is to look
| into how the LMSys leaderboard works and report back what its
| metric(s) are.
|
| [SPOILER] There's a really absurdly narrow argument you can
| make where all of its users are engineers, making engineer
| queries, and LLM makers are optimizing for it thus it's bad.
| But...it's humans asking queries then picking the better
| answer, blind. You can't narrowly optimize for that when
| making an LLM and it's hard to see how optimizing for "people
| think the answer is better" is the wrong metric here. It's
| just ELO. Might as well argue chess/checkers/pick your poison
| is bad because ELO optimizes for wins but actually talent is
| based on more than winning.
| numeri wrote:
| The linked leaderboard is actually very trustworthy, in that
| it consists not of scores on a test dataset, but of ELO
| ratings generated by actual humans' ratings of the models'
| responses.
|
| You can go and enter any prompt you like, wait a bit, and
| then get two LLM responses back, which you can then rank or
| mark as tied, after which you'll be shown which model each
| came from. Maybe you already knew this, maybe you didn't. In
| any case, I don't see any real way for Goodhart's Law to
| apply here - the metric and the goal are the same here, i.e.,
| human approval of answers.
| declaredapple wrote:
| Parent linked the Lmsys chatbot arena which is where humans
| blindly get the results from two different models, and vote
| for the response they liked more. So the LLMs are compared by
| elo.
|
| Do you think goodhart's law applies here, since this
| leaderboard doesn't use specific measures, but rather relies
| on whatever the human was looking for?
|
| This is the only leaderboard I personally care about at all.
| nohat wrote:
| It has been interesting seeing the sleuthing on this one. IMHO it
| is unfortunate to have this happen to a company that has been
| very pro open source.
| bee_rider wrote:
| Since these models are trained by just scraping the internet,
| "scraping" this thing and including it in your own model seems
| like fair game, right?
| zamadatix wrote:
| If you mean training your model on it I'd say yeah, fair game.
|
| Mistral is Apache 2.0 licensed though so the question is a bit
| moot, they'd like you to use it for your own model.
| speedgoose wrote:
| It sounds fair. Just make sure to not involve lawyers.
| dopa42365 wrote:
| Is this an advertisement for a "promise"? Entirely worthless
| until there's something to show. It smells.
| a_wild_dandan wrote:
| Guess I'll watch TheBloke's page until I can run his Miqu Q5
| quant on my MacBook. Mixtral is my daily driver, and if this (or
| the newer, official) release nears GPT-4, that's a wrap for my
| OpenAI subscription.
|
| The small team at Mistral is putting their competitors to shame.
| They're what "Open"AI should've been.
| MallocVoidstar wrote:
| Full weights aren't available, only Q2, Q4, Q5 quants via
| miqudev.
| throwaway9274 wrote:
| Unquantized model is here:
| https://huggingface.co/152334H/miqu-1-70b-sf
|
| This strikes me as less a leak and more clever marketing from
| Mistral.
| MallocVoidstar wrote:
| That isn't unquantized, it's _de_ -quantized. They went
| from Q5 to fp16 for use in Pytorch instead of the GGUF
| ecosystem.
| Taek wrote:
| I never thought people would be upscaling models by
| increasing quantization precision. The rationale makes
| sense bit its also a goofy outcome.
| throwaway9274 wrote:
| Yes, that's correct. Good correction.
| SparkyMcUnicorn wrote:
| Looks like the leaked model is already quants in gguf format,
| so no need to wait for TheBloke.
|
| https://huggingface.co/miqudev/miqu-1-70b/tree/main
| brucethemoose2 wrote:
| The GGML quants are the _only_ quantization we have lol. They
| were leaked in Q2K /Q4KM/Q5KM, you can grab them right now.
|
| Whats interesting is that Mistral apparently _distributed_
| these GGUFs. This is (in my experience) not a good format for
| production, so I am curious exactly who was wanting to test a
| model in GGUF.
| whatwhaaaaat wrote:
| It's by far the easiest to consume and run format no? Spans
| devices easy. No weights or extra stuff. For "production"
| maybe not but to get in into the hands of the masses this
| seems perfect.
| brucethemoose2 wrote:
| Yeah its by far the least trouble. Pretty much any other
| backend, even a "pytorch free" backend like MLC, is a utter
| nightmare to install, and that's if it uses a standardized
| quantization.
|
| However, the llama.cpp server is... very buggy. The OpenAI
| endpoint doesn't work. It hangs and crashes constantly. I
| don't see how anyone could use it for batched production as
| of last november/december.
|
| The reason I don't use llama.cpp personally is no flash
| attention (yet) and no 8 bit kv cache, so its not too great
| at long (32K+) contexts. But this is a niche, and being
| addressed.
| samstave wrote:
| What are you needing 32K contexts for, as use case
| examples plz.
|
| From [0] @sdo72 writes::
|
| >>> _...32k tokens, 3 /4 of 32k is 24k words, each page
| average is 500 or 0.5k words, so that's basically 24k /
| .5k = 24 x 2 =~48 pages...._"
|
| https://news.ycombinator.com/item?id=35841460
|
| EDIT: I may be ignorant: does the 32k mean its output
| context, or single conversation attention span, or how
| much it can ingest in a prompt?
| fragmede wrote:
| Feed it my source code to use as context for how to
| refactor things. the bigger the context window, the more
| source code it can "read".
| samstave wrote:
| Thanks, Curious as I havent been in a forum where Ive
| seen asked:
|
| Have you found a particular manner in which to feed it in
| - do you give it instructions for what it is looking for,
| what kind of phrases are you directing it to do?
|
| I am about to start a try at a gpt co-piloted effort, and
| I have only done art so far - so curious if there are
| good pointers on coding with gpt?
| gryn wrote:
| continue.dev works great for me, supports vs code and
| jetbrains ide, there's shortcuts to give it code snipets
| as context and in place editing. works with all kind of
| LLM sources. both gpt and local stuff.
|
| still haven't found anything that can read a whole
| project source code in a single click though
| refulgentis wrote:
| 32K tokens = "context size" = sum of input tokens + max
| output tokens
| samstave wrote:
| Thank you - so you want the most efficient small input
| tokens for the max output tokens, or at least the best
| answer with the smallest amount of tokens used - but
| enough headroom it wont lose context and start down the
| hallucination path?
| refulgentis wrote:
| Going to be extremely opinionated and straightforward
| here in service of being concise, please excuse me if it
| sounds rough or wrong, feel free to follow-up:
|
| You're "not even wrong", in that you don't really need to
| worry about ratio of input to output, or worry about
| inducing hallucinations.
|
| I feel like things went generally off-track once people
| in the ecosystem turned RAG into these weird multi-stage
| diagrams when really it's just "hey, the model doesn't
| know everything, we should probably give it web pages /
| documents with info in it"
|
| I think virtually all people hacking on this stuff daily
| would quietly admit that the large context sizes don't
| seem to be transformative. Like, I thought it meant I
| could throw a whole textbook in and get incredibly rich
| detailed answers to questions. But it doesn't. It still
| sort of talks the way it talks, but obviously now it has
| a lot more information to work with.
|
| Thinking out loud: maybe the way I think about it is the
| base weights are lossy and unreliable. But, if the
| information is in the context, that is "lossless". The
| only time I see it gets things wrong is when the
| information itself is formatted weird _.
|
| All that to say, in practice, I don't see much gains in
| _question-answering* when I provided > 4K tokens.
|
| But, the large context sizes are still nice because A) I
| don't need to worry as much about losing previous
| messages / pushing out history when I add documents as
| when it was just 4K for ChatGPT. B) It's _really_ nice
| for stuff like information extraction, ex. I can give it
| a USMLE PDF and have it extract Q+A without having to
| batch it into like 30 separate queries and reassamble. C)
| There 's some obvious cases where the long context length
| helps, ex. if you know for sure a 100 page document has
| some very specific info in it, you're looking for a
| specific answer, and you just don't wanna look it up
| again, perfect!
|
| * I've been working on "Siri/Google Assistant but cross
| platform and on LLMs", RAG + local + on all platforms +
| sync engine for about a [REDACTED]. It can nail ~every
| question at a high level, modulo my MD friend needs to
| use GPT-4 for that to happen. The failures I see are if I
| ask "what's the lakers next game", and my web page =>
| text algo can't do much with tables, so it's formatted in
| a way that causes it to error.
| speedgoose wrote:
| Have you tried ollama? It's another llama.cpp server
| implementation that is becoming popular.
| gbickford wrote:
| The authors don't seem to care about the principle of
| least privilege: https://github.com/ollama/ollama/issues/
| 851#issuecomment-177...
|
| It makes me wonder what other security issues they might
| now care about.
| smoothjazz wrote:
| This is a mac problem, not an ollama problem. It also
| sounds like it's solved by using homebrew (or linux).
| declaredapple wrote:
| Just to throw out there - I haven't had any issues with
| exllama personally, and it's a lot faster last I checked.
| htrp wrote:
| >The small team at Mistral is putting their competitors to
| shame.
|
| Them plus 500 million in funding
| api wrote:
| Much of the cost is probably compute.
| refulgentis wrote:
| Very unlikely: note best GPT-4 estimates put at it $45
| million
| a_wild_dandan wrote:
| Altman said that GPT-4 training was over $100 million.
| And you need significant additional resources beside just
| the training run cost.
| tomp wrote:
| so like 10x less funding than OpenAI and 100x less than MSFT,
| AAPL, Google, Facebook, ...
| jtonz wrote:
| I think it's fair to say when you hit the hundreds of
| millions of dollars mark the diminishing returns for making
| things happen faster have well and truly kicked in.
|
| Perhaps the only benefit would be extra computational power
| yet I would struggle to understand the benefit of jumping
| from 500 million to 5 billion with such short timeframes.
| karmasimida wrote:
| Near GPT-4 is definitely a stretch.
|
| The hype around Mixtral is huge, and my disappointment follows,
| it doesn't have very good knowledge on books, for example.
| terhechte wrote:
| It's a much smaller model, it didn't ingest the knowledge of
| the world. It is great at working with the information you
| give it, but if you want to extract information like a search
| engine, GPT4 is the king because it is so much bigger.
| avereveard wrote:
| That is a good thing you want the model to process language,
| knowledge is a side effect of how it gets there. If you rely
| on training knowledge it's going to be hard to know when you
| pass the boundary into hallucinations, what you want instead
| is a model that can pretend reasoning while supporting tools
| inject knowledge from the world into the context as it
| iterate toward an answer
| karmasimida wrote:
| But here is the thing, if you rely on RAG or any other
| knowledge injection for recommendation, you essentially
| make your LLM parrot machine, no better than a search
| engine, just much slower.
|
| Knowledge is one aspect of it, I found its instruction
| following ability, frustrating as well.
|
| I think Ilya Sutskever puts it very well, larger model
| brings stability to wider range of tasks, what smaller
| models are not capable of. Even though book recommendation
| with GPT might a niche, but it is not something really
| unexpected TBH, thus comes my disappointment.
| chasd00 wrote:
| > But here is the thing, if you rely on RAG or any other
| knowledge injection for recommendation, you essentially
| make your LLM parrot machine, no better than a search
| engine, just much slower.
|
| this is a really good point. I was working on some
| careful q/a data curation today that is then fed to a
| vector store where embeddings are calculated and served.
| I realized that my carefully curated q/a data in
| combination with the vector database works just fine for
| what i want to do all by itself. A really good semantic
| search of my q/a database turns up answers to my
| questions with no rag llm prompting required. When I
| added the llm it just put the same information in
| different words, not super useful when i could have just
| looked at the returned embeddings and gotten the same
| information.
| chrishare wrote:
| I hate when that happens
| accrual wrote:
| What's wild to me is that this "leak" won't matter in a couple of
| months. The official model will come out, then an even better
| model will come out. It's fun to get hyped but just like every
| other leak, it'll be surpassed by the real thing and its
| successor in a little bit. The fast pace of things is what has me
| excited, not any particular model.
| refulgentis wrote:
| I don't think so, TFA says it's an early version of an old
| model that's already been distributed openly. ;)
| 2OEH8eoCRo0 wrote:
| Seems paradoxical. How does something that is open source leak?
| vulcan01 wrote:
| According to the article, it _will_ be open source, but it 's
| not open source now.
| sharkjacobs wrote:
| Mistral reminds me of the good old days of pre-2015 when I
| thought that tech companies were cool.
| syntaxing wrote:
| What a class act. Going to the leaked model on huggingface, not
| demanding it to be taken down, and just making a post on the page
| saying "Might consider attribution" is so damn amazing.
___________________________________________________________________
(page generated 2024-01-31 23:01 UTC)