[HN Gopher] Deepseek R1-0528
___________________________________________________________________
Deepseek R1-0528
Author : error404x
Score : 211 points
Date : 2025-05-28 17:59 UTC (5 hours ago)
(HTM) web link (huggingface.co)
(TXT) w3m dump (huggingface.co)
| transcriptase wrote:
| Out of sheer curiosity: What's required for the average Joe to
| use this, even at a glacial pace, in terms of hardware? Or is it
| even possible without using smart person magic to append
| enchanted numbers and make it smaller for us masses?
| jacob019 wrote:
| I'm sure it will be on OpenRouter within the next day or so.
| Not really practical to run a 685B param model at home.
| behohippy wrote:
| About 768 gigs of ddr5 RAM in a dual socket server board with
| 12 channel memory and an extra 16 gig or better GPU for prompt
| processing. It's a few grand just to run this thing at 8-10
| tokens/s
| terhechte wrote:
| You can run the 4bit quantized version of it on a M3 Ultra
| 512GB. That's quite expensive though. Another alternative is a
| fast CPU with 500GB of DDR5 RAM. That of course, is also not
| cheap and slower than the M3 Ultra. Or, you buy multiple Nvidia
| cards to reach ~500GB of VRam. That is probably the most
| expensive option but also the fastest
| lodovic wrote:
| If you use the excess memory for AI only it's cheaper to rent
| . A single H100 costs less than $2 per hour. (incl power)
| diggan wrote:
| Vast.ai has a bunch of 1x H100 SXM available, right now the
| cheapest at $1.554/hr.
|
| Not affiliated, just a (mostly) happy user, although don't
| trust the bandwidth numbers, lots of variance (not
| surprising though, it is a user-to-user marketplace).
| omneity wrote:
| Worth mentioning that a single H100 (80-96GB) is not enough
| to run R1. You're looking at 6-8 GPUs on the lower end, and
| factor in the setup and download time.
|
| An alternative is to use serverless hosts such as Fireworks
| or Featherless which abstracts some of this for you, albeit
| at a higher cost and slow starts when you first use your
| model for some time.
| hu3 wrote:
| It's probably going to be free at OpenRouter.
|
| There's already a 685B parameter DeepSeek V3 for free there.
|
| https://openrouter.ai/deepseek/deepseek-chat-v3-0324:free
| latchkey wrote:
| It is free to use, but you're feeding OR data and someone is
| profiting off that.
| ankit219 wrote:
| Thats how a lot of application layer startups are going to
| make money. There is a bunch of high quality usage data.
| Either you monetize it yourself (cursor), get acquired
| (windsurf) or provide that data to others at a fee (lmsys,
| mercor). This is inevitable and a market for this is just
| going to increaase. If you want to prevent this as an org,
| there arent many ways out. Either use open source models
| you can deploy, or deal directly with model providers where
| you can sign specific contracts.
| 85392_school wrote:
| You're actually sending data to random GPUs connected to
| one of the Bittensor subnets that run LLMs.
| latchkey wrote:
| That can, today, collect that data and sell it. There is
| work being done to add TEE, but it isn't live yet.
| dist-epoch wrote:
| Not every prompt is privacy sensitive.
|
| For example you could use it to summarize a public article.
| latchkey wrote:
| Every prompt is valuable.
| criddell wrote:
| And you are getting something valuable in return. It's
| probably a good trade for many, especially when they are
| doing something like summarizing a public article.
| dist-epoch wrote:
| Using an AI for free is also valuable. Seems win/win.
| jacob019 wrote:
| I'm not so sure. I have agents that do categorization
| work. Take a title, drill through a browse tree to find
| the most applicable leaf category. Lots of other
| classification tasks that are not particularly sensitive
| and it's hard to imagine them being very good for
| training. Also transformations of anonymized numerical
| data, parsing, etc.
| hadlock wrote:
| As mentioned you can run this on a server board with 768+ gb
| memory in cpu mode. Average joe is going to be running
| quantized 30b (not 600b+) models on an $300/$400/$900 8/12/16gb
| GPU
| rahimnathwani wrote:
| I'm not sure that's enough RAM to run it at full precision
| (FP8).
|
| This guy ran a 4-bit quantized version with 768GB RAM:
| https://news.ycombinator.com/item?id=42897205
| SkyPuncher wrote:
| Practically, smaller, quantized versions of R1 can be run on a
| pretty typically Macbook Pro setup. Quantized versions are
| definitely less performant, but they will absolutely run.
|
| Truthfully, it's just not worth it. You either run these things
| so slowly that you're wasting your time or you have to buy 4-
| or 5-figures of hardware that's going to sit, mostly unused.
| danielhanchen wrote:
| We made DeepSeek R1 run on a local device via offloading and
| 1.58bit quantization :)
| https://unsloth.ai/blog/deepseekr1-dynamic
|
| I'm working on the new one!
| screaminghawk wrote:
| I use this a lot! Thanks for your work and looking forward to
| the next one
| danielhanchen wrote:
| Thank you!! New versions should be much better!
| CamperBob2 wrote:
| Your 1.58-bit dynamic quant model is a religious experience,
| even at one or two tokens per second (which is what I get on
| my 128 MB Raptor Lake+4090). It's like owning your own
| genie... just ridiculously smart. Thanks for the work you've
| put into it!
| danielhanchen wrote:
| Oh thank you! :) Glad they were useful!
| mechagodzilla wrote:
| I have a $2k used dual-socket xeon with 768GB of DDR4 - It runs
| at about 1.5 tokens/sec for the 4-bit quantized version.
| threeducks wrote:
| > even at a glacial pace
|
| If speed is truly not an issue, you can run Deepseek on pretty
| much any PC with a large enough swap file, at a speed of about
| one token every 10 minutes assuming a plain old HDD.
|
| Something more reasonable would be a used server CPU with as
| many memory channels as possible and DDR4 ram for less than
| $2000.
|
| But before spending big, it might be a good idea to rent a
| server to get a feel for it.
| z2 wrote:
| Hardware: any computer from the last 20 or so years.
|
| Software: client of choice to
| https://openrouter.ai/deepseek/deepseek-r1-0528
|
| Sorry I'm being cheeky here, but realistically unless you want
| to shell out 10k for the equivalent of a Mac Studio with 512GB
| of RAM, you are best using other services or a small distilled
| model based on this one.
| jacob019 wrote:
| Not much to go off of here. I think the latest R1 release should
| be exciting. 685B parameters. No model card. Release notes?
| Changes? Context window? The original R1 has impressive output
| but really burns tokens to get there. Can't wait to learn more!
| willchen wrote:
| I love how Deepseek just casually drops new updates (that deliver
| big improvements) without fanfare.
| hd4 wrote:
| On the day Nvidia report earnings too. Pretty sure it's just a
| coincidence, bro.
| margorczynski wrote:
| Yeah the timing seems strange. Considering how much money
| will move hands based on those results this might be some
| kind of play to manipulate the market at least a bit.
| consumer451 wrote:
| I believe that they are funded by a hedge fund. So, there
| are no coincidences here.
| Maxatar wrote:
| How does releasing it today affect the market compared to
| releasing it last week?
| doctoboggan wrote:
| Hard to say exactly how it will affect the market, but
| IIRC when deepseek was first released Nvidia stock took a
| big hit as people realized that you could develop high
| performing LLMs without access to Nvidia hardware.
| jimmyl02 wrote:
| I thought the reaction was more so that you can train
| SOTA models without an extremely large quantity of hyper-
| expensive GPU clusters?
|
| But I would say that the reaction was probably vastly
| overblown as what Deepseek really showed was there are
| much more efficient ways of doing things (which can also
| be applied with even larger clusters).
|
| If this checkpoint is trained using non-Nvidia GPUs that
| would definitely be a much bigger situation but it
| doesn't seem like there has been any associated
| announcements.
| TeMPOraL wrote:
| Plans take time to adjust; I imagine a big part of the
| impact was companies realizing that they need to buy/rent
| _much less_ expensive GPU compute to realize the plans
| they 've already committed to for the next couple years.
| Being able to spend less to get the same results is an
| immediate win; expanding the plan to make use of suddenly
| available surplus money/compute takes some time.
|
| And then part of the impact was just "woah, if some
| noname team from China can casually leapfrog major
| western players on a tiny budget _and_ kill one of their
| moats in the same move, what other surprises like this
| are possible? ". The event definitely invalidated a lot
| of assumptions investors had about what is or isn't
| possible near-term; the stock market reacted to suddenly
| increased uncertainty.
| belter wrote:
| Plenty of manipulation to go around..
|
| "Tech Chip software stocks sink on report Trump ordered
| halt to China sales" -
| https://www.cnbc.com/2025/05/28/chip-software-trump-
| china.ht...
| rwmj wrote:
| Is releasing a better product really "market manipulation"?
| It seems to me like regular, good competition.
| FirmwareBurner wrote:
| It's "manipulating the market" only when your
| geopolitical adversary brings the competition.
| doctoboggan wrote:
| Honest question, how do you know this is a big improvement? Are
| there any benchmarks anywhere?
| KeyBoardG wrote:
| There will be a video from FireShip if its a big one. /s
| modeless wrote:
| I like it too, but some benchmark numbers would be nice at
| least.
| esafak wrote:
| Anyone got benchmarks?
| ilaksh wrote:
| I think they did make an announcement on WeChat.
| acheong08 wrote:
| No information to be found about it. Hopefully we get benchmarks
| soon. Reminds me of the days when Mistral would just tweet a
| torrent magnet link
| aibrother wrote:
| getting a similar vibe yeah. given how adjacent they are,
| wouldn't be surprised if this was an intentional nod from
| DeepSeek
| swyx wrote:
| i think usually deepseek posts a paper after a model release
| about a day later.
|
| no idea why they cant just wait a bit to coordinate stuff. bit
| messy in the news cycle.
| Destiner wrote:
| honestly a power move.
|
| it's almost as if they don't care about creating a proper
| buzz.
| wyre wrote:
| From what I understand, isn't DeepSeek just a pet project
| from a Chinese hedge fund? They have much less reason to
| create a buzz compared to openAI, Anthropic, or Google.
| TeMPOraL wrote:
| None of those players you mention actually need to
| _create_ a buzz. People will do it for them for free.
| DeepSeek joined this group after releasing R1.
|
| Despite constant protestations of hype among the tech
| crowd, GenAI really is big enough of a deal that new
| developments don't need to be pushed onto market; people
| are voluntarily seeking them out.
| wongarsu wrote:
| OpenAI does a lot of work hyping themselves up and
| creating buzz around things they do or have a vague idea
| that they might try to do in the future.
|
| Not to make people aware of GenAI, but to make sure
| OpenAI continues to be perceived as _the_ AI company. The
| company that leads and revolutionizes, with everyone just
| copying them and trying to match them. That perception is
| a significant part of their value and probably their
| biggest moat
| chvid wrote:
| Benchmarks seem like a fools errand at this point; overly
| tuning models just to specific test already published tests,
| rather than focusing on making them generalize.
|
| Hugging face has a leader board and it seems dominated by
| models that are finetunings of various common open source
| models, yet don't seem be broader used:
|
| https://huggingface.co/open-llm-leaderboard
| lossolo wrote:
| The only benchmarks that match my experience with different
| models are here https://livebench.ai/#/
| EvgeniyZh wrote:
| There are quite a few benchmarks for which that's not the
| case:
|
| - live benchmarks (livebench, livecodebench, matharena, SWE-
| rebench, etc)
|
| - benchmarks that do not have a fixed structure, like games
| or human feedback benches (balrog, videogamebench, arena)
|
| - (to some extent) benchmark without existing/published
| answers (putnambench, frontiermath). You could argue that
| someone could hire people to solve those or pay off benchmark
| dev, but it's much more complicated.
|
| Most of the benchmarks that don't try to tackle future
| contamination are much less useful, that's true.
| Unfortunately, HLE kind of ignored it (they plan to add a
| hidden set to test for contamination, but once the answers
| are there, it's a lost game IMHO); I really liked the
| concept.
|
| Edit: it is true that these benchmarks are focusing only on a
| fairly specific subset of the model capabilities. For
| everything else vibe check is your best bet.
| z2 wrote:
| There's a table here showing some "Overall" and "Median" score,
| but no context on what exactly was tested. It appears to be in
| the ballpark as the latest models, but with some cost
| advantages with the downside of being just as slow as the
| original r1 (likely lots of thinking tokens).
| https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
| xelos wrote:
| It's appeared on the Livecodebench leaderboard too.
| Performance on par with O4 Mini -
| https://livecodebench.github.io/leaderboard.html
| heyhuy wrote:
| Deepseek bought some NVidia puts last night
| htrp wrote:
| You're gonna need at least 8 h100 80s for this....
| overfeed wrote:
| That's about $16-24 per hour - depending on the number of
| tokens you're slinging in that period, it may be much cheaper
| than paying OpenAI for similar functionality.
| canergly wrote:
| I want to see it in groq asap !
| porphyra wrote:
| Groq doesn't even have any true deepseek models --- I thought
| they only had `deepseek-r1-distill-llama-70b` which was
| distilled onto llama 70b [1].
|
| [1] https://console.groq.com/docs/models
| jacob019 wrote:
| Groq has a weak selection of models, which is frustrating
| because their inference speed is insane. I get it though,
| selection + optimization = performance.
| sergiotapia wrote:
| the only reason they are fast is because the models they
| host are severely quantized so i've heard.
| jacob019 wrote:
| Huh. I heard a podcast with the founder talking about
| their custom hardware, but quantization would explain it.
| dubrado wrote:
| You can test it now at Hyperbolic.
| app.hyperbolic.xyz/compute?utm_source=hackernews You get $1 free
| for signing up for a free account, and $15 if you have an EDU
| account. Not to mention $5 for referring others.
| jacob019 wrote:
| Well that didn't take long, available from 7 providers through
| openrouter.
|
| https://openrouter.ai/deepseek/deepseek-r1-0528/providers
|
| May 28th update to the original DeepSeek R1 Performance on par
| with OpenAI o1, but open-sourced and with fully open reasoning
| tokens. It's 671B parameters in size, with 37B active in an
| inference pass.
|
| Fully open-source model.
___________________________________________________________________
(page generated 2025-05-28 23:00 UTC)