[HN Gopher] GPT4 is up to 6 times more expensive than GPT3.5
___________________________________________________________________
GPT4 is up to 6 times more expensive than GPT3.5
Author : behnamoh
Score : 149 points
Date : 2023-03-14 19:31 UTC (3 hours ago)
(HTM) web link (openai.com)
(TXT) w3m dump (openai.com)
| Veen wrote:
| I imagine this price will come down over time. OpenAI has
| repeatedly said that they haven't the infrastructure in place yet
| to handle too much load and expect to be "severely capacity
| constrained", so I assume the high pricing and usage caps on the
| ChatGPT version are to keep the load manageably low for the
| moment.
| jazzyjackson wrote:
| You're not wrong but I think many people still have an
| intuition informed by a pre-silicon-capacity-crunch era, it
| cannot be taken for granted that the demand for GPU / TPU will
| be met over the next few years.
| avisser wrote:
| I imagine those dollars from papa Nadella are buying LOTS of
| capacity as we speak.
| albntomat0 wrote:
| This may be because they recently found ways to make GPT3.5 much
| more efficient.
|
| https://openai.com/blog/introducing-chatgpt-and-whisper-apis
| sparrc wrote:
| 6 times? My reading of these prices is that it's 45x more
| expensive (for 8K) and 90x more expensive (32k)
| behnamoh wrote:
| The title compares GPT3.5 (not ChatGPT) with GPT4. Although the
| question is: If ChatGPT can do almost anything GPT3.5 can do,
| then why is it priced almost 10x less? Is it because GPT3.5 is
| less "censored"?
| dragonwriter wrote:
| > Although the question is: If ChatGPT can do almost anything
| GPT3.5 can do, then why is it priced almost 10x less?
|
| ChatGPT _is_ GPT-3.5; with specific fine tuning / chat-
| oriented training, and without customer fine-tuning (at
| least, currently) available. It's the particular GPT-3.5
| interface that OpenAI wants people to preferentially use, so
| the price structure artificially encourages this.
| eldenring wrote:
| The ChatGPT API is using a smaller model than the original
| "GPT 3.5". The original, presumably larger, model is used
| by Legacy ChatGPT.
| frognumber wrote:
| If I were to speculate, OpenAI is doing two types of
| optimizations:
|
| 1) Cost
|
| 2) Function
|
| I suspect ChatGPT is 90% a cost-optimized version of GPT3,
| and 10% a function-enhanced one.
|
| GPT3 was the first major public wow-able model, and I suspect
| it was not running on any sort of optimized infrastructure.
| rvnx wrote:
| Didn't the CEO say during the GPT-4 demos that humans were
| involved to do the captions during all the images tests
| (using Be My Eyes volunteers) and that's why it took some
| time to show the Discord examples ?
|
| If so, it could explain some costs indirectly.
| IshKebab wrote:
| Wait did they just use Be My Eyes volunteers as AI
| labellers without telling them? Or were they told? Pretty
| unethical if not!
| [deleted]
| tagawa wrote:
| I think you mean "up to 60" ($0.12 / 1K tokens as opposed to
| $0.002).
| minimaxir wrote:
| For clarification, $0.002 is the ChatGPT API price, not the
| text-davinci-003 price ($0.02).
| tagawa wrote:
| Ah, OK. thanks for pointing that out.
| colesantiago wrote:
| No surprise here, this is Classic Microsoft & OpenAI.
|
| If you know anything about business models this is by design.
|
| The minute that ChatGPT's pricing came close to free, it was
| obvious they had a way better model.
|
| It was never about OpenAI having found a more efficient way to
| reduce costs, it was to price out competitors like Google, Meta,
| Open Source, etc with a "good-enough" breakthrough model.
|
| Then introduce an expensive superior corrected model.
| wongarsu wrote:
| I'm more impressed that it's less than 3 times as expensive as
| Davinci GPT3. In usecases where most tokens are in the context
| even only 50% more expensive, while allowing 4x as much context.
| And you can pay a 100% surcharge to get another fourfold increase
| in context size.
|
| It only looks expensive when compared to the GPT-3.5-Turbo
| ChatGPT offering which is incredibly cheap (or alternatively
| Davinci is overpriced by now)
| tlogan wrote:
| Can you please explain how GPT-4 is cheaper than davinci GPT3.
| Maybe I do not understand the pricing here...
| dogma1138 wrote:
| Designer leather jackets boutiques world wide are rejoicing.
| aiappreciator wrote:
| I remember as a young gamer, asking my Dad whether Nvidia or
| Intel was a bigger company. It seemed obvious to me, since
| people always bragged about their GPUs online, but never the
| CPU, that Nvidia is the more important company.
|
| At that time, Nvidia was worth like 1% of Intel.
|
| Today, regarding online discussions, nothing has changed.
| People still brag about GPUs, never the CPU. But now Nvidia is
| worth 5 times as much as Intel.
| sebzim4500 wrote:
| Why is the 32K context only twice as expensive as the 8K context?
|
| Are they using sparse attention or something? I don't think flash
| attention on it's own can explain it.
|
| EDIT: Oh right, if the cost is per token so if you actually fill
| the context then it is 8x more, which makes much more sense.
| cscheid wrote:
| Ask it the other way around, "Why is the 8K context only twice
| as cheap as the 32K context?" and your original answer is
| clearer: because they think the demand curve supports it.
|
| Price is not determined by cost, but by how much people are
| willing to pay.
| sebzim4500 wrote:
| No I'm just an idiot. Since the price is per token the actual
| ratio is more like 8x more (assuming you fill both contexts)
| which is plausibly the ratio between the costs.
| Maxion wrote:
| In the end the cost for you is how many tokens consumed, if
| you can get more value out of one request with 32k context
| that multiple with lower, then it's cheaper (and faster).
|
| Some problems also require a larger context to be able to
| solve.
| speedgoose wrote:
| The OpenAI pricing models seem to be decided by ChatGPT.
| sebzim4500 wrote:
| How so? They seem reasonable to me.
|
| You have the models which are pretty good that are cheap, and
| the models which are far ahead of the competition which are
| expensive.
| tlogan wrote:
| The title should be changed to "GPT4 is up to 90x more expensive
| than GPT3.5"
| thelittleone wrote:
| Is it conceivable that ChatGPT could review itself and produce a
| superior revision to a human?
| obblekk wrote:
| Worth remembering GPT3 was 10-20 times more expensive 2yrs ago.
| There's a super-moore's law type learning function going on here
| and I suspect in 1yr GPT4.5 will be the same cost as GPT3.5
| today.
|
| It's insane... feels like iPhone 3G to iPhone 4 level quality
| improvement every year.
| behnamoh wrote:
| That's incorrect information. The original GPT3 price was about
| x3 times its current price, not 10-20 times.
|
| https://the-decoder.com/openai-cuts-prices-for-gpt-3-by-two-...
| sebzim4500 wrote:
| I think he means it was 30x more than gpt-3.5 is today.
| stavros wrote:
| Yeah but that's also 30x times worse. ChatGPT is much
| worse-performing than davinci-003.
| avereveard wrote:
| yeah, I used to be able to create valid units for
| WarStuff with original chatgpt, but now all the unit it
| creates are full of hallucinated traits and special rules
| that aren't in the source.
| stavros wrote:
| If you try davinci-003, it will definitely be able to do
| it.
| Maxion wrote:
| GPT-4 is a scary improvement over 3.5, especially for handling
| code. It will be the literal definition of awesome when these
| models get a large enough context space to hold a small-medium
| sized codebase.
|
| I've been playing around with it for an hour seeing what it can
| do to refactor some of the things we have with the most tech
| debt, and it is astounding how well it does with how little
| context I give it.
| int_is_compress wrote:
| There are already some cool projects that help LLM go beyond
| the context window limitation and work with even larger
| codebases like https://github.com/jerryjliu/llama_index and
| https://github.com/hwchase17/langchain.
| Der_Einzige wrote:
| The fundamental techniques that they use are highly lossey
| and are far inferior to ultra-long context length models
| where you can do it all in one prompt. Hate to break it to
| you and all the others.
| nico wrote:
| Where can someone find and try ultra-long context length
| models?
|
| Any links?
| intelVISA wrote:
| The longest one that is generally available is always
| going to be yourself :)
| thelittleone wrote:
| My context model is getting shorter and fuzzier.
| hombre_fatal wrote:
| > Hate to break it to you and all the others.
|
| Jeez. Their comment is quite obviously a complementary
| one in response to the limitation rather than a
| corrective one about the limitation.
| speedgoose wrote:
| The 32k tokens version should do fairly well on such
| codebase. I don't know if it's the one used in ChatGPT.
| Maxion wrote:
| I'm not even kidding when I say I just did 6h or so of work
| in around 45 minutes
| worldsayshi wrote:
| Do you let it refactor your code base? How do you feed
| the code to it? Just copy paste?
| Maxion wrote:
| Yup, it's a bit of a learning curve how to get it to
| output the most useful stuff with the least amount of
| prompts.
|
| Figure out what your goal is, and then start by giving a
| wide context and at first it will give wrong answers due
| to lack of context. With each wrong answer give it some
| more context fro what you think the solution provide is
| missing the most. Eventually you get 100% working code,
| or something so close to it that you can easily and
| quickly finish it yourself.
|
| This is one strategy, but there are many that you can use
| to get it to reduce the burden of refactoring.
| [deleted]
| intelVISA wrote:
| I'm worried it's going to decimate the already wobbly jr
| market - who are the sr devs of tomorrow to wrangle the
| LLM output after it replaces the jr's job of coding all
| day?
| woeirua wrote:
| The only thing holding this back now is lack of enough
| context. That's the big nut to crack. How do you hold enough
| information in memory at once?
| alfalfasprout wrote:
| Keep in mind OpenAI is able to lose money on GPT*. Inference
| for the same predictive power will indeed improve (Facebook's
| recent model is a step in the right direction) but efficiency
| of running on given hardware likely won't follow moore's law
| for long.
| Raidion wrote:
| Are there truely top tier ML optimized chips? I would imagine
| the hardware space still having room to grow if just because
| most of the brainpower having been focused on x86, etc.
| moffkalast wrote:
| There are analog neural net accelerators that can get
| around lots of existing limitations, but still have a few
| of their own that need to be figured out.
| icapybara wrote:
| Well just the other day people were shocked at how cheap GPT3.5
| was.
| jsyolo wrote:
| Can GPT be used for something like feeding it a 2000 page PDF of
| pure text(not english) and ask questions about its contents?
| Kiro wrote:
| This supports up to 2000 pages: https://www.chatpdf.com/
| jimmyechan wrote:
| Not yet, but my bet is that we will be able to in the near
| future. The gpt-4-32k (with a 32K context window) allows for
| about 52 pages of text
| wmf wrote:
| No. In the near future they will support ~50 pages.
| fswd wrote:
| It's $3.80 for the full 1000 prompt tokens plus 32k token long
| context. But OpenAI doesn't have a monopoly on long context, RWKV
| is free and open source and has virtually an unlimited context
| (as long as you remind it every once and a while). However
| ChatGPT really cannot be matched at this point perhaps except
| LLaMa _Alpaca_ 7B
|
| EDIT: I accidentially typed Alpine, it's Alpaca.
| https://github.com/tatsu-lab/stanford_alpaca
| sebzim4500 wrote:
| Alpaca is supposed to be roughly comparable to GPT-3.5, which
| means it is probably far worse than GPT-4.
| stavros wrote:
| But also gpt-3.5-turbo is far worse than davinci-003.
| sebzim4500 wrote:
| By what metric? For most stuff 3.5 is better in my
| experience. Assuming you are trying to complete an actual
| task rather than have fun.
| stavros wrote:
| I've tried to make an assistant with ChatGPT, because
| it's cheaper, and it's abysmally worse than davinci-003
| at following instructions and reasoning.
| artdigital wrote:
| 3.5-turbo is already worse than 3.5-legacy, you can try
| this with chatgpt plus. Use the normal (turbo) model and
| then use the legacy model, you will almost certainly feel
| a difference in quality
|
| Turbo prioritizes speed (probably due to cost?), the
| legacy model has far higher quality of output. You can
| confirm this on Reddit where none of the chatgpt plus
| userbase seems happy with turbo and the general
| recommendation is to switch it back to legacy.
|
| I tried porting a mini project from davinci to the new
| turbo api and quickly reverted it, turbo output is a lot
| more all over the place and very hard to get into
| something useful. It's a fun chatbot though and still
| great for simple tasks
|
| You get what you pay for, and there's a reason 3.5-turbo
| is so cheap
| Metus wrote:
| There is also https://open-assistant.io by LAION.
| Mizza wrote:
| Does anybody have their hands on it? Is it actually 6 times
| better at performing tasks, or are we paying more for improved
| corporate-friendly bumper bowling?
| nonfamous wrote:
| If you've used Bing Chat, you've used GPT-4:
| https://blogs.bing.com/search/march_2023/Confirmed-the-new-B...
| wetpaws wrote:
| _heavily lobotomized version of GPT-4_
| sp332 wrote:
| GPT-4 has a knowledge cutoff from 2001. Bing can talk about
| current events.
| stevanl wrote:
| 2021*
| alchemist1e9 wrote:
| Bing Chat is more useful in many cases because it provides
| references that connect to it's responses. I don't like the
| mobile UI and that it doesn't save your chats for you. Maybe
| I should try the web interface.
| caturopath wrote:
| It's difficult to be sure exactly what six times "better"
| means, but I wouldn't expect something to have to be six times
| better necessarily to be six times better to be six times more
| valuable. I wouldn't expect a pitcher making $30MM/yr to be 6x
| better than one making $5MM/yr, but I could buy that they were
| 6x more valuable to have on your team.
| VWWHFSfQ wrote:
| In my anecodotal experience, GPT-4 is at least a million times
| better than GPT-3.
|
| It's like night and day.
| alchemist1e9 wrote:
| How are you trying it? I'm ChatGPT Plus subscriber but don't
| see any change.
| logical_proof wrote:
| You should have the option when you start a new chat to use
| default or 4
| alchemist1e9 wrote:
| Yes now it gave me that. Not sure why it didn't before.
| simonw wrote:
| Not the API, but if you pay for ChatGPT Plus you can try it out
| there.
|
| It's very, very impressive so far - I've tried a bunch of
| things I've previously run against GPT-3 and got noticeably
| better results - maybe not a 2x or 6x multiple in "quality"
| (depending on how you measure that) but still very clearly
| improved.
|
| Posted a few examples here:
| https://fedi.simonwillison.net/@simon/110022949941148725
| ThrowawayTestr wrote:
| Those coffee puns are surprisingly good.
| Mizza wrote:
| Thanks for the reply. I'm not all that impressed by "what
| is"/information retrieval tasks, is it any better at
| "thought" tasks like explanatory reasoning and ideogensis?
| For instance - "How might an agoraphobic kleptomaniac satisfy
| both of their desires simultaneously?"
| kenjackson wrote:
| That's a tough question even for me. The answer it did give
| me was to steal something online. Which was better than
| anything I was thinking. What did you have in mind?
| Mizza wrote:
| Ah, interesting! It's a tough question and not one that
| has an obvious answer, which is what makes it a good
| test, it requires a little creativity. In my tests,
| ChatGPT/3 can't/won't answer it.
| kenjackson wrote:
| GPT-3.5 for me also didn't answer the question, and also
| gave me a MUCH longer response. I guess score a win for
| GPT-4.
| djbusby wrote:
| Steal from a friends house?
| sebzim4500 wrote:
| You'd still have to go outside to get to said friends
| house.
|
| I think GPT4 gave the better answer here.
| lewdev wrote:
| And website still says "openai."
| newZWhoDis wrote:
| Maybe they mean "open" as in "open your wallet"?
| saberd wrote:
| At last they didn't call it freeai
| icapybara wrote:
| Should they not be able to make money off a world changing new
| technology they bet billions on?
|
| Does your company offer all its products for free?
| JohnFen wrote:
| The problem with the name isn't that they're charging. It's
| that they're implying that they're something in the ballpark
| of open source when they're not.
| djbusby wrote:
| OpenTable isn't Open Source. Open, as a word, has more than
| one context.
| JohnFen wrote:
| But OpenTable was never open source to begin with, so
| that association was never made.
|
| Open AI started with a commitment to being open source
| (thus the "open"), but then they changed their minds and
| went closed source. I think in this context, keeping the
| "Open" in their name is a bit deceptive.
|
| It's not the biggest issue in the world, admittedly, but
| it does leave a bad taste in the mouth.
| djbusby wrote:
| Oh! that's new information for me. That case does feel
| like a -n-switch.
| evilduck wrote:
| OpenTable references restaurants tables and times that
| are still open to be reserved.
|
| What contextual definition of "open" does OpenAI have? A
| website open to subscribers?
| wmf wrote:
| _A website open to subscribers?_
|
| You can scoff at this but Google's models weren't
| available at any price before today.
| icapybara wrote:
| An open API you can build products off of. Consider the
| alternative. They could not let anyone use their models
| and just release AI products (their own search engine,
| their own version of Teams, their own self driving car
| software, their own photoshop competitor, etc).
| jackmott wrote:
| [dead]
| sebzim4500 wrote:
| I think it was borderline defensible for them to not release
| the weights, but now they aren't even telling people the size
| of the model.
|
| From their website:
|
| >Our mission is to ensure that artificial general
| intelligence benefits all of humanity
| OmarAssadi wrote:
| I pay for ChatGPT Plus and have paid a fair bit in OpenAI API
| fees; I don't care that they are making money. The problem is
| the openwashing, not the monetization.
___________________________________________________________________
(page generated 2023-03-14 23:01 UTC)