[HN Gopher] OpenAI's Foundry leaked pricing says a lot
___________________________________________________________________
OpenAI's Foundry leaked pricing says a lot
Author : Michelangelo11
Score : 59 points
Date : 2023-02-28 19:26 UTC (3 hours ago)
(HTM) web link (cognitiverevolution.substack.com)
(TXT) w3m dump (cognitiverevolution.substack.com)
| blibble wrote:
| phoning customer support is already painful when there's a human
| on the end
|
| a bot that gives sometimes random answers seems like a
| particularly cruel form of torture to inflict on your customers
| mc32 wrote:
| With reports[1] (maybe exaggerated, maybe apocryphal) of
| companies replacing FTEs with chatGPT, even at these high prices,
| it may make sense in some use cases, no, tho presumably this
| kills the playpen use cases.
|
| [1]https://it.slashdot.org/story/23/02/27/009234/survey-
| claims-...
| buildbot wrote:
| Nobody is doing this, unless you have a source you would like
| to link?
| celestialcheese wrote:
| Here's one anecdotal source confirming this - specifically
| some copywriting contractors. Significantly less hours needed
| per month writing. A lot more hours for our editors but it's
| still significant cost savings when netted out.
| hudsonjr wrote:
| I think the source was this:
| https://www.resumebuilder.com/1-in-4-companies-have-
| already-...
|
| Now I would ask, what "business leaders" would fill out a
| random survey for a resume company. Additionally there's no
| real meaty info provided.
| buildbot wrote:
| Yeah, not sure I would trust that data...
| mc32 wrote:
| According to this survey:
| https://it.slashdot.org/story/23/02/27/009234/survey-
| claims-...
|
| "Earlier this month, job advice platform Resumebuilder.com
| surveyed 1,000 business leaders who either use or plan to use
| ChatGPT. It found that nearly half of their companies have
| implemented the chatbot. And roughly half of this cohort say
| ChatGPT has already replaced workers at their companies...."
|
| There are anecdotes elsewhere as well.
| seydor wrote:
| It looks like OpenAI's first mover advantage won't last long,
| considering the speed at which these models are improving and
| compacting themselves. Seems like this new 'Moore's law' will be
| even more steep, at least in the beginning . So steep, that we
| can hope to be running these in our desktops instead of on
| someone else's computer.
| sharemywin wrote:
| but what kind of model will run in the cloud?
| eppp wrote:
| We could call it a weather model
| interestica wrote:
| A model that augments the local? Runs above it? A supermodel?
| Tepix wrote:
| I don't think Moore's law is the most important factor, instead
| algorithmic improvements will enable to have smaller models
| that are as capable as these humongeous models.
|
| Llama 65b is hopefully just the beginning of this trend. It
| outperforms OPT-175b.
| buildbot wrote:
| That holds true if scaling laws don't hold true, otherwise a
| hypothetical Llama 175b would be even better. So the high end
| will always be on big clusters.
| [deleted]
| dragonwriter wrote:
| > This really should not be a surprise, because even the
| standard-issue ChatGPT can pass the Bar Exam
|
| No, it can't.
|
| The two things that together have sometimes gotten misrepresented
| that way in "game of telephone" presentations are:
|
| (1) that when tested on the multiple choice component of the
| multistate bar exam (not the whole bar exam), it got passing
| grades in two subjects (evidence and torts), not the whole
| multiple choice section; which is very much _not_ the same thing
| as being able to pass the exam, and 50.3% overall (better than
| chance, since its four choices per question, but also very much
| not passing.)
| https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839
|
| (2) that a set of law professors gave it the exams for four
| individual law courses and it got an average of C+ scores on
| those exams, which are minimally-passing grades (but not on the
| bar exam.) https://www.reuters.com/legal/transactional/chatgpt-
| passes-l...
| version_five wrote:
| I know this isnt the same, but:
|
| I've long had it on my list of things to try to train a
| classifier to predict the answer to multiple choice tests based
| on embedding of the questions. Many tests I've seen don't
| require actual intelligence to pass, just a plausible answer
| relative to the question phrasing
| tshaddox wrote:
| A lot of multiple choice tests are only there to provide some
| minimum bar of plausible deniability for the examiner. The
| multiple choice section of a driving test (at least in the
| two U.S. states I've taken them) is a great example. The
| questions are almost entirely of the form:
|
| _Which of these activities is legally permitted while
| operating an automobile?_
|
| _A. Wearing your seatbelt._
|
| _B. Being intoxicated._
|
| _C. Driving 120 miles per hour._
|
| _D. Intentionally colliding with pedestrians._
|
| That way, when someone drives drunk, it clearly isn't the
| fault of the examiner, because they clearly verified that the
| person knew that was illegal! (Or at least, if they got that
| question wrong, they got enough other ones correct.)
| klabb3 wrote:
| Correct me if I'm wrong but aren't virtually all tests and
| exams designed to minimize ambiguity, make them fair or easy to
| grade and questions are designed to have a clear correct
| answer? This is a stark difference to most real-world human
| activity.
|
| Add to the fact that LLMs perform _much better_ on questions
| with a lot of training data.
|
| And also add the hallucinations or more generally: they don't
| ask for help or admit they don't know, they seem unaware of
| their own confidence levels.
|
| No doubt GPT is fascinating and exciting, but boy we're
| oversubscribing their abilities. LLMs are even worse than
| crypto (from a fad perspective) because we naturally
| anthropomorphize their higher level abilities, which are
| emergent and not well understood even by experts. And we're
| about to plug them straight into critical business flows? Bring
| the popcorn!
| whatshisface wrote:
| > _Add to the fact that LLMs perform much better on questions
| with a lot of training data._
|
| The answer key can get a 100% on the exam.
| ccakes wrote:
| > Streamline hiring - in such a hot market, personalizing
| outreach, assessing resumes, summarizing & flagging profiles, and
| suggesting interview questions. For companies who have an
| overabundance of candidates, perhaps even conducting initial
| interviews?
|
| That's a hiring red flag if I've ever seen one. The nightmare
| dystopia is just around the corner it seems.
| beambot wrote:
| Just have to jailbreak DAN to force the system to offer you the
| job...
|
| Modern day Goodwill Hunting interview experience. Retainer!
| 3throwaway3141 wrote:
| Dystopia? I think of it as a great comedy! We could be only
| months away from the first-ever hire where both the applicant
| and the company have automated the entire process and may not
| even realize there's been an accepted offer.
| sourcecodeplz wrote:
| Such a great read. Nothing eye opening really but well put and
| tied together.
|
| At this point any OpenAI leak is THE leak.
| Imnimo wrote:
| >Handle customer interactions - natural language Q&A, appointment
| setting, account management, and even tech support, available
| 24/7, pick up right where you left off, and switch from text to
| voice as needed. Customer service and experience will improve
| dramatically. For example, Microsoft will let companies create
| their own custom versions of ChatGPT -- read here.
|
| If I can prompt-hack your ChatGPT customer service agent into
| giving me a discount or accepting an otherwise invalid return or
| giving me an appointment at 3AM, how binding is that? And if the
| answer is "obviously it's not binding!", why should I trust
| anything else your bot tells me?
| mike_hearn wrote:
| That would presumably be treated the same as a human CSA doing
| the same thing.
| ianmcgowan wrote:
| Presumably the same is true for social engineering people - the
| original "prompt hacking"? I've had customer service
| interactions that didn't go the way I wanted, hung up, and
| tried again with a friendlier person. Not sure how "binding"
| most customer service promises are either - have you ever had
| to call Comcast? Broken promises are the only promises.
|
| Also, if you can lower your customer service costs by 90%,
| maybe accepting some prompt hacking that erodes margins a
| little is a good tradeoff?
| ilaksh wrote:
| The Foundry pricing, $78,000 for a few months minimum, is
| absolutely the opposite of Open. It could completely kill my
| small startup if the API goes from less than a penny per request
| to requiring a huge up-front investment. It means that anyone
| bootstrapping is now locked out.
| candiddevmike wrote:
| That's the risk you take when you create irreplaceable
| dependencies.
| tikkun wrote:
| They only require an up front payment if you're renting a
| dedicated instance.
| wmf wrote:
| Yeah but will "DV" be available any other way?
| riskneutral wrote:
| Dedicated instances (and high price tags, like $1.5 million
| per year) are necessary for large corporations to feel safe
| about sending their proprietary/private data to OpenAI, and
| to have performance/availablity SLAs in place if/when they
| start depending on OpenAI for critical workloads. Right now
| many companies are blocking access to OpenAI entirely because
| of the data privacy issues. I wouldn't be surprised if this
| "leak" is intentional and a way of getting feedback from the
| market on proposed product/pricing. I also wouldn't be
| surprised if some larger customers are knocking on OpenAI's
| door and demanding to run OpenAI's models on their own
| infrastructure to avoid sending any data outside their
| network.
| Nifty3929 wrote:
| $78k is very small, even for bootstrapped companies. Only the
| very smallest shops would be inhibited by that. And I admit,
| that's a lot of them! I get it. It sucks to be a solo developer
| who wants to get involved and doesn't want to spend a bunch of
| cash.
|
| But still, for even small players $250k annually just isn't
| that much.
|
| What we might hope for are trimmed down, subsidized options for
| small shops who want to get started, with the hope to upgrade
| them to the full option when they are ready.
|
| In the meanwhile, you might consider partnering up with other
| small startups.
|
| Also, "open" doesn't mean free. Things take a lot of effort to
| build and maintain, and that effort needs to be accounted for
| somewhere.
| eindiran wrote:
| > "open" doesn't mean free
|
| Yeah but none of the other definitions of "open" apply to
| OpenAI either.
| tiagod wrote:
| Maybe Microsoft should change their name as well?...
| dragonwriter wrote:
| The "open for business" sense certainly does.
| layer8 wrote:
| That's the general risk of basing a startup on someone else's
| service without having a solid contractual agreement, though.
| It's always a gamble, in particular when the underlying service
| is an entirely new type of business.
| xwdv wrote:
| Good.
|
| I'm tired of fly by night tech bros flooding the markets with
| shitty AI business ideas they learned about on Youtube. Pay to
| play. Take a risk. Like a _real_ entrepreneur.
|
| I hope the price goes even higher.
| inawarminister wrote:
| It's time to replicate OpenAI services with a real Open Source
| (or at least source-available, commercial-allowed) licenses.
|
| It's time to make our own Linux. Our own Emacs.
|
| Go to open-assistant.io and other similar initiatives.
|
| Unfortunately GPT-NEOX, LLaMA, and OPT-IML are non-commercial-
| only. We small scale players should make our own.
|
| Please contact me if anyone is interested in this space.
| gjsman-1000 wrote:
| So much for "Open" AI.
|
| Open-source algorithm? No.
|
| Open access for free, or reasonable prices? No.
|
| Open emails and conversations of directors (accountability)? No.
|
| Open conversations of issues and ethics? No.
| moffkalast wrote:
| Open your wallet? Yes.
| HDThoreaun wrote:
| Open access for a fee? Yes.
|
| The original point of openAI was to make sure google and
| facebook don't completely dominate AI and keep their work
| hidden due to the innovator's dilemma they face. Seems like
| they've absolutely accomplished that. Yes, they've stretched
| the meaning of "open" but I don't think they've ever tried to
| claim they had some sort of open source aspirations.
| rnd0 wrote:
| Thank fuck they weren't called "HomelandAI" or "LibertyAI" or
| somesuch!
| all2 wrote:
| I smell a business opportunity. "HomelandAI" sounds rather
| nationalist.
| honkler wrote:
| well atleast now they are Open about their intentions
|
| :)
___________________________________________________________________
(page generated 2023-02-28 23:01 UTC)