[HN Gopher] Improvements to the fine-tuning API and expanding ou...
___________________________________________________________________
Improvements to the fine-tuning API and expanding our custom models
program
Author : Josely
Score : 50 points
Date : 2024-04-04 14:58 UTC (8 hours ago)
(HTM) web link (openai.com)
(TXT) w3m dump (openai.com)
| spxneo wrote:
| Did you hear that?
|
| It's the sound of another opencore AI startup unknowingly
| participating in NVDA arbitrage becoming irrelevant overnight.
| minimaxir wrote:
| > As a result, Indeed was able to improve cost and latency by
| reducing the number of tokens in prompt by 80%.
|
| There's some exact words shenanigans here. Indeed may have
| reduced the _number of tokens_ in the prompt by 80%, but they
| didn 't reduce the _cost_ by 80%: the prompt cost of inferring
| from a fine-tuned GPT-3.5-turbo ($3.00 / 1M tokens) is 6x the
| prompt cost of inferring from the base GPT-3.5-turbo ($0.50 / 1M
| tokens). If prompt tokens are cut to 20% of normal, then that
| implies the overall cost of the prompt tokens is _120%_ relative
| to their normal prompt: a cost increase! That 's not even getting
| into the 4x cost of the completion tokens for a finetuned model.
|
| Of course, Indeed likely has an enterprise contract reducing
| costs further.
| spxneo wrote:
| The SK telecom one is highly suspect too. My guess is we are
| not going to see ChatGPT-5 until the AI bubble begins to
| deflate.
|
| My question is will it be before or after US elections?
| leosanchez wrote:
| I remember reading somewhere GPT5 will be released in summer
| this year
| throwaway74432 wrote:
| Number of tokens is still a useful metric, as their endpoints
| have Tokens Per Minute quotas. Decreasing number of tokens used
| means increasing throughput, up until the Request Per Minute
| quota.
| afro88 wrote:
| But the sentence was "Indeed was able to improve cost..."
| mechagodzilla wrote:
| From OpenAI's perspective the cost improved!
| throwaway74432 wrote:
| "...and latency" Higher throughput = lower latency*
|
| *Under certain conditions
| Randalthorro wrote:
| Indeed was going to use a fine tuned endpoint on their
| enterprise contract regardless..
| IanCal wrote:
| It may simply be a timing thing. 3.5-turbo saw a price drop
| between the launch of fine tuning and now.
| Workaccount2 wrote:
| Can't wait till I have to pay a lawyer $500/hr to turn around and
| ask a bot what to do.
|
| Given the disproportionate amount of lawyers in any given
| legislature, never mind the make-up of judges, I can only imagine
| that a legal stranglehold insulating lawyers from the threat of
| AI will be one of, if not the, first bit of AI protectionist
| legislation.
| visarga wrote:
| Computers can never be responsible or accountable for their
| actions. AI is like the genie from the lamp, it will grant your
| 3 wishes and they will turn out weird. And you can't do
| anything to it. What can you do, it has no body. Only a human
| has skin in the game.
| LunaSea wrote:
| That's great because lawyers can't either.
|
| You can't retry a case if you had a bad lawyer, only if
| procedural mistakes were made.
| anon373839 wrote:
| Trust me, existing LLMs are nowhere near being able to provide
| that kind of legal analysis. With the diminishing rate of
| improvement in the frontier models that we've seen, I'm also
| doubtful this particular technology is on a path to get there.
| Its_Padar wrote:
| OpenAI are pretty much the most dominant LLM company at the
| moment.
|
| I just wish they let you make new accounts after deleting an old
| account, apparently it's something to do with the database when I
| ask them about it, which suggests that they store data about
| users even after someone deletes their account.
| mirekrusin wrote:
| ...seriously, that's your only wish?
| benreesman wrote:
| These people can't just lay low for even a few days. It's been
| one reminder after another that for some almost certainly bad
| reason, OpenAI wants everyone to remember that some of the most
| scandalous revelations with a _mountain_ of even more damning and
| extremely credible allegations hot on their heels is a new record
| for SV insanity, at least since the early 90s that I've been
| watching.
|
| The CEO was (up to a credible consensus of primary sources)
| _fired for self-dealing in borderline criminal fraud-level ways_.
| I'm not aware of anyone disputing this with any vigor. Then that
| CEO was _fired again_ for more vague but thematically consistent
| allegations of flagrantly profiteering off an ostensible charity
| actively lobbying for de-facto regulator status. Then that CEO
| made a token concession by disposing of yet another in this
| cavalcade of personal financial gain presented as urgent
| imperative.
|
| And the most moderating influence is a guy who has been run out
| of everything from politics to finance to prestigious academic
| posts over scandal after scandal in print for 35 years running,
| and has yet to issue an apology I'm aware of.
|
| But the really high thread-count noose is that Opus is mopping
| the floor with GPT-4-1106 and 0125, on money that would fall out
| of Nadella's couch cushions. And it's dramatically operator-
| aligned.
|
| You can't be a society-scale criminal, keep it up for over a
| decade, _and_ get lapped by vastly less-resourced competitors in
| the Valley. You shouldn't be able to do any of that and still
| wield any power.
|
| This new breed of tech titans have forgotten something their
| predecessors knew: it took Wall Street a century to utterly
| capture its regulators.
|
| Tech people who go big enough? Holmes is serving time, SBF is
| serving time, they're not the first.
|
| This isn't about corrupting the competitive landscape anymore
| geniuses: this is about staying out of prison.
| visarga wrote:
| > Opus is mopping the floor with GPT 4 1106 and 0125
|
| True. I only hop on GPT-4 when I finish my quota for Claude. I
| much prefer Claude's long form writing.
| dannyw wrote:
| There are some serious allegations here. Do you have links that
| recap / substantiate this in more detail?
| benreesman wrote:
| Sure.
|
| https://www.newyorker.com/magazine/2016/10/10/sam-altmans-
| ma...
|
| https://en.m.wikipedia.org/wiki/Lawrence_Summers
|
| https://en.m.wikipedia.org/wiki/Summers_memo
|
| https://www.pbs.org/wgbh/frontline/documentary/warning/
|
| https://en.m.wikipedia.org/wiki/Loopt
|
| https://en.m.wikipedia.org/wiki/Socialcam
|
| https://www.wsj.com/tech/sec-investigating-whether-openai-
| in...
|
| https://www.businessinsider.com/sam-altman-hype-openai-
| silic...
|
| https://archive.is/2023.12.02-101904/https://www.washingtonp.
| ..
|
| https://www.reuters.com/technology/openai-ceo-sam-altman-
| ste...
|
| https://qz.com/sam-altman-openai-fund-1851380019
|
| https://fortune.com/2023/09/20/instacart-ipo-stock-shares-
| ce...
| mewpmewp2 wrote:
| Could you have GPT summarize all those links please? And
| into 3 sentences max ELI5 please.
| benreesman wrote:
| I'll do it myself.
|
| Altman/Loopt, Siebel/Socialcam, Emmett/Twitch failed and
| got rich. The people who trusted them lost a lot. People
| still say they're smart.
|
| Larry said we should put as much poison in Africa as we
| can. He wrote it down. He tried to get someone fired, by
| ratting out his friends. He made bankers angry. They
| never forgave him. So he moved. Also, Brooksley was going
| to save America. Larry said she was a woman and be quiet.
| Also stealing is ok.
|
| Bankers don't go to prison. They can do anything. But
| it's not because they're rich. It's because they know
| important people. Also rich.
|
| Creepy tech people aren't like bankers. They don't get in
| trouble much. But if they're bad enough, they go to jail.
|
| Someone did the worst thing in a long time. They've done
| it a lot.
|
| He should say sorry and get a different job, because jail
| sucks.
| raincole wrote:
| _A little bit_ hard to see how this is relevant to a
| thread about _Introducing improvements to the fine-tuning
| API and expanding our custom models program_.
| mirekrusin wrote:
| "There are a variety of techniques that developers can use to
| (...) reduce costs."
|
| they forgot to mention using open models.
| kaycebasques wrote:
| Kind of a nothing burger announcement for me. Doesn't seem to be
| anything here that would noticeably change how I'm approaching my
| fine-tuning projects.
| john_allard wrote:
| Is there anything specific you'd like to see? What would be the
| highest value improvements we could make to our service in your
| eyes?
| msp26 wrote:
| Friendly notice for anyone optimising their openai token usage:
| multi_tool use wastes 200 tokens on every call with function
| calling. You're better off implementing function calling yourself
| than using their API implementation.
___________________________________________________________________
(page generated 2024-04-04 23:00 UTC)