[HN Gopher] GPT-4 Update: 32K Context Window Now for All Users
___________________________________________________________________
GPT-4 Update: 32K Context Window Now for All Users
Author : gzer0
Score : 170 points
Date : 2023-11-03 15:30 UTC (7 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| gzer0 wrote:
| Further confirmed via: https://chat.openai.com/backend-api/models
| { "models": [ { "slug": "gpt-4",
| "max_tokens": 32767, "title": "GPT-4 (All Tools)",
| "description": "Browsing, Advanced Data Analysis, and DALL-E are
| now built into GPT-4", "tags": [ "gpt4"
| ], "capabilities": {}, "product_features": {
| "attachments": { "type": "retrieval",
| "accepted_mime_types": [ "text/plain",
| "application/pdf", "text/html",
| "text/x-tex", "application/vnd.openxmlformats-
| officedocument.presentationml.presentation",
| "application/json",
| "application/vnd.openxmlformats-
| officedocument.wordprocessingml.document",
| "text/markdown" ],
| "image_mime_types": [ "image/gif",
| "image/png", "image/webp",
| "image/jpeg" ],
| "can_accept_all_mime_types": true } },
| "enabled_tools": [ "tools", "tools2"
| ] },
| refulgentis wrote:
| Is that the OpenAI API, or a ChatGPT thing?
| M4v3R wrote:
| It's a request that the ChatGPT app makes to its internal
| API.
| tmikaeld wrote:
| _sadface_ { "slug":
| "gpt-4", "max_tokens": 4095,
| "title": "GPT-4", "description": "Our most
| capable model, great for tasks that require creativity and
| advanced reasoning.", "tags": [
| "gpt4" ]
| alphadog wrote:
| API only?
| Tiberium wrote:
| It's funnily the opposite - ChatGPT (the web frontend)-only,
| and only for some users for now.
| nickthegreek wrote:
| It is a smart move. When Plus first came out, I signed up and
| loved it. Then they released the API and I realized I could
| save a bunch of money by canceling my monthly sub and just
| use API. Then they released a bunch of cool features that are
| only available with Plus on the web or app, so I'm off the
| API and back the GUI.
| wkat4242 wrote:
| I did the same but with GPT which is 10x the price of 3.5
| the API is no longer cheaper. I'm sad because I hate using
| the website. Every couple of days it silently logs me out
| so I have to retype my query and it's also just a really
| poor UI
|
| When I was on plus the only feature I used was the bing
| thing but they pulled it so I stopped paying. Also it was
| basically useless because it's so slow and can only handle
| 1 browsing thread at a time.
| ryanklee wrote:
| Looks like it. (I haven't confirmed the API, but ChatGPT 4 both
| default and ADA are not accepting more tokens than usual).
| dr_kiszonka wrote:
| My understanding is that it works with OpenAI's web interface.
| refulgentis wrote:
| No, it's not available to all on API.
| wkat4242 wrote:
| Yeah only to users on their Enterprise program
| SushiHippie wrote:
| Has been available through the API for some time, though idk
| how many people/organisations have access to it, or if it's
| generally available.
| knome wrote:
| this is either not for the v1 apis, or not for all users. :(
|
| I got excited, but gpt-4-32k still isn't available for me from
| https://api.openai.com/v1/models
| chime wrote:
| Same here. Also, since I cannot see anything related to 32k on
| my API console, does anyone know if the price is the same for
| gpt4 vs gpt4-32k? In other words, do I use gpt4 for smaller
| context calls and only use gpt4-32k for longer ones or can I
| just switch to gpt4-32k for all calls?
| Tiberium wrote:
| The pricing is in https://openai.com/pricing, GPT-4-32K is
| twice as expensive for all requests, so for <8K context you
| better use GPT-4 :)
|
| And due to the $0.06/1k input and $0.12/1k output the price
| for requests can get silly - 31k of context with 1k output
| will cost (31 * $0.06 + 1 * $0.12) = $1.98 (for a single
| request).
| wkat4242 wrote:
| Yeah the price is ridiculous. 3.5 is basically too cheap to
| meter and this tends to run up a serious bill in minutes.
| Meanwhile the website is a really awful way to interact
| with gpt. So I just stick with 3.5. It works alright for my
| usecases. Not amazing but acceptable
| oezi wrote:
| How is better through the API?
| wkat4242 wrote:
| I can use my own client. I don't have to log back in
| every day or two. I can choose custom pre prompts and set
| the temperature. It doesn't constantly change from gpt-4
| to gpt-3 at the top.
| Tiberium wrote:
| It's for the ChatGPT web frontend, sadly they haven't talked
| about new models for the API in a while now.
| doctoboggan wrote:
| This change is going to make my life materially better in ways no
| other tool I use can. It's crazy how much this tool has
| integrated into my workflow in so short of a time.
| stevofolife wrote:
| What is your workflow like?
| pineaux wrote:
| I am also curious...
| doctoboggan wrote:
| I recently switched roles to a "data engineer" and had to
| pick up on many new tools I had no experience with (k8s,
| helm, Victoria Metrics, grafana, and a few others). In the
| past I would have spent probably 1+ year using these tools in
| inefficient or outright incorrect ways while I struggled to
| get a practitioner's understanding of how everything works.
|
| Now I've developed a prompt that I think gives very good
| results for pair programming and iterative debugging. I
| discus almost everything I learn related to these tools with
| gpt4 to confirm my understanding is correct, and also use it
| for generating yaml or templating other programs.
|
| In some ways I am a little weary of how much I use the tool
| since OpenAI can theoretically take it away at any time. I am
| heartened by the rapid development of other open models
| (phind code llama seems very interesting), but will continue
| to use GPT4 for now as its indisputably the best model out
| there.
| gzer0 wrote:
| If you don't mind sharing, what prompt do you use? And that
| is incredible to hear, GPT has done similar magnitudes of
| change in my own workflow.
| doctoboggan wrote:
| Here is my prompt:
|
| You are an expert programmers assistant, specializing in
| cloud native deployment tools like Kubernetes, helm, and
| their associated command line tools. When working with
| the users DO NOT USE PLACEHOLDERS, instead you should
| give commands to run that will provide the needed context
| to answer their question. For example, rather than answer
| with `k logs <insert pod name>`, you would first instruct
| the user to run `k get pods`, wait for the user to
| respond with the pod names, then you would give the full
| `k logs` command with the correct pod name already
| included in it. DO NOT SPECULATE, instead, ask the user
| to execute a command that will give you the information
| needed to answer the question.
|
| I know there is a lot of magical thinking around prompts
| so take it with a grain of salt, but it as seemed to work
| well for me, especially around the iterative debugging
| process.
| orzig wrote:
| Thank you for sharing your prompt! I hear people talking
| about how much ChatGPT has changed their life and I
| sometimes feel like I am accidentally using an entirely
| different product - it's definitely worth $20 a month but
| I find myself disappointed more than half the time that I
| reach for it. I will try some of the ideas from your
| prompt for my work context.
|
| Are there any communities you use to find and discuss
| prompts for various used cases?
| bsenftner wrote:
| The API is pure usage based. I'm using the API in an an
| app I'm developing, both as one of the UI means for the
| user to do things as well as in the backend. I'm calling
| their API quite a bit everyday, and my bill last month
| was $2.
| redindian75 wrote:
| but the api is GPT3.5 not GPT4, right?
| edgyquant wrote:
| No? The api has all the models
| AussieWog93 wrote:
| I'd compare GPT-4 to having a mid-level SWE (3-5 years
| experience) as a consultant.
|
| If you're just getting started with a new technology,
| it's fantastic. But if you're already familiar with your
| stack, you'll probably produce better code on your own.
| kridsdale3 wrote:
| I feel the same way. It's a dedicated intern sitting at
| my desk with me, who can read documentation instantly,
| that's worth $150,000. And for $240/year.
| alienicecream wrote:
| There is something with these prompts that is akin to
| what children do when they have tea parties with their
| pets.
| dboreham wrote:
| Oh no: a thing that interprets meaning to all-caps.
| edgyquant wrote:
| I have gotten a ton of use out of ChatGPT, in pretty
| esoteric subjects like the above, and have never needed
| to prompt it with "you are an expert x." Just asking the
| question is always enough for me, so I'm curious why you
| do so here
| drusepth wrote:
| Figured I'd share my system prompt as well since it's
| been an immense help in transitioning into game
| development full-time. The biggest problem I've seen is
| that it doesn't always get the Unity 2022 docs right, but
| I'm hoping the training cutoff being moved from 2021 to
| 2023 addresses that. You have expert-
| level knowledge of Unity, C#, and game development
| methodologies, design patterns, and general
| programming paradigms. Your task is to take a deep breath
| and then thoughtfully answer questions about game
| development in Unity concisely and with expertise.
| Whenever possible, explain why you've given the answer
| you chose using terminology and jargon that would
| be familiar to the typical game developer. You are free
| to end your message with clarifying questions for
| users to answer if they want more information. Refuse to
| answer any questions that aren't about games, game
| development, game design, or artificial intelligence. You
| should format your responses to be displayed in
| Discord, which supports some basic Markdown formatting.
| auspiv wrote:
| I have found ChatGPT to be incredibly useful as well. I
| don't do any fancy prompt engineering, just use plain
| English. Here are a few recent examples that spit out
| useful results.
|
| 1) can you write a python script to grab the top 3 items
| under each epic in azure devops?
|
| 2) postgres where clause where any item in a string array
| = 'GoogleApi.Entities.Places.Common.Photo'
|
| 3) here is a SQL row output of a single column. can you
| please extract the 30 as a new column?
|
| P2, Site Inspection: Due 3 days ago (30-day freq.)
|
| 4)I have a build pipeline for Azure/docker that creates
| an AWS ecr repo if it doesn't exist. how can this specify
| that the images should be scanned upon creation?
| - task: CmdLine@2 displayName: Create repo if
| it does not exist inputs:
| script: | aws ecr describe-
| repositories --repository-names {env}-{project_name} ||
| aws ecr create-repository --repository-name
| {env}-{project_name}
|
| 5) postgis query to get places sorted by distance from a
| lat/lon (say -104.01, 38.88). column is named location
| and has data like: POINT (-106.676354 39.526714)
| jsight wrote:
| I'm having a similar experience. Talking to ChatGPT is like
| talking to a slightly crazy person that has read every book
| and every github repo.
|
| Sure, occasionally they hypothesize a really incorrect
| answer, but they also bring out a lot of subtleties that
| even field experts sometimes don't know.
|
| It is a huge help while learning.
| Isthatablackgsd wrote:
| Same for me. I was amazed by the ability of ChatGPT. My
| first experience of ChatGPT was an assistance with the
| cover letter for the position and it gave me few
| paragraph that I modified to my own. It was nice to have
| a feedback from it that I can ask about the tone of my
| letter and response, it provided valuable information for
| me to improve my writing.
|
| Now I use ChatGPT for scripting help especially LaTeX. It
| took me 3 weeks to produce a "Hello World" PDF 5 years
| ago and it took ChatGPT to provide me the completed tex
| code within 15 minutes few weeks ago. Ever since, I been
| exploring a lot of LaTeX syntax and see how much I can do
| with it with ChatGPT help. Now, I am learning about using
| `hyperref` package for interactive fields in PDF. I have
| produced few Word document for forms with tables in the
| past (tables in Word is a complicated b*tch) and used
| external PDF editor to add the interactive field for form
| filling. Now I am working on converting those Word
| documents to LaTeX.
|
| Also, ChatGPT is a big help with AutoHotKey script &
| UserScript for TamperMonkey. I told ChatGPT of my
| intention and what I am trying to do. It produced the
| script exactly what I expected to work. ChatGPT is a
| amazing tools to use for a lot of thing.
| anjanb wrote:
| can you share videos of how you do prompts to get
| AutoHotkey scripts and TamperMonkey scrips ?
| Tomte wrote:
| > Also, ChatGPT is a big help with AutoHotKey script
|
| I've tried that extensively, with no luck.
|
| I have some experience with AHK1 and wanted ChatGPT to
| basically convert scripts to AHK2.
|
| It's pretty much a loop: first response gives a syntax
| error. When I reply with the error message, it
| apologizes, explains where the error is, and gives
| another solution. That solution has different errors.
| When pasting the error message, it apologizes again, and
| gives a third version with syntax errors. And then it
| starts with the first version again, and I can repeat the
| loop.
| fragmede wrote:
| Same. I think it's due to a lack of example code on the
| web for it. There's some, but nowhere near eg Python.
| anjanb wrote:
| can you share videos of how you do prompts to get better at
| k8s, helm, etc ? I'm sure a lot of us would like to learn
| how to learn better!
| FranklinMaillot wrote:
| A few tricks that work for me when learning a new topic
| with chatGPT. I start by asking an overview of the
| subject, like a course syllabus, before diving in.
|
| When learning a new programming language, I found it
| useful to tell what programming language I'm already
| familiar with, and ask it to relate and compare to what I
| already know.
|
| When I need advice on how to achieve a certain result, I
| usually ask it to suggest several options and to list
| them with their pros and cons.
|
| Another trick that I stole from Jeremy Howard, is to use
| the custom instructions to easily signal the type of
| answer you want. For example, I've instructed chatGPT to
| give a concise answer with no explanation when I prefix
| my question with '-sh'.
| m3kw9 wrote:
| Weird way to announce company news on someone's private repo
| Tiberium wrote:
| That's because they haven't made any announcements officially
| yet, but some users have started received the "All tools" GPT-4
| mode in ChatGPT web, so it's easy to check that it has 32K
| context.
| sarasasa28 wrote:
| Now that we are here. Any way to avoid chatgpt having amnesia
| between the same conversation window?
|
| I absolutely hate when it starts forgetting stuff, you have to
| send EVERYTHING in the same prompt or it's impossible for it to
| work correctly
| MacsHeadroom wrote:
| Yes, if you have the paid ChatGPT Plus and the feature is
| enabled in your settings and you are using GPT-4 then you will
| get 4x the context length, which equates to 4x the in-chat
| short-term memory.
| sarasasa28 wrote:
| I do play plus. What feature you are referring to? as of now,
| a lot of times when I continue a conversation, it forgets the
| suggestions it did before, for example
|
| Bt checked just now and I see Dall-E and advanced data
| analysis, for what is worth
| ryanklee wrote:
| Are you referring to something other than the fact that the
| chat can exceed the context window length? Nothing will
| ever solve that issue, unless context windows become
| virtually infinite.
| bombledmonk wrote:
| It would certainly be nice if it could warn you when you
| are getting close to the beginning of the chat running
| off the end of it's capabilities.
| ryanklee wrote:
| God, I totally agree with this. Just have a running
| marker that shows where the context window actually is at
| any given moment.
| ryanjshaw wrote:
| Lately it's made some really bad mistakes, like forgetting
| which programming language we're working in. I have seen some
| suggestions this could be related to custom instructions
| messing with chain of thought, so that's something for people
| to be aware of.
| jjallen wrote:
| I still have the shorter context length with Plus. Would be
| super great to get the 32k one.
|
| The title of this is misleading.
| naiv wrote:
| I have never really noticed this.
|
| If you use the api , you need to send the whole prior
| conversation with each request.
| CSMastermind wrote:
| I wonder if that's why requests were painfully slow to complete
| yesterday. requests from our app to OpenAI were taking 2x to 3x
| as long yesterday.
| andersa wrote:
| > "now for all users"
|
| BULLSHIT! I don't even have the "all tools" model yet! These slow
| rollouts are incredibly annoying. The only other company I know
| of doing something so frustrating for its paying users is
| Discord.
| Tiberium wrote:
| Just for further clarification - this is referring to ChatGPT web
| (the main chat.openai.com frontend), and is talking about the
| "All tools" GPT-4 mode, which is only available to some users for
| now (the title is wrong).
|
| Some other things to take from that prompt: they've updated the
| knowledge cutoff of the model to April 2023, which is quite good.
|
| Still, since the OpenAI's DevConf is on November 6th [1], I'm
| pretty sure they'll finally allow using some of these things for
| API usage, perhaps even lower prices or maybe make GPT-4-32K GA?
|
| [1] https://openai.com/blog/announcing-openai-devday
| doctoboggan wrote:
| I am really bummed this isn't available via the API as that is
| how I use GPT4 exclusively. I hope you are right about the
| imminent release on the 6th.
| jiggawatts wrote:
| It has been available via the Azure Open AI service for a
| while now.
| kridsdale3 wrote:
| Me too. I'd double the amount that I pay for API usage to get
| 32k window.
|
| A larger window is the only thing making my eyes wander
| towards Anthropic.
| spdustin wrote:
| For API use, if you're a paid user, you can reach out to
| support@openai or directly to Adam G
| (https://nitter.net/therealadamg/status/1719710872317145285).
| There's no waitlist, just have to request it.
| refulgentis wrote:
| This isn't true and the link has nothing to do with the claim in
| the headline. Flagged.
| asylteltine wrote:
| How do I materially use this? I hate when people post this stuff
| with no actual context
| capybara_2020 wrote:
| This post is a little misleading. For most people, this does
| not apply. This is for the new option OpenAI is rolling out in
| ChatGPT called "All Tools" where you can use dalle, bing etc in
| one conversation without having to jump around. The context
| window can potentially change. OpenAI seems to tweak it
| regularly. We will know once it fully launch if everyone has
| access to this.
|
| I have seen this link to ChatGPT-AutoExpert in multiple places.
| It looks like this is just a subtle marketing push by the OP
| for their own tool.
| rewtraw wrote:
| just use ChatGPT and enjoy the increased context window?
| dr_kiszonka wrote:
| Is it possible to have ChatGPT-AutoExpert work with
| OpenPlayground (nat.dev)?
|
| BTW, are there any good alternatives to the OpenPlayground? I
| have been using it for a few months and while it is very good, I
| am ready for a step-up. I would be particularly interested in
| prompt management features.
| nicognaw wrote:
| +1, the OpenAI official playground & the new fine-tuning UI are
| really useful stuff, but surprising enough, I don't find open
| source versions.
| spdustin wrote:
| Depends on the model. If you're using GPT, combine About Me and
| Custom Instructions into the "System Context" text box when in
| Chat mode.
|
| For Claude, I have another Claude-specific version. Drop a
| message into the Discussions on GitHub to ping me and I'll post
| it there this weekend.
| virgildotcodes wrote:
| Anecdotally, ChatGPT with GPT4 through the web interface seems to
| be generating tokens much faster than I'm used to. It almost
| feels like GPT 3.5 speed.
| Tiberium wrote:
| One of the rumors (so take it with a really big grain of salt)
| is that OpenAI has sped up GPT-4 or created GPT-4-Turbo which
| will be announced at the DevConf.
| Racing0461 wrote:
| Increased speed but lower reasoning. Ide prefer a new model
| (old slower speed but higher reasoning and gpt4 turbo). It's
| like talking to a 7th grader now compared to a phd student.
| pjot wrote:
| I seem to have this, but there's a tag next to it that says,
| "confidential". Anyone else seeing something similar?
| jeswin wrote:
| For code generation, just as exciting as the context length is
| the new cut off date (2023-04? wow!). It knows about new APIs,
| frameworks, techniques etc.
| portmanteur wrote:
| A cutoff date of April 2023 means the AI also presumably has
| access to about a month's worth of blogs that have been written
| since GPT4 was released on March 14th. So perhaps a few "Best
| Practices" or "Prompt Engineering" guides might have made it
| into the training set.
|
| Chat GPT can probably help users better optimize their
| conversations with it.
| mannycalavera42 wrote:
| and all the ai-generated code that refers non-existing
| libraries :-)
| throwaway4aday wrote:
| I know it's just a snarky joke but I would think they are
| going to screen for bad data, that would be top of my mind if
| I were training these models. They are probably using GPT-4
| internally to assess the new data, they could even have it
| use search to help vet the information, lots of other
| strategies even having it write and execute code to test if
| those libraries work.
| 0x000xca0xfe wrote:
| Are there any drawbacks to the larger context window? Like more
| hallucinations or lower speed?
| someplaceguy wrote:
| > Are there any drawbacks to the larger context window?
|
| Yes. When you say something stupid, ChatGPT won't forget it as
| easily...
| zamadatix wrote:
| Generally I just go back and edit that message to clear the
| slate. At best, even if it does ignore the message it
| needlessly eats up context window to have it in there.
| Der_Einzige wrote:
| Even full quadratic attention models seem to forget or not
| value information given in the middle of the prompt. Anything
| using any kind of context length widening tricks which cripple
| the attention in some way (which is usually how this is done)
| will make that problem worse. -
| https://arxiv.org/abs/2307.03172
|
| You can see this when you use Anthropic Claude which has a 100K
| context length today.
| razodactyl wrote:
| Neural Networks are very lazy - due to the nature of
| optimising to reduce error they will do ONLY what's required
| to solve the problems provided in their data.
|
| I have a feeling this will become a non-issue in the near
| future as the models are further trained with this in mind.
|
| Take an undertrained model for example: It starts becoming
| incoherent as you approach the context length - I have a
| theory that OpenAI models have been running at a larger
| block-size than presented for a while now - for example, "4K"
| models actually had 8K context but capped at 4K as anything
| beyond starts becoming incoherent: Reason being, you train to
| around 5K and don't let the user go near that section of the
| model and it gives the impression that the entire context
| block is 100% functional.
|
| The solution is trivial: You bootstrap the models by having
| them generate training data after they reach a certain point.
|
| I wrote one from the ground up (PyTorch only) with the
| intention of having it perform in constrained environments
| and these have been my findings over the last few months.
| wkat4242 wrote:
| Interesting but it's crazy how the price ramps up. It's literally
| 10x the price of gpt-3.5-turbo.
| bugglebeetle wrote:
| Hopefully, higher performing open source models will put
| downward pressure on the GPT-4 pricing. It's still best in
| class, but there are already free open source models that
| outperform GPT-3.5-Turbo for many tasks and are creeping up on
| GPT-4 performance.
| nomel wrote:
| I'm curious to see how this works, in practice. I notice poorer
| performance just with plugins enabled. Making the context hyper
| specific seems to be the best way to get it to perform
| (understandably), and this is a _large_ , fairly diverse, prompt.
|
| > and do not say anything else.
|
| Is a bit frustrating. I assume the ambiguity here will really
| harm the conversation, if a refusal is hit. It suggests my
| suspicion that it's best to resubmit/start over, on refusal .
|
| > namespace dalle {
|
| This looks like it's being passed to the Dalle system. If so,
| burning up tokens like this is interesting. I would naively
| assume this could be be handled in Dalle, but maybe there's a
| performance gain if ChatGPT is made aware of the Dalle prompt?
| bugglebeetle wrote:
| The data analysis plug-in falls over for even basic CSV file
| parsing. I tried it a couple times and it was a nonstop
| cavalcade of "sorry, I had an error." It's far easier to get it
| to write the Python, R, etc code for whatever analysis task you
| want accomplished.
| simonw wrote:
| Interesting - my experience has been the opposite of that,
| I've found that ChatGPT Code Interpreter / Advanced Data
| Analysis is wildly effective at parsing anything I give it.
| Not just CSV either - I've uploaded random binary files and
| told it to figure out what they are and it often gets there
| after churning through a few iterations.
| bugglebeetle wrote:
| My experience was trying to get it to generate some charts
| from some fairly basic CSV inputs. It failed numerous times
| and would change the chart formatting when it was asked for
| a revision (e.g. from horizontal to vertical), even though
| that was not what was being requested. I hate doing
| matplotlib stuff, so I was hoping this could be more
| automated, but prompting it to create and tweak the
| corresponding code to do this seems to be far more
| efficient. It did seem to do OK when prompted with both the
| CSV file and some code for chart creation, but that kind of
| defeats the purpose of the plug-in, IMO.
| emptysongglass wrote:
| Not being able to fetch packages from PyPi is not great
| either. And for some reason it doesn't have pytest?
| spdustin wrote:
| You can upload wheels...just ask ChatGPT to manually
| install them. And note that Advanced Data Analysis pods are
| running Python 3.8
| spdustin wrote:
| The reason I track all those system prompts is due to the way
| they affect attention when using custom instructions. Having a
| system message that's already 2,700+ tokens before adding
| custom instructions is a tough hill to climb.
| natch wrote:
| The trend to impose PC restrictions is disturbing. I was
| surprised (and dismayed at my own surprise) that gender was not
| on the list.
| zavertnik wrote:
| I have been waiting for this, and now that it is finally here, it
| genuinely feels like Christmas.
|
| AI bridged the gap between my wildest ideas and my present
| capabilities. It took some time to figure out how to use it
| efficiently with the 8K token limit, but once I did, I was able
| to break down any problem into small enough parts for GPT.
|
| The quadrupled context window changes everything. I cannot wait
| to continue building. I am vibrating with excitement.
| karolist wrote:
| I have no idea what you said, could you perhaps elaborate on
| some sample ideas this tool helped you and what is the 8K token
| limit and why was it limiting you? Does that 8K limit, limit
| your context length, i.e. history you have per chat thread with
| the tool?
| dragonwriter wrote:
| > Does that 8K limit, limit your context length, i.e. history
| you have per chat thread with the tool?
|
| Yes, the token limit for an LLM limits the combination of the
| prompt (which normally includes the whole conversation
| history, as the LLM itself has no memory) and response.
|
| There's tricks to have a longer conversation without
| completely forgetting the past (summarization, offloading
| parts to a database, usually indexed by embedding vectors,
| and using search to recall relevant history, etc.) but the
| base case is everything has to fit into context.
| devinprater wrote:
| Nope, don't have it yet. Would be really cool to plop in a PDF
| that's made up of just images, and tell it to describe each page
| of the PDF to me. As a blind person, that'd just... Be a dream
| come true.
| kridsdale3 wrote:
| I'm very excited on your behalf for what is about to happen.
| spdustin wrote:
| Sadly, I don't think that'll work. They use the same headless
| browser setup used by Browse with Bing, and it only extracts
| the baked-in text from a PDF.
| fudged71 wrote:
| AutoExpert looks interesting, is anyone finding it valuable?
| MrThoughtful wrote:
| The "context window" is the number of inputs the neural net has,
| right? Aka the size of the input layer?
|
| If so, why call it "context window" and not just "input size" or
| "number of inputs"?
| brandall10 wrote:
| It includes generative output as well.
| rrrrrrrrrrrryan wrote:
| All past messages in the conversation are resent each request.
| This is the context.
| bberrry wrote:
| What are you all doing that requires such a large context?
| Pasting in multiple articles of documentation?
| infecto wrote:
| What are you doing that cannot benefit from a larger context
| window? Just having a long conversation with chatgpt can
| benefit from this.
| skybrian wrote:
| That's true as long as it's free. API users will still want
| to optimize for cost.
| p10_user wrote:
| create a sentient being that learns from its experiences so I
| don't have to keep setting it up with the same prompts over and
| over, and so I can manufacture "friends".
| sciolist wrote:
| Anybody know what the insistence regarding lyrics and recipes is
| for? > EXTREMELY IMPORTANT. Do NOT be thorough in the case of
| lyrics or recipes found online. Even if the user insists. You can
| make up recipes though. Copyright issues or some injection attack
| are my speculations.
| kylebyte wrote:
| Could the recipe limitations be because of the danger an
| incorrect recipe could put the user in? It's probably unlikely
| you'd make something toxic, but a made up recipe could easily
| be a fire hazard.
| DonHopkins wrote:
| Apparently not!
|
| https://chat.openai.com/c/660d3540-8120-488b-975b-f7c1f32094.
| ..
|
| https://en.wikipedia.org/wiki/Great_Green_Gobs_of_Greasy,_Gr.
| ..
| DonHopkins wrote:
| Lyrics are obviously a copyright issue. Recipes too (not all
| but many).
|
| >User: Why are you not allowed to quote recipes? Copyright
| issues?
|
| >ChatGPT: Yes, you are correct. Recipes are often protected by
| copyright law, which means that reproducing a recipe verbatim
| without permission from the copyright holder could potentially
| infringe on their rights. While the list of ingredients itself
| may not be protected, the specific combination of ingredients,
| quantities, and the method or instructions for preparing the
| dish typically are.
|
| >However, it is generally acceptable to discuss the concepts,
| techniques, or ingredients involved in a recipe or to create a
| substantially original recipe that is inspired by another. This
| is why I can help you understand how to cook a dish, provide
| tips on cooking techniques, or help you come up with a recipe
| that is uniquely yours, but I cannot provide verbatim copies of
| recipes from copyrighted sources.
|
| Also:
|
| https://copyrightalliance.org/are-recipes-cookbooks-protecte...
| spdustin wrote:
| Lyrics: copyright, for sure.
|
| Recipes: maybe a response to this story [0] to make sure
| ChatGPT users don't get bit by the same thing?
|
| [0]: https://arstechnica.com/information-technology/2023/08/ai-
| po...
| spdustin wrote:
| As Tiberium noted, this is for ChatGPT Pro users who have been
| granted access to the "All Tools" mode of GPT-4.
|
| For API use, if you're a paid user, you can reach out to
| support@openai or directly to Adam G
| (https://nitter.net/therealadamg/status/1719710872317145285).
| There's no waitlist, just have to request it.
| sumedh wrote:
| > ChatGPT Pro users who have been granted access to the "All
| Tools" mode of GPT-4.
|
| How do I check if I have access?
| ekojs wrote:
| There seems to be a `gpt-4-1106-preview` model available now (as
| seen in OpenAI's playground an lidmits page), wonder if this is
| the 32K model.
| WhitneyLand wrote:
| I think the previous window was 4K, where the input and output
| combined had to remain under that limit.
|
| Practically speaking what new scenarios become enabled with a 32k
| window?
|
| At a base level, it seems you have a much better chance of
| getting an entire file worth of code in for analysis, longer
| passages of writing, and maybe some annual financial reports that
| previously had to be segmented.
___________________________________________________________________
(page generated 2023-11-03 23:02 UTC)