Post AUPT6rpZPit2uIkEXQ by bwyble@neuromatch.social
(DIR) More posts by bwyble@neuromatch.social
(DIR) Post #AUMSqGHEgg1mKQt408 by pbloem@sigmoid.social
2023-04-02T13:33:56Z
0 likes, 1 repeats
I feel the stochastic parrot crowd is doing as much damage in the discourse around LLMs as the longtermists. I don't see how you can try GPT-4 and come away thinking that all it's doing is clever collaging of its training set.It's harmful to shout that "the terminators are coming", but it's equally harmful to downplay what LLMs are capable of.
(DIR) Post #AUMT7Ukw6kN89ozrZQ by TedUnderwood@sigmoid.social
2023-04-06T02:35:43Z
0 likes, 0 repeats
@pbloem I don't understand why anyone is still listening to writers who have been repeatedly wrong on this topic and positively take pride in refusing to update their priors when wrong. And yes, they're fostering a kind of denialism that does actual harm.
(DIR) Post #AUPNFYVY1fuTrL8gZk by bwyble@neuromatch.social
2023-04-06T02:52:03Z
0 likes, 0 repeats
@pbloem I recently asked Gpt-4 to help me understand how to do something with Amazon's elastic beanstalk. It took some basic advice on configuring EC2 instances and claimed that it was about Elastic Beanstalk. Took me a few minutes to figure out that it was just stringing things together. I went back to Google and found the answer I needed.
(DIR) Post #AUPNFZ8tfNNxpNY6uO by pbloem@sigmoid.social
2023-04-06T06:42:28Z
0 likes, 0 repeats
@bwyble I think this is a subtle point to understand about large machine learning models: they always both memorize and generalize.That means you can always hit a weak spot where it can't give you a proper solution, and it may revert to remixing its training data.But that isn't proof that that's what it's always doing. It's the low-level behavior from which its more advanced behaviors are bootstrapped.
(DIR) Post #AUPNFZmFJ4rRnPxXF2 by bwyble@neuromatch.social
2023-04-06T11:05:56Z
0 likes, 0 repeats
@pbloem isn't that just another way of saying that it's a stochastic parrot?BTW I run into these situations quite a lot in my efforts to use GPT-4. I wouldn't say it's particularly rare
(DIR) Post #AUPNFaOX0jUBi9s6uu by pbloem@sigmoid.social
2023-04-06T13:20:33Z
0 likes, 0 repeats
@bwyble It's saying it can behave like one, but it can also do things that are proper generalization beyond the training data by any definition.
(DIR) Post #AUPNFaxysvqHU6SQAi by bwyble@neuromatch.social
2023-04-06T21:02:17Z
0 likes, 0 repeats
@pbloem how do you know it can generalize rather than just interpolate?
(DIR) Post #AUPNFbS74twezYYU8e by pbloem@sigmoid.social
2023-04-07T09:43:50Z
0 likes, 0 repeats
@bwyble Here's one example. I asked it to explain the joke in this Onion article: https://www.theonion.com/trump-all-arrests-are-politically-motivated-as-the-le-1850288923
(DIR) Post #AUPNFcDGFbevLmc8cy by bwyble@neuromatch.social
2023-04-07T11:35:06Z
0 likes, 0 repeats
@pbloem The difficulty with GPT is that we don't know what is in its training set. We can be comfortably certain that it has digested articles that explain how satire works, and probably even articles that explain the Onion's flavor of satire. I'm not saying that it's impossible that LLMs generalize but rather that it's very hard to demonstrate that this is so given their massive input, the list of which is inaccessible to us.
(DIR) Post #AUPNFcy3Rd5bguVVZ2 by pbloem@sigmoid.social
2023-04-07T11:52:54Z
0 likes, 0 repeats
@bwyble Are you saying that interpreting an article on how satire works and applying it to an example from the Onion is not generalization?
(DIR) Post #AUPNFdXVJpRhSr5ooq by bwyble@neuromatch.social
2023-04-07T12:05:38Z
0 likes, 0 repeats
@pbloem Creating a new thing is not necessarily generalization. Even the equation y = ax +b can generate novel exemplars along its line. It depends on how well the training distribution covered this space. Whether something is interpolation vs generalization is extremely hard to know for high dimensional spaces and large, unknown training sets.
(DIR) Post #AUPNFeDKoIuFYafE1I by TedUnderwood@sigmoid.social
2023-04-07T12:14:02Z
0 likes, 0 repeats
@bwyble @pbloem The equation y = ax+ b + ε is a textbook example of a generalization; it’s where statistical learning starts. Anything between that and applying an article on satire to a new context is definitely generalization. The question is whether it overfits the data. But a model as widely useful as an LLM is clearly not just overfitting.
(DIR) Post #AUPOfxffBziRNv1enQ by bwyble@neuromatch.social
2023-04-07T12:30:02Z
0 likes, 0 repeats
@TedUnderwood @pbloem that equation does do generalization yes but it also fills in an infinity of data points between training points.
(DIR) Post #AUPSFtphEvgzlhXxnk by TedUnderwood@sigmoid.social
2023-04-07T13:10:10Z
0 likes, 0 repeats
@bwyble @pbloem And there should be an infinity. I don't know that "interpolation" vs (what else? extrapolation?) is a well-defined alternative with stakes we need to care about. Overfitting is the concept whose utility I understand--and as I understand it, that's relative to new data. If all the new data you get is in practice within the old distribution, then you haven't overfit the function. The question of whether LLMs can be "truly original" is trickier, but not v well defined. +
(DIR) Post #AUPSVq6Ws0D2q11MCu by TedUnderwood@sigmoid.social
2023-04-07T13:13:03Z
0 likes, 0 repeats
@bwyble @pbloem I would guess the answer is no, but it's hard to say what that means because we don't have a very good metric of human originality (originality that is not *in any way* interpolative). My hand-wavy guess is that it's pretty rare and more of a collective phenomenon than an individual achievement.
(DIR) Post #AUPT6rpZPit2uIkEXQ by bwyble@neuromatch.social
2023-04-07T13:19:44Z
0 likes, 0 repeats
@TedUnderwood @pbloem I think that's exactly right. We know that humans as a species are creative, but it's possible that individuals are mostly derivative/interpolating. For me the definitive thought experiment about LLMs (or image generators) would be to take a few dozen of them, pretrained on human work and lock them into a server cluster where they train on each other's output for a long time... what would you get? It might be that they create amazing new works, or it could devolve into noisy garbage, or converge on a small set of recurring outputs. Which of those three you get would be effective in settling debates about whether they are truly creative.(and I realize that I have been conflating generative and creative in this discussion which is not exactly right)