Posts by moultano@sigmoid.social
(DIR) Post #ASUYJs6TTzcn56SQAC by moultano@sigmoid.social
2023-02-08T22:39:00Z
0 likes, 1 repeats
If this is the first time you are posting on here in the last month, reconsider! I would much rather hang out with y'all on here.
(DIR) Post #ASWCfOxehidGOT75ii by moultano@sigmoid.social
2023-02-09T22:33:47Z
0 likes, 1 repeats
Language is a good compression of our thoughts, by design, so operating in language space rather than experience space makes it easier to learn to think.
(DIR) Post #ASeqT6YoyQ8RPSl7nU by moultano@sigmoid.social
2023-02-13T23:30:12Z
0 likes, 0 repeats
@ct_bergstrom Disagree. They're designed to mimic what a human would write. If they end up bullshitting it's because the models aren't good enough, not because that's what they're designed to do.
(DIR) Post #ASeqT7kYYH7H69G0jA by moultano@sigmoid.social
2023-02-13T23:40:15Z
0 likes, 0 repeats
@ct_bergstrom LLMs represent whether they "believe something to be true" in a way that you can extract unsupervised. Not disagreeing that their world model isn't good enough to be used without auxiliary retrieval, but there's some evidence they have one. https://arxiv.org/abs/2212.03827
(DIR) Post #ASeqT9Cx8BQPcVYCcy by moultano@sigmoid.social
2023-02-14T03:24:03Z
0 likes, 0 repeats
@ct_bergstrom @not2b Huge fractions of what I know have no anchor point outside of text, like nearly all science and math.
(DIR) Post #ASeqT9Icn5xhu6CjT6 by moultano@sigmoid.social
2023-02-13T23:51:22Z
0 likes, 0 repeats
@ct_bergstrom Another way of disentangling things. For the question you're asking, does the answer exist on the web? If it does, then the problem can't be with the "design" (I. e. the training regime) but rather the power of the model.
(DIR) Post #ASlvPTEEc1SCQqw4um by moultano@sigmoid.social
2023-02-16T21:29:20Z
0 likes, 1 repeats
Shibboleths rule every statistical system trained on text. The meaning of a word is deeply coupled to the characteristics of the slice of the people who use it.It would not surprise me at all if fine tuning to use a lot of emojis is what unleashed the unhingedness characteristic of someone who uses a lot of emojis.
(DIR) Post #ASlvPVGR2oP0kFyrce by moultano@sigmoid.social
2023-02-16T21:41:30Z
0 likes, 0 repeats
Consider the difference between "Did dinosaurs really exist?" and "When did dinosaurs go extinct?" Both of these would be satisfied by similar completions, but the social context of the former implies either a young-earth creationist or a kid, so entails different subsequent text.
(DIR) Post #ASlvPWustuLtrzugJE by moultano@sigmoid.social
2023-02-16T21:48:08Z
0 likes, 0 repeats
LLMs don't generally include the domain name of their source in their context window (They should!) so you don't get magic keywords like "Artstation" to elicit a certain social context like in image models. Instead, if you talk like a particular website, you keep doing so.
(DIR) Post #ASoUIBEP0qz9IcC2hE by moultano@sigmoid.social
2023-02-18T19:15:55Z
0 likes, 1 repeats
It is a mistake to turn authors into "brands" instead of people. That's what happened with Dr. Seuss and doomed his work to rewrites and censorship and what's happening here. Everybody wants to model Disney where the person, Walt, disappears, and leaves behind only the G rating.https://www.telegraph.co.uk/news/2023/02/17/roald-dahl-books-rewritten-offensive-matilda-witches-twits/
(DIR) Post #ASoUICyWWrTKhwmODw by moultano@sigmoid.social
2023-02-18T19:16:09Z
0 likes, 0 repeats
Roald Dahl isn't a generic brand of children's entertainment, he's a misanthropic antisemite who wrote strange dark books for children. They are not the same books without the misanthropy. The careless captiousness and violence is what makes his books what they are.
(DIR) Post #ASqptGN7LkQNJJiHhY by moultano@sigmoid.social
2023-02-19T21:14:27Z
0 likes, 1 repeats
The thing about "stochastic parrots" is that when a model is under trained it is one, but when it's trained enough it isn't, and it's really hard to tell the difference from just the output in any particular context.
(DIR) Post #AT5dlnLGbDIDAbv2A4 by moultano@sigmoid.social
2023-02-27T01:51:15Z
0 likes, 1 repeats
I'm RETVRN about approximately zero things, except for bread. Bread is just way worse. The first leavened bread some neolithic farmer made by accident was better than any bread you can find in an ordinary grocery.
(DIR) Post #AT9isr32UmSb5uQ0qO by moultano@sigmoid.social
2023-03-01T01:08:39Z
0 likes, 0 repeats
@alex It could be that the Chinese government doesn't know, and doesn't want to know. Characteristic of authoritarian systems is that bad information doesn't flow up.
(DIR) Post #AT9mB8DDiYvRP3aTYW by moultano@sigmoid.social
2023-03-01T01:45:30Z
0 likes, 0 repeats
@alex Yeah, it seems like there's no upside for them. Evidence for stricter lab precautions vs global embarrassment? No contest.
(DIR) Post #ATDYZ6Plwi0x9HtJa4 by moultano@sigmoid.social
2023-03-02T21:31:40Z
0 likes, 1 repeats
What's the best vernacular phrase for "moved my priors?" The best I can come up with is "shifted my thinking.""Shifted" implies continuity, and the possibility of small changes."Thinking" emphasizes that it's a constant process through time, not a one time decision.
(DIR) Post #ATH6ivMzqQuNhoKN7I by moultano@sigmoid.social
2023-03-04T04:33:47Z
0 likes, 2 repeats
"I am not a predictive text engine!" I scream as I call my family member by the wrong name because the rest of the sentence and circumstance makes that name the more common token.
(DIR) Post #AUPfdgJh41UNcaK1p2 by moultano@sigmoid.social
2023-04-07T14:41:02Z
0 likes, 0 repeats
People intuitively doubt that language influences their color perception, but it's easy to demonstrate with sound. Hindi speakers can't hear the difference between v and w, and English speakers can't hear the difference between त and ट.
(DIR) Post #AUPfdhljfFVw7qRwAa by moultano@sigmoid.social
2023-04-07T15:26:25Z
0 likes, 1 repeats
Would more people discover that they are tetrachromats if the tetrachromats had words for the colors they see?
(DIR) Post #AUW6Qr30TTccwQKgfg by moultano@sigmoid.social
2023-04-10T18:08:36Z
0 likes, 0 repeats
@TedUnderwood There's no way to wean people off getting paid though.