https://nostalgebraist.tumblr.com/post/648383245570949120/meta-post-on-meta-learning trees are harlequins, words are harlequins tumblrhit tracking tool * ask * tags * fiction * robot * avatar by doni19 Install Theme meta-post on meta-learning There's an LW post I keep trying to write. I have several unpublished draft versions of it. The point I want to make is simple and straightforward, but when I try to write it down, I get worried I'm not ... like, "messaging" it correctly? Not striking the right tone? The point of the post is roughly: People don't use the term "meta-learning" consistently when they're talking about GPT-3. The paper uses the term one way (and they are 100% explicit, they spell out their definition in the text), the blogging community uses it another way. The bloggers are excited/scared that GPT-3 does "meta-learning" by which they mean something like "general reasoning on the fly without training." If you're excited/scared by this capability (and you should be), then you should really care whether GPT-3 actually has it, to what extent, how the capability scales, etc. There is very little public evidence on this topic, because the paper is (explicitly!) 95% not about the topic, the remaining 5% is pretty weak evidence, and the only other evidence out there is like ... some subjective user impressions? gwern saying "GPT-3 has the capability" in a really eloquent and forceful way? It would be easy to test the capability much more rigorously than this. This ought to be done since the topic is important. It can only be done by people with API access (AI Dungeon doesn't count). But it ... feels hard to say this in a way that could actually convince anyone who doesn't already agree? Like, 1. These points seem so clearly true to me that when I try to "argue for them," I feel pedantic and like I'm belaboring the obvious. Do I actually have to say "no, few-shot translation from French to English is not an example of general reasoning on the fly?" Surely no one thinks the model is like ... learning how to speak French from ~2000 words of data? Do I have to quote the part of the paper where it says what it means by meta-learning? It's right there! You can just read the paper! 2. I made most of this argument already in my original GPT-3 post, immediately after reading the paper. So (A) I feel like I'm repeating myself and (B) if the point didn't get across then, why would it now? 3. There is an element of "mere semantics" to the point and it's hard to clarify to my satisfaction that no, I don't just care that blog posts are using a word incorrectly. But I have to bring up the semantic issue to even describe what I am saying. 4. It feels inevitably like picking on gwern's choice of words, since blogosphere beliefs about "GPT-3 meta-learning" basically all trace back to gwern's blog. I don't care about whether gwern is using the right words, he's just the most detailed "primary source" we have on the topic due to the closed API I was thinking about this yesterday because @slatestarscratchpad linked my original GPT-3 post in his April linkpost. I actually sat down and wrote up another one of those drafts and ... nope, gave up again. I notice I am able to write this on tumblr with no problems at all. Perhaps this is yet another point of evidence that using tumblr lets me do much more "real blogging" than I could if I had "a real blog." * #ai tag * #gpt-3 13th Apr 202139 notes 1. [avatar_d80]universalplsujlu reblogged this from nostalgebraist 2. [avatar_2f9]thecurioustale liked this 3. [avatar_e42]urpriest liked this 4. [avatar_f3b]danup liked this 5. [98f289abe0]reclezon liked this 6. [avatar_560]nostalgebraist liked this 7. [default_av]the-moti reblogged this from nostalgebraist and added: Thanks for the enlightening backstory!......You may be able... 8. [525d70ca3c]beansprouts liked this 9. [avatar_74b]raginrayguns said: i regularly get replies to tumblr posts one meta level below the one im writing at. "evidence E for hypothesis H seems weak" "ok but there's evidence E2, E3..." look im writing about E, do you care about conditionals or not. And I'm trying to start give very short replies to these, like "not relevant because x" instead of doubling down on explaining what im really talking about. 10. [avatar_af7]vulturaldeterminants liked this 11. [avatar_df1]birth-muffins-death liked this 12. [avatar_503]nightpool liked this 13. [avatar_d32]gen-adder liked this 14. [ad8446d8ac]madboiimurder liked this 15. [avatar_106]kit-peddler liked this 16. [avatar_735]discoursedrome liked this 17. [avatar_340]akkaaaaaalltyyyn liked this 18. [pyramid_cl]nermbley reblogged this from nostalgebraist 19. [1688f03349]transhumanoid liked this 20. [avatar_86d]stumpyjoepete liked this 21. [default_av]marlemane liked this 22. [avatar_560]nostalgebraist reblogged this from the-moti and added: I share the sense that it's best to focus on the proposed experiments -- it's actionable advice, it feels constructive,... 23. [avatar_a0f]minitiate liked this 24. [034faaf75a]the-question-is-now liked this 25. [cone_open_]pretends-to-be-a-catgirl-online liked this 26. [b4e5634626]lexisjourney reblogged this from nostalgebraist 27. [b4e5634626]lexisjourney liked this 28. [4ce2bcb486]north-of-everything liked this 29. [avatar_a0d]illuminatiswag liked this 30. [avatar_3a7]typicalacademic liked this 31. [245f9d267d]crabgirlclaw liked this 32. [avatar_b2c]eclairs-of-emptiness reblogged this from nostalgebraist 33. [0dc043c51c]candleprism liked this 34. [avatar_a36]di--es---can-ic-ul-ar--es liked this 35. [default_av]the-moti liked this 36. [avatar_6ef]whothefuckstoledonut235 liked this 37. [946c17d1e0]kata4a liked this 38. [e47f81fdb5]eikotheblue liked this (c) 2011-2021 - trees are harlequins, words are harlequins - Powered by Tumblr UltraZen Theme by UltraLinx [impixu][impixu]