Posts by ngaylinn@tech.lgbt
(DIR) Post #B45NWTEJ5ZGwGHril6 by ngaylinn@tech.lgbt
0 likes, 0 repeats
A lab mate shared this write up of Don Knuth using LLMs to solve a math problem: https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cycles.pdfIt's clear that using Claude did help them arrive at some new understanding here, which is wonderful. I'm happy for them.However, I'm upset by how much they personify Claude and attribute the solution to "him."From this narrative, it's clear that the humans were very actively involved from beginning to end. Claude was a helpful tool, but it did not solve this problem on its own. What role did it actually play? How was it like or unlike a human collaborator on this problem?It did generate a crucial insight, but where did that come from? Was it plagiarized from some unknown source? Did it "just emerge" from text completion and interpolation in latent space? Do we need some other explanation for Claude's apparent creativity?These folks don't care. They just wanted a solution, which they attribute to Claude, and leave it at that. I think that's a serious problem.#llm #ai #science #math
(DIR) Post #B498aPJuqRU6cAkR2e by ngaylinn@tech.lgbt
0 likes, 0 repeats
@pbloem I fear the implausibility is kind of the point. He's above reasons now. 🙄
(DIR) Post #B5A36VQpIboY4MNVr6 by ngaylinn@tech.lgbt
0 likes, 0 repeats
I thought this was a particularly good analysis of the problem of using LLMs for science. It explores the purpose of science, the perverse incentives that drive people to use LLMs, and the impact this has on skill building and training future scientists.My lab group has been struggling with this topic lately, without much consensus. This blog post captures a lot of our thinking, and very clearly made some good points that we appreciated. It mostly just describes the mess we're in without offering much useful advice, but just laying out the problems do nicely is helpful. That said, I do worry the author may be underestimating the impact these tools might have on experienced researchers.https://ergosphere.blog/posts/the-machines-are-fine/#academicchatter #llm
(DIR) Post #B5FpFJWMie6bKF2C4u by ngaylinn@tech.lgbt
0 likes, 0 repeats
@abucci Honestly, I'm with you. I boycott LLMs. It's not even slightly difficult for me. They're harmful and unethical, but also they solve a problem I don't have. I'm a fluent and prolific writer / coder, who enjoys craft, expression, and hard work.I wish it were as simple for my peers. They are very tempted by LLMs, and not dissatisfied with the results. I think it stems from feelings of inadequacy. Publish or perish. English as a second language. Coding imposter syndrome. Job prospects. FOMO. Whatever. They feel a strong temptation, despite knowing all the down sides.I push back where I can, and I think that's having a positive impact. My PI even put a moratorium on gen AI for now. But I don't think telling people they're simply wrong is a productive way of changing their behavior. I do not agree with or endorse all the things in this article, but I do sympathize with the author, and I think he does a good job of explaining the sort of conflict my lab and many others are struggling with right now.
(DIR) Post #B5G3dOh2XVHffuPC1w by ngaylinn@tech.lgbt
0 likes, 0 repeats
@abucci Completely agreed. I find the amount of learned helplessness in tenured professors pretty alarming.In this case, the PI is choosing to let the discussion and ultimate policy emerge from discussion with the group, rather than simply putting his foot down.I like that he's trying to get more opinions and make the whole lab feel involved. However, this same tendency when taken too far can be an abdication of responsibility. It's a bit of a pattern with him: conflating freedom and autonomy of his students with never providing them strong guidance based on his much greater experience.We also have never attempted to codify what values the lab works by, which makes it hard to talk about whether use of LLMs aligns with those values. At least this discussion is raising some of those value questions.
(DIR) Post #B5HgwvT304RACA7hcu by ngaylinn@tech.lgbt
0 likes, 0 repeats
@abucci That's mostly how we do it, too. ;)
(DIR) Post #B5M7hdDQ3MyMddaIuu by ngaylinn@tech.lgbt
0 likes, 0 repeats
It's amazing how often I'll stumble across whole fields of study that are relevant to my work, but which I never even knew existed until I accidentally bumble my way into the right papers.The academic literature has a*massive* discoverability problem. I'd be lost without personal recommendations and citation trails!#academicchatter
(DIR) Post #B5MDZMohjrCtBfT0Oe by ngaylinn@tech.lgbt
0 likes, 0 repeats
@abucci I do like that perspective. Maybe that's really just the nature of what we do. It's just hard to accept that this is the best option, when so much valuable knowledge is hidden where laypeople will never find it (and even the experts sometimes don't!)
(DIR) Post #B6ixFkoiBynCMaRMtk by ngaylinn@tech.lgbt
0 likes, 0 repeats
I hate how LLMs are often used, but as a model of cognition I think they're fascinating. Take this paper, for example.Here they look at reasoning over long sequences of data that can't fit into working memory. The latest LLMs can solve problems like this, using hybrid memory models that consolidate "short-term" memories in an LRU cache into "long-term" memories in the form of persistent weights (which is good to know about generally).However, this paper shows that this is still a very hard problem. In particular, it's hard to organize the consolidated information in a way that's useful for doing future work, and that a "sleep-like" process of iterative offline processing can help significantly.It remains to be seen how important this will be and how "sleep-like" it really is. Still, I love how this work draws inspiration from nature and could potentially help us learn about animal minds and cognition generally. Much better than many LLM studies.#science #ai #llm