[HN Gopher] AI Blindspots - Blindspots in LLMs I've noticed whil...
       ___________________________________________________________________
        
       AI Blindspots - Blindspots in LLMs I've noticed while AI coding
        
       Author : rahimnathwani
       Score  : 218 points
       Date   : 2025-03-19 16:48 UTC (6 hours ago)
        
 (HTM) web link (ezyang.github.io)
 (TXT) w3m dump (ezyang.github.io)
        
       | ezyang wrote:
       | Hi Hacker News! One of the things about this blog that has gotten
       | a bit unwieldy as I've added more entries is that it's a sort of
       | undifferentiated pile of posts. I want some sort of organization
       | system but I haven't found one that's good. Very open to
       | suggestions!
        
         | datadrivenangel wrote:
         | Maybe organize them more clearly split between observed
         | pitfalls/blindspots and prescriptions. Some of the articles
         | (Use automatic formatting) are Practice forward, while others
         | are pitfall forward. I like how many of the articles have
         | examples!
        
         | smusamashah wrote:
         | How about listing all if these on 1 single page? Will be easy
         | to navigate/find.
        
           | ezyang wrote:
           | They are listed on one page right now! Haha
        
             | elicash wrote:
             | They're _indexed_ on one page, but you can 't scan/scroll
             | through these short posts without clicking because the
             | content itself isn't all on a single page, at least not
             | that I can find.
             | 
             | (I also like the other idea of separating out pitfalls vs.
             | prescriptions.)
        
               | lelandfe wrote:
               | Wordpress's approach to this is giving each post a short
               | description in addition to the main content. The excerpt
               | gets displayed on the main list, which helps both to grok
               | the post and keep the list from becoming unwieldy.
        
         | cookie_monsta wrote:
         | Some sort of navigation would be nice a prev/next or some way
         | to avoid having to go back to the links page all the time.
         | 
         | All of the pages that I visited were small enough that you
         | could probably wrap them them <details> tags[1] and avoid
         | navigation altogether
         | 
         | [1] https://developer.mozilla.org/en-
         | US/docs/Web/HTML/Element/de...
        
         | rav wrote:
         | My suggestion: Change the color of visited links! Adding a
         | "visited" color for links will make it easier for visitors to
         | see which posts they have already read.
        
         | sfink wrote:
         | When I saw the title, I knew what this was going to be. It made
         | me want to immediately write a corresponding "Human Blindspots"
         | blog post to counteract it, because I knew it was going to be
         | the usual drivel about how the LLMs understand <X> but
         | sometimes they don't quite manage to get the reasoning right,
         | but not to worry because you can nudge them and their logical
         | brains will then figure it out and do the right thing. They'll
         | stop hallucinating and start functioning properly, and if they
         | don't, just wait for the next generation and everything will be
         | fine.
         | 
         | I was wrong. This is great! I really appreciate how you not
         | only describe the problems, but also describe why they happen
         | using terminology that shows you understand how these things
         | work (rather than the usual crap that is based on how people
         | imagine them to work or want them to work). Also, the examples
         | are excellent.
         | 
         | It would be a bunch of work, but the organization I would like
         | to see (alongside the current, not replacing it, because the
         | one-page list works for me already) would require sketching out
         | some kind of taxonomy of topics. Categories of ways that Sonnet
         | gets things wrong, and perhaps categories of things that humans
         | would like them to do (eg types of tasks, or
         | skill/sophistication levels of users, or starting vs fixing vs
         | summarizing/reviewing vs teaching, or whatever). But I haven't
         | read through all of the posts yet, so I don't have a good sense
         | for how applicable these categorizations might be.
         | 
         | I personally don't have nearly enough experience using LLMs to
         | be able to write it up myself. So far, I haven't found LLMs
         | very useful for the type of code I write (except when I'm
         | playing with learning Rust; they're pretty good for that). I
         | know I need to try them out more to really get a feel for their
         | capabilities, but your writeups are the first I've found that I
         | feel I can learn from without having to experience it all for
         | myself first.
         | 
         | (Sorry if this sounds like spam. Too gushing with the praise?
         | Are you bracing yourself for some sketchy URL to a gambling
         | site?)
        
         | joshka wrote:
         | What about adding a bit more structure and investing in a
         | pattern language approach like what you might find in a book by
         | Fowler or a site like https://refactoring.guru/. You're much of
         | the way there with the naming and content, but could refactor
         | the content a bit better into headings (Problem, Symptoms,
         | Examples, Mitigation, Related, etc.)
         | 
         | You could even pretty easily use an LLM to do most of the work
         | for you in fixing it up.
         | 
         | Add a short 1-2 sentence summary[1] to each item and render
         | that on the index page.
         | 
         | [1]: https://gohugo.io/content-management/summaries/
        
       | datadrivenangel wrote:
       | Almost all of these are good things to consider with human coders
       | as well. Product managers take note!
       | 
       | https://ezyang.github.io/ai-blindspots/requirements-not-solu...
        
       | fizx wrote:
       | The community seems rather divided as to whether these are
       | intrinsic, or we solve these with today's tech, and more
       | training, heuristics and workarounds.
        
       | mystified5016 wrote:
       | Recently I've been writing a resume/hire-me website. I'm not a
       | stellar writer, but I'm alright, so I've been asking various LLMs
       | to review it by just dropping the HTML file in.
       | 
       | Every single one has completely ignored the "Welcome to nginx!"
       | Header at the top of the page. I'd left it in half as a joke to
       | amuse myself but I expected it would get some kind of reaction
       | from the LLMs, even if just a "it seems you may have forgotten
       | this line"
       | 
       | Kinda weird. I even tried guiding them into seeing it without
       | explicitly mentioning it and I could not get a response.
        
         | SparkyMcUnicorn wrote:
         | Have you tried "Let's get this production ready" as a prompt
         | for this or any other coding tasks?
         | 
         | Sometimes when I ask for "production ready" it can go a bit too
         | far, but I've found it'll usually catch things like this that I
         | might miss.
        
         | eschaton wrote:
         | Why would you expect it to "get some kind of reaction?" That
         | strongly implies that you perceive what the LLM doing as
         | "understanding" the tokens you're feeding it, which *is not
         | something LLMs are capable of*.
        
           | ozmodiar wrote:
           | Come on man, even chemicals react.
        
         | namaria wrote:
         | It didn't ignore it. There just wasn't any pattern in the
         | training data about responding to such a line.
         | 
         | Having the mental model that the text you feed to an LLM
         | influences the output but is not 'parsed' as 'instructions'
         | helps understand its behaviors. The website GP linked is
         | searching for a zoo of problems and missing the biology behind.
         | 
         | LLMs don't have blindspots, they don't reason nor hallucinate.
         | They don't follow instructions. They pattern match on high
         | dimensional vector spaces.
        
       | antasvara wrote:
       | This highlights a thing I've seen with LLM's generally: they make
       | _different_ mistakes than humans. This makes catching the errors
       | much more difficult.
       | 
       | What I mean by this is that we have thousands of years of
       | experience catching human mistakes. As such, we're really good at
       | designing systems that catch (or work around) human mistakes and
       | biases.
       | 
       | LLM's, while impressive and sometimes less mistake-prone than
       | humans, make errors in a fundamentally different manner. We just
       | don't have the intuition and understanding of the way that LLM's
       | "think" (in a broad sense of the word). As such, we have a hard
       | time designing systems that account for this and catch the
       | errors.
        
         | tharkun__ wrote:
         | I don't think that's universally true. We have different humans
         | with different levels of ability to catch errors. I see that
         | with my teams. Some people can debug. Some can't. Some people
         | can write tests. Some can't. Some people can catch stuff in
         | reviews. Some can't.                   I asked Sonnet 3.7 in
         | Cursor to fix a failing test. While it made the necessary fix,
         | it also updated a hard-coded expected constant to instead be
         | computed using the same algorithm as the original file, instead
         | of preserving the constant as the test was originally written.
         | 
         | Guess what?
         | 
         | Guess the number of times I had to correct this from humans
         | doing it in their tests over my career!
         | 
         | And guess where the models learned the bad behavior from.
        
           | __MatrixMan__ wrote:
           | I agree. I've been been struck by how remarkably
           | understandable the errors are. It's quite often something
           | that I'd have done myself if I wasn't paying attention to the
           | right thing.
        
         | sorokod wrote:
         | You may find this interesting: "AI Mistakes Are Very Different
         | from Human Mistakes"
         | 
         | https://www.schneier.com/blog/archives/2025/01/ai-mistakes-a...
        
         | MostlyStable wrote:
         | This is, I think, a better way to think about LLM mistakes
         | compared to the usual "hallucinations". I think of them as
         | similar to human optical illusions. There are things about the
         | human visual cortex (and also other sensory systems, see the
         | McGurk Effect [0]), that, when presented with certain kinds of
         | inputs, will consistently produce wrong
         | interpretations/outputs. Even when we are 100% ware of the
         | issue, we can't prevent our brains from generating the
         | incorrect interpretation.
         | 
         | LLMs seem to have similar issues along dramatically different
         | axes, axes that humans are not used to seeing these kinds of
         | mistakes; where nearly no human _would_ make this kind of
         | mistake and so we interpret it (in my opinion incorrectly) as
         | lack of ability or intelligence.
         | 
         | Because these are engineered systems, we may figure out ways to
         | solve these problems (although I personally think the best we
         | will ever do is decrease their prevalence), but more important
         | is probably learning to recognize the places that LLMs are
         | likely to make these errors, and, as your comment suggests,
         | design work flows and systems that can deal with them.
         | 
         | [0] https://youtu.be/2k8fHR9jKVM
        
           | admiralrohan wrote:
           | Hallucinating is fine but overconfidence is the problem. But
           | I heard it's not an easy problem to solve.
        
           | pydry wrote:
           | I dunno hallucinations seem like a pretty human type of
           | mistake to me.
           | 
           | when i try to remember something my brain often synthesizes
           | new things by filling in the gaps.
           | 
           | This would be where I often say "i might be imagining it,
           | but..." or "i could have sworn there was a..."
           | 
           | In such cases the thing that saves the human brain is double
           | checking against reality (e.g. googling it to make sure).
           | 
           | Miscounting the number of r's in strawberry by glancing at
           | the word also seems like a pretty human mistake.
        
           | j45 wrote:
           | Some of the errors are caused by humans. Say, due to changing
           | the chat to only pay attention to recent messages and not the
           | middle, omitting critical details.
        
         | woopwoop wrote:
         | Agree, but I would point out that the errors that I make are
         | selected on the fact that I don't notice I'm making them, which
         | tips the scale toward LLM errors being not as bad.
        
           | worldsayshi wrote:
           | Yeah it's the reason pair programming is nice. Now the bugs
           | need to pass two filters instead of one. Although I suppose
           | LLM's aren't that good at catching my bugs without me
           | pointing them out.
        
             | diggan wrote:
             | I've found both various ChatGPT and Claude to be pretty
             | good at finding unknown bugs, but you need a somewhat hefty
             | prompt.
             | 
             | Personally I use a prompt that goes something like this
             | (shortened here): "Go through all the code below and
             | analyze everything it's doing step-by-step. Then try to
             | explain the overall purpose of the code based on your
             | analysis. Then think through all the edge-cases and
             | tradeoffs based on the purpose, and finally go through the
             | code again and see if you can spot anything weird"
             | 
             | Basically, I tried to think of what I do when I try to spot
             | bugs in code, then I just wrote a reusable prompt that
             | basically repeats my own process.
        
         | vanschelven wrote:
         | Nevermind designing _systems_ that account for this, even just
         | debugging such errors is much harder than ones you create
         | yourself:
         | 
         | https://www.bugsink.com/blog/copilot-induced-crash/
        
           | fragmede wrote:
           | For that case, it sounds more like having your tools commit
           | for you after each change, as is the default for Aider, is
           | the real winner. "git log -p" would have exposed that crazy
           | import in minutes instead of hours.
           | 
           | commit early, commit often.
        
       | teraflop wrote:
       | > I had some test cases with hard coded numbers that had wobbled
       | and needed updating. I simply asked the LLM to keep rerunning the
       | test and updating the numbers as necessary.
       | 
       | Why not take this a step farther and incorporate this methodology
       | directly into your test suite? Every time you push a code change,
       | run the new version of the code and use it to automatically
       | update the "expected" output. That way you never have to worry
       | about failures at all!
        
         | ezyang wrote:
         | In fact, the test framework I was using at the time (jest) did
         | in fact support this. But the person who had originally written
         | the tests hadn't had the foresight to use snapshot tests for
         | this failing test!
        
           | diggan wrote:
           | I don't know if your message is a continuation of the sarcasm
           | (I feel like maybe no?), but I'm pretty sure parent's joke is
           | that if you just change the expected values whenever the code
           | changes, you aren't really effectively "testing" anything as
           | much as "recording" outputs.
        
       | Mc91 wrote:
       | One thing I do is go to Leetcode, see the optimal big O time and
       | space solutions, then give the LLM the Leetcode medium/hard
       | problem, and limit it to the optimal big O time/space solution
       | and suggest the method (bidirectional BFS). I ask for the
       | solution in some fairly mainstream modern language (although not
       | Javascript, Java or Python). I also say to do it as compact as
       | possible. Sometimes I reiterate that.
       | 
       | It's just a function usually, but it does not always compile. I'd
       | set this as a low bar for programming. We haven't even gotten
       | into classes, architecture, badly-defined specifications and so
       | on.
       | 
       | LLMs are useful for programming, but I'd want them to clear this
       | low hurdle first.
        
         | bongodongobob wrote:
         | You're using a shitty model then or are lying. 4o one or two
         | shotted the first 12 days of advent of code for me without
         | anything other than the problem description.
        
       | logicchains wrote:
       | I found Gemini Flash Thinking Experimental is almost unusable in
       | an agent workflow because it'll eventually accidentally remove a
       | closing bracket, breaking compilation, and be unable to identify
       | and fix the issue even with many attempts. Maybe it has trouble
       | counting/matching braces due to fewer layers?
        
         | ezyang wrote:
         | Yeah, Sonnet 3.5/3.7 are doing heavy lifting. Maybe the SOTA
         | Gemini models would do better, I haven't tried them. Generating
         | correct patches is a funny minigame that isn't really solved,
         | despite how easy it is to RL on.
        
           | logicchains wrote:
           | o3-mini works well enough for me, it makes mistakes but
           | generally it can always fix them eventually. Interestingly I
           | found even if I include the line numbers as comments in the
           | code it sees, it still often gets the line numbers wrong for
           | edits (most often, off by one errors, likely due to it mixing
           | up whether the line numbers are inclusive or exclusive). What
           | does work a bit better is asking it to provide regex matching
           | the first and last line of what it wants to replace, along
           | with nearby line numbers (so if there are multiple matches in
           | that file for the regex, it gets the right one).
        
           | diggan wrote:
           | > Maybe the SOTA Gemini models would do better, I haven't
           | tried them
           | 
           | As I had to upgrade my Google Drive storage like a month ago,
           | I gave them all a try. Short version: If you have paid plan
           | with OpenAI/Claude already, none of them come even close, for
           | coding at least. I thought I was trying the wrong models at
           | first, but after confirming it seems like Google is just
           | really far behind.
        
       | taberiand wrote:
       | Based on the list, LLMs are at a "very smart junior programmer"
       | level of coding - though with a much broader knowledge base than
       | you'd expect from even a senior. They lack bigger-picture
       | thinking, and default to doing what is asked of them instead of
       | what needs to be done.
       | 
       | I expect the models will continue improving though, I feel like
       | most of it comes down to the ephemeral nature of their context
       | window / the ability to recall and attach relevant information to
       | the working context when prompted.
        
         | threeseed wrote:
         | I wonder if people who say LLMs are a smart junior programmer
         | have ever used LLMs for coding or actually worked with a junior
         | programmer before. Because for me the two are not even remotely
         | comparable.
         | 
         | If I ask Claude to do a basic operation on all files in my
         | codebase it won't do it. Half way through it will get
         | distracted and do something else or simply change the
         | operation. No junior programmer will ever do this. And similar
         | for the other examples in the blog.
        
           | zarathustreal wrote:
           | Since when is "do something on every file in my codebase"
           | considered coding?
        
             | threeseed wrote:
             | Refactoring has been a thing since well forever.
        
             | andoando wrote:
             | Maybe its not but its a comparatively simple task a junior
             | developer can do.
        
           | curious_cat_163 wrote:
           | > If I ask Claude to do a basic operation on all files in my
           | codebase it won't do it.
           | 
           | Not sure exactly how you used Claude for this, but maybe try
           | doing this in Cursor (which also uses Claude by default)?
           | 
           | I have had pretty good luck with it "reasoning" about the
           | entire codebase of a small-ish webapp.
        
         | nomel wrote:
         | > and default to doing what is asked of them instead of what
         | needs to be done.
         | 
         | I don't think it's that simple.
         | 
         | From what I've found, there are "attractors" in the statistics.
         | If a part of your problem is too similar to a _very common_
         | problem, that the LLM saw a million times, the output will be
         | _attracted_ to those overwhelming statistical next-words, which
         | is understandable. That is the problem I run into most often.
        
           | Groxx wrote:
           | It's a constant struggle for me too, both "in the large" and
           | small situations. Using a library which provides special-
           | cased versions of common concepts, like "futures"? You'll get
           | non-stop mistakes and misuses, even if you've got correct
           | ones right next to it, or feed it reams of careful
           | documentation. Got a variable with a name that sounds like it
           | might be a dictionary (e.g. `storesByCity`), but it's
           | actually a list? It'll try to iterate over it like a
           | dictionary, point out "bugs" related to unsorted iteration,
           | and will return `var.Values()` instead of `var` when your
           | func returns a list. Practically every single time, even
           | after multiple rounds of "that's a list"-like feedback or
           | giving it the compilation errors. Got a Clean-Code-like
           | structure in some things but not others? Watch as it assumes
           | everything follows it all the time despite massive evidence
           | to the contrary.
           | 
           | They're rather impressive when building common things in
           | common ways, and a LOT of programming does fit that. But once
           | you step outside that they feel like a pretty strong net
           | negative - some occasional positive surprises, but lots of
           | easy-to-miss mistakes.
        
         | lelanthran wrote:
         | > I expect the models will continue improving though,
         | 
         | How? They've already been trained on all the code in the world
         | at this point, so that's a dead end.
         | 
         | The only other option I see is increasing the context window,
         | which has diminishing returns already (double the window for a
         | 10% increase in accuracy, for example).
         | 
         | We're in a local maxima here.
        
           | dcre wrote:
           | This makes no sense. Claude 3.7 Sonnet is better than Claude
           | 3.5 Sonnet and it's not because it's trained on more of the
           | world's code. The models are improving in a variety of ways,
           | whether by being larger, faster, using the same number of
           | parameters more effectively, better RLHF techniques, better
           | inference-time compute techniques, etc.
        
             | lelanthran wrote:
             | > The models are improving in a variety of ways, whether by
             | being larger, faster, using the same number of parameters
             | more effectively, better RLHF techniques, better inference-
             | time compute techniques, etc.
             | 
             | I didn't say they weren't improving.
             | 
             | I said there's diminishing returns.
             | 
             | There's been more effort put into LLMs in the last two
             | years than in the two years prior, but the gains in the
             | last two years have been much _much_ smaller than in the
             | two years prior.
             | 
             | That's what I meant by diminishing returns: the gains we
             | see are not proportional to the effort invested.
        
       | dataviz1000 wrote:
       | Are you using Cursor? I'm using Github Copilot in VSCode and I'm
       | wondering if I will get more efficiency from a different coding
       | assistant.
        
         | dsabanin wrote:
         | You will. Cursor is much further along on the journey of
         | building an actually powerful AI coding system. Since they are
         | smaller, they can afford to iterate more quickly and experiment
         | with a much tighter feedback loop.
        
         | diggan wrote:
         | I've tried Copilot, Aider and Cursor and the best I've found is
         | to just use the various chat interfaces. I sometimes throw
         | hundreds of lines straight in there, and the models seem to
         | understand the full context much better than any "LLM Editor"
         | I've tried so far. Then different models are good for different
         | things (obvious maybe). For example, O1 Pro is miles ahead any
         | models when it comes to overall architecture, R1 is great for
         | finding nasty bugs and Sonnet great for small and fast feature
         | additions/modifications with strict requirements.
        
         | ezyang wrote:
         | I have used Cursor and my own MCP codemcp. Cursor has a lot of
         | nice QoL that you can't get from an MCP package; the TAB is
         | really good for traditional coding. Haven't used copilot so I
         | don't have a comparison there. Definitely use agent mode.
        
         | hooloovoo_zoo wrote:
         | It doesn't matter. They're all thin layers on functionally
         | equivalent models. Stick with whatever text editor you prefer.
        
       | boredtofears wrote:
       | Great read, I can definitely confirm a lot of these myself. Would
       | be nice to see this aggregated into some kind of "best practices"
       | document (although hard to say how quickly it'd be out of date).
        
       | submeta wrote:
       | > Preparatory refactoring
       | 
       | > Current LLMs, without a plan that says they should refactor
       | first, don't decompose changes in this way. They will try to do
       | everything at once.
       | 
       | Just today I leaned the hard way. I had created an app for my
       | spouse and myself for sharing and reading news-articles, some of
       | them behind paywalls.
       | 
       | Using Cursor I have a FastAPI backend and a React frontend. When
       | I added extracting the article text in markdown and then
       | summarizing it, both using openai, and when I tasked Cursor with
       | it, the chaos began. Cursor (with the help of Claude 3.7) tackled
       | everything at once and some more. It started writing a module for
       | using openai, then it also changed the frontend to not only show
       | the title and url, but also the extracted markdown and the
       | summary, by doing that it screwed up my UI, deleted some rows in
       | my database, came up with as module for interacting with Openai
       | that did not work, the ectraction was screwed, the summary as
       | well.
       | 
       | All of this despite me having detailed cursorrules.
       | 
       | That's when I realized: Divide and conquer. Ask it to write one
       | function that workd, then one class where the function becomes a
       | method, test it, then move on to next function. Until every piece
       | is working and I can glue them together.
        
         | AustinDev wrote:
         | Use claude 3.5 if you have detailed instructions that you want
         | it to follow. I've found over many hours of using these models
         | that 3.7 loves to go off-script no matter how many rules you
         | provide.
        
         | pomatic wrote:
         | This is the way, small bite-sized pieces of the elephant.
         | Unfortunately it means you do need to understand programming
         | concepts, composition and to a lesser degree, architecture. On
         | the positive side - these are new tools, and we need to learn
         | how to work with them. They do have the power to nX times the
         | person who has a bit of knowledge and can also adapt to their
         | ways.
        
       | colonCapitalDee wrote:
       | > Preparatory Refactoring says that you should first refactor to
       | make a change easy, and then make the change. The refactor change
       | can be quite involved, but because it is semantics preserving, it
       | is easier to evaluate than the change itself.
       | 
       | > In human software engineering, a common antipattern when trying
       | to figure out what to do is to jump straight to proposing
       | solutions, without forcing everyone to clearly articulate what
       | all the requirements are. Often, your problem space is
       | constrained enough that once you write down all of the
       | requirements, the solution is uniquely determined; without the
       | requirements, it's easy to devolve into a haze of arguing over
       | particular solutions.
       | 
       | > When you're learning to use a new framework or library, simple
       | uses of the software can be done just by copy pasting code from
       | tutorials and tweaking them as necessary. But at some point, it's
       | a good idea to just slog through reading the docs from top-to-
       | bottom, to get a full understanding of what is and is not
       | possible in the software.
       | 
       | > The Walking Skeleton is the minimum, crappy implementation of
       | an end-to-end system that has all of the pieces you need. The
       | point is to get the end-to-end system working first, and only
       | then start improving the various pieces.
       | 
       | > When there is a bug, there are broadly two ways you can try to
       | fix it. One way is to randomly try things based on vibes and hope
       | you get lucky. The other is to systematically examine your
       | assumptions about how the system works and figure out where
       | reality mismatches your expectations.
       | 
       | > The Rule of Three in software says that you should be willing
       | to duplicate a piece of code once, but on the third copy you
       | should refactor. This is a refinement on DRY (Don't Repeat
       | Yourself) accounting for the fact that it might not necessarily
       | be obvious how to eliminate a duplication, and waiting until the
       | third occurrence might clarify.
       | 
       | These are lessons that I've learned the hard way (for some
       | definition of "learned", these things are simple but not easy),
       | but I've never seen them phrased to succinctly and accurately
       | before. Well done OP!
        
       | admiralrohan wrote:
       | Even in the age of Vibe coding, I always try to learn as much as
       | possible.
       | 
       | For example, yesterday I was working with the Animation library
       | Motion which I never worked earlier. I used the code suggested by
       | AI but at least picke 2-3 basic animation concepts while
       | reviewing the code.
       | 
       | Kind of unfocused passive learning I always tried even before AI.
        
         | worldsayshi wrote:
         | > Even in the age of Vibe coding, I always try to learn as much
         | as possible.
         | 
         | Even? It kind of has become easier than ever to learn new ways
         | to code? Just as it opens up building things that you
         | previously wouldn't because of time constraints, you can now
         | learn how to X in language Y in a few minutes instead of hours.
         | 
         | Although I suppose it may be easier than ever for the brain to
         | think that "I can look this up whenever so I might just forget
         | about it".
        
       | akomtu wrote:
       | LLMs aren't AI. They are more like librarians with eidetic
       | memory: they can discuss in depth any book in the library, but
       | sooner or later you notice that they don't really understand what
       | they are talking about.
       | 
       | One easy test for AI-ness is the optimization problem. Give it a
       | relatively small, but complex program, e.g. a GPU shader on
       | shadertoy.com, and tell it to optimize it. The output is clearly
       | defined: it's an image or an animation. It's also easy to test
       | how much it's improved the framerate. What's good is this task
       | won't allow the typical LLM bullshitting: if it doesn't compile
       | or doesn't draw a correct image, you'll see it.
       | 
       | The thing is, the current generation of LLMs will blunder at this
       | task.
        
         | ezyang wrote:
         | The thing is that, as many junior engineers can attest,
         | randomly blundering around can still give you something useful!
         | So you definitely can get value out of AI coding with the
         | current generation of models.
        
       | kleton wrote:
       | Most of the things are applicable to the current top models, but
       | he frequently references Claude sonnet, which is not even above
       | the fold on the leaderboard
        
       | lukev wrote:
       | This is exceptionally useful advice, and precisely the way we
       | should be talking about how to engage with LLMs when coding.
       | 
       | That said, I take issue with "Use Static Types".
       | 
       | I've actually had more success with Claude Code using Clojure
       | than I have Typescript (the other thing I tried.)
       | 
       | Clojure emphasizes small, pure functions, to a high degree.
       | Whereas (sometimes) fully understanding a strong type might
       | involve reading several files. If I'm really good with my
       | prompting to make sure that I have good example data for the
       | entity types at each boundary point, it feels like it does a
       | better job.
       | 
       | My intuition is that LLMs are fundamentally context-based, so
       | they are naturally suited to an emphasis on functions over pure
       | data, vs requiring understanding of a larger type/class hierarchy
       | to perform well.
       | 
       | But it took me a while to figure out how to build these prompts
       | and agent rules. A LLM programming in a dynamic language without
       | a human supervising the high-level code structure and data model
       | is a recipe for disaster.
        
       ___________________________________________________________________
       (page generated 2025-03-19 23:00 UTC)