[HN Gopher] My LLM codegen workflow
___________________________________________________________________
My LLM codegen workflow
Author : lolptdr
Score : 204 points
Date : 2025-02-18 19:33 UTC (3 hours ago)
(HTM) web link (harper.blog)
(TXT) w3m dump (harper.blog)
| runoisenze wrote:
| Great write up! Roughly how many Claude tokens are you using per
| month with this workflow? What's your monthly API costs?
|
| Also what do you mean by "I really want someone to solve this
| problem in a way that makes coding with an LLM a multiplayer
| game. Not a solo hacker experience." ?
| harper wrote:
| most of this is aider / codegen:
|
| Total tokens in: 26,729,994 Total tokens out: 1,553,284
|
| Last month anthropic bill was $89.30
|
| --
|
| I want to program with a team, together. not a group of people
| individually coding with an agent, and then managing merges. I
| have been playing a lot with merging team context - but haven't
| gotten too far yet.
| fragmede wrote:
| Have you tried using OpenHands so you can just give it the
| todo.md until it gets stuck/is finished?
| ggulati wrote:
| Nice, I coincidentally wrote a blog post today exploring
| workflows as well:
| https://ggulati.wordpress.com/2025/02/17/cursorai-for-fronte...
|
| Your workflow is much more polished, will definitely try it out
| for my next project
| harper wrote:
| let me know how it works!
| hnuser123456 wrote:
| Looks like your blog crashed, I've been wanting to read it
| fragmede wrote:
| > paste in prompt into claude copy and paste code from
| claude.ai into IDE
|
| is more polished? What's your workflow, banging rocks together?
| shoemakersteve wrote:
| This made me laugh audibly. Thank you.
| ggulati wrote:
| More or less, I tried out Cursor for the first time a week
| ago. So very much in the newbie stage and looking to learn
| sprobertson wrote:
| Strange capitalization of atm as ATM in the HN title, but great
| tips in there
| Philpax wrote:
| HN will implicitly modify the title, including uppercasing
| acronyms. Very possible this was one of those changes.
| jrexilius wrote:
| The first part of this, where you told it to ask YOU questions,
| rather than laboriously building prompts and context yourself was
| the magic ticket for me. And I doubt I would have stumbled on
| that sorta inverse logic on my own. Really great write up!
| danphilibin wrote:
| This is the key to a lot of my workflows as well. I'll usually
| tack some form of "ask me up to 5 questions to improve your
| understanding of what I'm trying to do here" onto the end of my
| initial messages. Over time I've noticed patterns in
| information I tend to leave out which has helped me improve my
| initial prompts, plus it often gets me thinking about aspects I
| hadn't considered yet.
| daxfohl wrote:
| Frankly getting used to doing this may help our communication
| with other engineers as well.
| rotcev wrote:
| This is the first article I've come across that truly utilizes
| LLMs in a workflow the right way. I appreciate the time and
| effort the author put into breaking this down.
|
| I believe most people who struggle to be productive with language
| models simply haven't put in the necessary practice to
| communicate effectively with AI. The issue isn't with the
| intelligence of the models--it's that humans are still learning
| how to use this tool properly. It's clear that the author has
| spent time mastering the art of communicating with LLMs. Many of
| the conclusions in this post feel obvious once you've developed
| an understanding of how these models "think" and how to work
| within their constraints.
|
| I'm a huge fan of the workflow described here, and I'll
| definitely be looking into AIder and repomix. I've had a lot of
| success using a similar approach with Cursor in Composer Agent
| mode, where Claude-3.5-sonnet acts as my "code implementer." I
| strategize with larger reasoning models (like o1-pro, o3-mini-
| high, etc.) and delegate execution to Claude, which excels at
| making inline code edits. While it's not perfect, the time
| savings far outweigh the effort required to review an "AI Pull
| Request."
|
| Maximizing efficiency in this kind of workflow requires a few key
| things:
|
| - High typing speed - Minimizing time spent writing prompts means
| maximizing time generating useful code.
|
| - A strong intuition for "what's right" vs. "what's wrong" - This
| will probably become less relevant as models improve, but for
| now, good judgment is crucial.
|
| - Familiarity with each model's strengths and weaknesses - This
| only comes with hands-on experience.
|
| Right now, LLMs don't work flawlessly out of the box for
| everyone, and I think that's where a lot of the complaints come
| from--the "AI haterade" crowd expects perfection without
| adaptation.
|
| For what it's worth, I've built large-scale production
| applications using these techniques while writing minimal human
| code myself.
|
| Most of my experience using these workflows has been in the web
| dev domain, where there's an abundance of training data. That
| said, I've also worked in lower-level programming and language
| design, so I can understand why some people might not find models
| up to par in every scenario, particularly in niche domains.
| brokencode wrote:
| > "I appreciate the time and effort the author put into
| breaking this down."
|
| Let's be honest. The author was probably playing cookie clicker
| while this article was being written.
| hooverd wrote:
| I think LLM codegen still requires a mental model of the problem
| domain. I wonder how many upcoming devs will simply never develop
| one. Calculators are tools for engineers /and/ way too many
| people can't even do basic receipt math.
| jack_pp wrote:
| Calculations are for calculators. I was good at math in school
| but now I struggle / take so much time doing receipt math and
| for what? What's the purpose of the time you spend doing it,
| when do you need to have your brain trained for this specific
| task?
| hooverd wrote:
| For me, being able to notice when you mess up your own
| calculations. It doesn't help that we teach arithmetic
| operations ass-backwards (smallest to largest instead of
| largest to smallest).
| rd wrote:
| Has anyone who evolved from a baseline of just using Cursor chat
| and freestyling to a proper workflow like this got any anecdata
| to share on noticeable improvements?
|
| Does the time invested into the planning benefit you? Have you
| noticed less hallucinations? Have you saved time overall?
|
| I'd be curious to hear because my current workflow is basically
|
| 1. Have idea
|
| 2. create-next-app + ShadCN + TailwindUI boilerplate
|
| 3. Cursor Composer on agent mode with Superwispr voice
| transcription
|
| I'm gonna try the author's workflow regardless, but would love to
| hear others opinions.
| cynicalpeace wrote:
| I'm wondering the same thing.
|
| Most of these workflows are just context management workflows
| and in Cursor it's so simple to manage the context.
|
| For large files I just highlight the code and cmd+L. For short
| files, I just add them all by using /+downarrow
|
| I constantly feed context like this and then usually come to a
| good solution for both legacy and greenfield features/products.
|
| If I don't come to a good solution it's almost always because I
| didn't think through my prompt well enough and/or I didn't
| provide the correct context.
| mike_hearn wrote:
| Aider + AI generated maps and user guides for internal modules
| has worked well for me. Just today I did my own version of a
| script that uses Gemini 2 Flash (1M context window) to generate
| maps of each module in my codebase, i.e. a short one or two
| sentence description of what's in every file. Aider's repo maps
| don't work well for me, so I disable them, and I think this
| will work better.
|
| I also have a scratchpad file that I tell the model it can
| update to reflect anything new it learns, so that gives it a
| crude form of memory as it works on the codebase. This does
| help it use internal utility APIs.
| ghuntley wrote:
| If you steer it and build a stdlib, you get better outcomes.
| See https://ghuntley.com/stdlib
| Isamu wrote:
| I'm curious, is adding "do not hallucinate" to prompts effective
| in preventing hallucinations? The author does this.
| krainboltgreene wrote:
| It seems absurd, but I suppose it's the same as misspelling
| with similar enough trigrams as to get the best autocorrect
| results.
| becquerel wrote:
| I don't know about this specific technique, but I have found it
| useful to add a line like 'it's OK if you don't know or this
| isn't possible' at the end of queries. Otherwise LLMs have a
| tendency to tilt at whatever windmill you give them. Managing
| tone and expectations with them is a subtle but important art.
| simonw wrote:
| Apple were using that in their Apple Intelligence system
| prompts last year, I don't know if they still have that in
| there. https://simonwillison.net/2024/Aug/6/apple-intelligence-
| prom...
|
| I have no idea if it works or not!
| bambax wrote:
| This is all fine for a solo dev, but how does this work with a
| team / squad, working on the same code base?
|
| Having 7 different instances of an LLM analyzing the same code
| base and making suggestions would not just be economically
| wasteful, it would also be unpractical or even dangerous?
|
| Outside of RAG, which is a different thing, are there products
| that somehow "centralize" the context for a team, where all
| questions refer to the same codebase?
| staindk wrote:
| I've only recently switched to Cursor so am not clued up about
| everything, but they mention that the embedded indexing they do
| on your code is shared with others (others who have access to
| that repository? Unsure).
|
| It did seem to take a while to index, even though my colleagues
| had been using Cursor for a while, so I'm likely
| misunderstanding something.
| jacooper wrote:
| I find making the LLMs think and plan the project a bit worrying,
| I understand this helps with procrastination but when these
| systems eventually get better and more integrated, the most
| likely thing to happen to software devs is them moving away from
| purely coding to more of a solution architect role (aka Planning
| stuff), not taking into account the negative impact of giving up
| critical thinking to LLMs.
|
| https://news.ycombinator.com/item?id=43057907
|
| Other than that a great article! Very insightful.
| maelito wrote:
| Given a 3 648 318 tokens repository (number from Repomix), I'm
| not sure what would be the cost of using a leading LLM to analyse
| it and ask improvements.
|
| Isn't the input token number way more limited than that ?
|
| This is part is unclear to me in the "non-Greenfield" part of the
| article.
|
| Iterating with aider on very limited scopes is easy, I've used it
| often. But what about understanding a whole repository and act on
| it ? Following imports to understand a Typescript codebase as a
| whole ?
| kridsdale3 wrote:
| Well, do you as a human have the whole codebase loaded in to
| your memory with the ability to mentally reason with it? No,
| you work on a small scope at a time.
| layer8 wrote:
| You may work in a limited scope at a time, but you are aware
| how it fits into the larger scope, and more often than not
| you actually have to connect things across different scopes.
| jack_pp wrote:
| Well you can use an LLM similarly. Have it write docs for
| all your files including a summary for each function /
| class, ideally in order of dependency. Then use only the
| summaries in context. This should significantly lower your
| token count.
|
| Haven't tried it personally but it should work
| lanthissa wrote:
| you do the same thing with the llm, you have it describe
| the api of modules not related to your code and that in
| place of those segments of the code.
| rollinDyno wrote:
| Something I quickly learned while retooling this past week is
| that it's preferable not to add opinionated frameworks to the
| project as they increase the size of the context the model should
| be aware of. This context will also not likely be available in
| the training data.
|
| For example, rather than using Plasmo for its browser extension
| boilerplate and packaging utilities, I've chosen to ask the LLM
| to setup all of that for me as it won't have any blindspots when
| tasked with debugging.
| sampton wrote:
| The end of artisan frameworks - probably for the better.
| balls187 wrote:
| It's likely the end of a lot of abstractions that made
| programming easier.
|
| At some point, specialized code-gen transformer models should
| get really good at just spitting out the lowest level code
| required to perform the job.
| zackify wrote:
| Cline over everything for me
| dfltr wrote:
| > Legacy modern code
|
| As opposed to Vintage Pioneer code?
| thedeep_mind wrote:
| This is effing great...thanks for sharing your experience.
|
| I was just wondering how to give my edits back to in-browser
| tools like Claude or ChatGPT, but the idea of repo mix is great,
| will try!
|
| Although I have been flying bit with copilot in vscode, so right
| now I have essentially two AI, one for larger changes (in the
| browser), and then minor code fixes (in vscode).
| snowwrestler wrote:
| Spelling nit:
|
| "Over my skis" ~ in over my head.
|
| "Over my skies" ~ very far overhead. In orbit maybe?
| bionhoward wrote:
| I don't mind LLMs, but what irks me is the customer noncompete,
| you have these systems that can do almost anything and the legal
| terms explicitly say you're not allowed to use the thing for
| anything that competes with the thing. But if the things can do
| almost anything then you really can't use it for anything. Making
| a game with Grok? No, that competes with the xAI game studio.
| Making an agents framework with ChatGPT? No, that competes with
| Swarm. Making legal AI with Claude? No, that competes with
| Claude. Seems like the only American companies making AI we can
| actually use for work are HuggingFace and Meta.
| thornewolf wrote:
| Ignore the noncompetes. Never get sued. If you do, everyone
| else is on your side.
| bionhoward wrote:
| Meh, why pay to teach someone else's bot? I'm sticking with
| open source
| biddit wrote:
| Form a Nonprofit X and a Corp Y:
|
| Noprofit X publishes outputs from competing AI, which is not
| copyrightable.
|
| Corp Y injests content published by Nonprofit X.
| avandekleut wrote:
| This is pretty much my flow that I landed on as well. Dump
| existing relevant files into context, explain what we are trying
| to achieve, and ask it to analyze various approaches,
| considerations, and ask clarifying questions. Once we both align
| on direction, I ask for a plan of all files to be
| created/modified in dependency order with descriptions of the
| required changes. Once we align on the plan I say lets proceed
| one file at a time, that way I can ensure each file builds on the
| previous one and I can adjust as needed.
| fullstackwife wrote:
| Looks similar to my experience, except this part:
|
| > if it doesn't work, Q&A with aider to fix
|
| I fix errors myself, because LLMs are capable of producing large
| chunks of really stupid/wrong code, which needs to be reverted,
| and thats why it makes sense to see the code at least once.
|
| Also I used to find myself in a situation when I tried to use LLM
| for the sake of using LLM to write code (waste of time)
___________________________________________________________________
(page generated 2025-02-18 23:00 UTC)