[HN Gopher] What we learned copying all the best code assistants
___________________________________________________________________
What we learned copying all the best code assistants
Author : stevekrouse
Score : 69 points
Date : 2025-01-03 14:45 UTC (8 hours ago)
(HTM) web link (blog.val.town)
(TXT) w3m dump (blog.val.town)
| stevekrouse wrote:
| This post is the latest in a series about Townie, our AI
| assistant.
|
| Our first had a nice discussion on HN:
| https://blog.val.town/blog/codegen/
|
| The other posts in the series:
|
| - https://blog.val.town/blog/townie/
|
| - https://blog.val.town/blog/building-a-code-writing-robot/
| simonw wrote:
| It's really interesting seeing the progression here, integrating
| AI-assisted coding tools into something like Val Town is a great
| arena for exploring different patterns for this stuff.
|
| Worth checking out their Cerebras-powered demo too - LLMs at 2000
| tokens/second make applying proposed changes absurdly
| interactive: https://cerebrascoder.com/
| afro88 wrote:
| I'm interested in what stopped you from finishing diffs and diff
| based editing. I built an AI software engineering assistant at my
| last company and we got decent results with Aider's method (and
| prompts, and hidden conversation starter etc). I did have to have
| a fallback to raw output, and a way to ask it to try again. But
| for the most part it worked well and unlocked editing large files
| (and quickly).
| stevekrouse wrote:
| Excellent question! We just didn't have the resources at the
| time on our small team to invest in getting it to be good
| enough to be default on. We had to move on to other more core
| platform features.
|
| Though I'm really eager to get back to it. When using Windsurf
| last week, I was impressed by their diffs on Sonnet. Seems like
| they work well. I would _love_ to view their system prompt!
|
| I hope that when we have time to resume work on this (maybe in
| Feb) that we'll be able to get it done. But then again, maybe
| just patience (and more fast-following) is the right strategy,
| given how fast things are moving...
| simonw wrote:
| An interesting alternative to diffs appears to be
| straightforward find and replace.
|
| Claude Artifacts uses that: they have a tool where the LLM
| can say "replace this exact text with this" to update an
| Artifat without having to output the whole thing again.
|
| ChatGPT's new Canvas feature apparently does a more
| sophisticated version of that using regular expressions as
| opposed to simple text matching:
| https://twitter.com/sh_reya/status/1875227816993943823
| stevekrouse wrote:
| Very useful! Thank you!
| zamfi wrote:
| Oh that is super interesting! I wonder if they track how
| often it succeeds in matching and replacing, I'd love to
| see those numbers in aggregate.
|
| Total anecdote, but I worked on this for a bit for a
| research-level-code code editor (system paper to come soon,
| fingers crossed!) and found that basic find-and-replace was
| pretty brittle. I also had to be confident the source
| appears only once (not always the case for my use case),
| and there was a tradeoff of fuzziness of match / likelihood
| of perfectly correct source.
|
| But yeah, diffs are super hard because the format requires
| far context and accurate mathematical computation.
|
| Ultimately, the version of this that worked the best for me
| was a total hack:
|
| Prefix every line of the code with L#### -- the line
| number. Ask for diffs to be the original text and the
| complete replacement text _including_ the line number
| prefix on both original and replacement. Then, to apply,
| fuzzy match on both line number and context.
|
| I suspect this worked as well as it did because it
| transmutes the math and computation problems into pattern-
| matching and copying problems, which LLMs are (still) much
| better at these days.
| stevekrouse wrote:
| Yes, adding line numbers on each line is one of the ideas
| we've been considering trying. Thanks for the reminder!
| zamfi wrote:
| I suspect any other "hook" would work just as well, a
| comment with a nonce--and could serve as block boundaries
| to make changes more likely to be complete?
|
| Graphologue used a version of this too:
| https://hci.ucsd.edu/papers/graphologue.pdf
| reilly3000 wrote:
| I gotchu:
|
| https://www.reddit.com/r/LocalLLaMA/comments/1h7sjyt/windsur.
| ..
| stevekrouse wrote:
| wow, magic. thank you!
| furyofantares wrote:
| I wonder if you didn't try cursor's Composer tab, especially set
| to Agent?
|
| I didn't care that much for cursor when I was just using Chat but
| once I switched to Composer I was very happy, and my experience
| is in total disagreement that it's not so good for smaller
| projects.
|
| They also must have a good prompt for diff-based completions, I
| don't know how hard it is to extract that.
| stevekrouse wrote:
| Ok, good to know! I'll have to find time to try out Composer
|
| Yes, I wonder if reilly3000 will swing by with a leaked system
| prompt from them too
| fudged71 wrote:
| I really enjoyed Steve's guest appearance on Dan Shipper's AI & I
| https://open.spotify.com/episode/6hoiweWkAv84zYTC8DAi5B?si=C...
| stevekrouse wrote:
| Thanks!!
| viewhub wrote:
| Windsurf + Haskell w/ CLI tools has been pretty amazing.
| Windsurf's agent will loop for minutes on its own to figure out
| the right structure of a program. You just need to tell it to:
|
| - use the hoogle cli to search for the right types and functions
|
| - include a comprehensive test suite
|
| - run a build after every code change
|
| - run tests after every successful build
|
| GHC + a Claude-based agent is a thing to behold.
| stevekrouse wrote:
| Woah, that sounds awesome! I'd love to see how you set that up
| and how much it can do without your intervention/approval for
| various actions. Might you have a video of your workflow that
| you could share?
___________________________________________________________________
(page generated 2025-01-03 23:00 UTC)