[HN Gopher] Launch HN: Codebuff (YC F24) - CLI tool that writes ...
___________________________________________________________________
Launch HN: Codebuff (YC F24) - CLI tool that writes code for you
Hey HN! We're James and Brandon building Codebuff
(https://codebuff.com). Codebuff is like Cursor Composer, but in
your terminal: it modifies files based on your natural language
requests. You can try it with `npm i -g codebuff` and start using
it immediately for free. We have no login gate, and we give all
accounts up to $20 worth of credits. Codebuff is different because
we simplified the input to one step: you type what you want done in
your terminal and hit enter. Then Codebuff looks through your whole
codebase and makes the edits it wants, to existing source files or
new ones. It also can run your tests, the type checker, or install
packages to fulfill your request. Demo video:
https://www.youtube.com/watch?v=dQ0NOMsu0dA It all started at a
hackathon. I was trying out Sonnet 3.5 which had recently come out
and seeing if I could use it to write code. The script I cobbled
together that day pulled codebase context in one step and used it
to rewrite files with changes in the second step. This two step
process still exists today. Incidentally, my hackathon script
worked rather poorly and my demo failed to produce any useful code.
But that weekend I thought about the kind of errors it made, and
realized that with more context on our codebase, it might have been
able to get the change right. For example, it tried to create an
endpoint on our server (at my previous startup), but it didn't know
that you needed to edit 3 specific files to do this (yeah... our
backend was not that clean). So I hand-wrote a guide to our
codebase, like I was instructing a new hire. I put it in a markdown
file and passed it into Sonnet 3.5's system prompt. And the crazy
thing is that it started producing wayyy better code. So, I started
getting excited. In fact, this code guide idea still exists in
Codebuff today as knowledge.md files which are automatically read
on every request. I didn't think of this project as a startup idea
at first. I thought it was just a simple script anyone could write.
But after another week, I could see there were more problems to
solve and it should be a product. In the week between applying to
YC and the interview, I could not get Codebuff to edit files
consistently. I tried many prompting strategies to get it to
replace strings in the original file, but nothing worked reliably.
How could I face my interviewer if I could not get something basic
like this to work? On the day before my interview, in a Hail Mary
attempt, I fine-tuned GPT-4o to turn Claude's sketch of changes
into a git patch, which would add and remove lines to make the
edits. I only finished generating the training data late at night,
and the fine-tuning job ran as I slept. And, holy hell, the next
morning it worked! I pushed it to production just in time for my YC
interview with Dalton. Soon after, Brandon joined and we were off
to the races. So, how does Codebuff work exactly? You invoke it in
your terminal, and it starts by running through the source files in
that directory and subdirectories and parsing out all the function
and class names (or equivalents in 11 languages). We use the tree-
sitter library to do this. It builds out a codebase map that
includes these symbols and the file tree. Then, it fires off a
request to Claude Haiku 3.5 to cache this codebase context so user
inputs can be responded to with lower latency. (Prompt caching is
OP!). We have a stateless server that passes messages along to
Anthropic or OpenAI. We use websockets to ferry data back and forth
to clients. We didn't have authentication or even a database for
the first three months. Codebuff was free to install and used our
API keys for all requests. Luckily, no one exploited us for too
much free Claude usage haha. Major thanks to Brandon for saving
this situation by building out our database (Postgres + Drizzle),
server (Bun, hosted on Render, auth (using the free Auth.js),
website (NextJS also hosted on Render), billing (Stripe), logging
(BetterStack), and dashboard (Retool). This is the best tech stack
I've ever had. When the user sends an input message, we prompt
Claude to pick files that would be relevant (step 1). After picking
files, we load them into context and the agent responds. It invokes
tools using xml tags that we parse. It literally writes out
<edit_file path="src/app.ts">...</edit_file> to edit a particular
file, and has other tags to run terminal commands, or to ask to
read more files. This is all we really need, since Anthropic has
already trained Claude with very similar tools reach state of the
art on the SWE benchmark. Codebuff has limited free usage, but if
you like it you can pay $99/mo to get more credits. We realize this
is a lot more than competitors, but that's because we do more
expensive LLM calls with more context. We're already seeing
Codebuff used in surprising ways. One user racked up a $500 bill by
building out two Flutter apps in parallel. He never even looked at
the code it generated. Instead, he had long conversations with
Codebuff to make progress and fix errors, until the apps were built
to his satisfaction. Many users built real apps over a weekend for
their teams and personal use. Of course, those aren't the typical
use cases. Users also frequently use Codebuff to write unit tests.
They would build a feature in parallel with unit tests and have
Codebuff do loops to fix up the code until the tests pass. They
would also ask it to do drudge work like set up Oauth flows or API
scaffolding. What's really exciting with all of these examples is
that we're seeing people's creativity becoming unbridled. They're
spending more of their time thinking about architecture and design,
instead of implementation details. It's so cool that we're just at
the beginning, and the technology is only going to improve from
here. If you would want to use Codebuff inside your own systems,
we have an alpha SDK that exposes the same natural language
interface for your apps to call and receive code edits! You can
sign up here for early access:
https://codebuff.retool.com/form/c8b15919-52d0-4572-aca5-533....
Thank you for reading! We're excited for you to try out Codebuff
and let us know what you think!
Author : jahooma
Score : 157 points
Date : 2024-11-07 17:06 UTC (5 hours ago)
| brandonchen wrote:
| whooooot! it's been a wild ride thus far, but we've been super
| thrilled at how people are using it and can't wait for you all to
| try it out!
|
| we've seen our own productivity increase tenfold - using codebuff
| to build buff our own code hah
|
| let us know what you think!
| palebone wrote:
| I'm a big fan! It's better than cursor in many ways
| handfuloflight wrote:
| Could you elaborate on those ways please?
| hiatus wrote:
| Your comment is a bit suspicious given that your previous
| submissions are limited to manifold market links, and this tool
| came from that company.
| jahooma wrote:
| Yes, he's the lead Manifold eng. Please discount
| appropriately.
| loondri wrote:
| This is much needed! Gonna try this out. I haven't seen a good
| tool that lets me generate code via CLI.
| jc4883 wrote:
| The ergonomics of using unit tests + this to pass said unit tests
| is actually pretty good. Just tried it.
| shardool97 wrote:
| Very excited for codebuff, its been a huge productivity boost for
| me! I've been putting it to use on a monorepo that has Go,
| Typescript, terraform and some sql and it always looks at the
| right files for the task. I like the UX way better than cursor -
| I like reviewing all changes at once and making minor tweaks when
| necessary. Especially for writing Go, i love being able to stick
| with Goland IDE while using codebuff.
| brandonchen wrote:
| Thanks for being one of our early users and dealing with our
| bugs! I love that we can fit into so many developers' workflows
| and support them where _they are_, as opposed to forcing them
| to use us for everything.
| anonzzzies wrote:
| Comparison with Aider?
| jahooma wrote:
| Great question!
|
| In Codebuff you don't have to manually specify any files. It
| finds the right ones for you! It also pulls more files to get
| you a better result. I think this makes a huge difference in
| the ergonomics of just chatting to get results.
|
| Codebuff also will run commands directly, so you can ask it to
| write unit tests and run them as it goes to make sure they are
| working.
| handfuloflight wrote:
| > In Codebuff you don't have to manually specify any files.
|
| Alright, I'm in.
| anonzzzies wrote:
| Ah thanks, that's excellent! That is a massive issue for
| Aider; it was supposed to be solved, but last I tried I still
| had to do that manually.
|
| Nice work!
| nprateem wrote:
| That's why I like aider tbh. I know it's not going nuts on my
| repo.
| evnix wrote:
| I think Aider does this to save tokens/money. It supports a
| lot of models so you can have Claude as your architect and
| another cheap model that does the coding.
| jahooma wrote:
| Yup, there's a tradeoff in $$$, but for a lot of people it
| should be worth it, since Codebuff can find more relevant
| files with example code that will make the output higher
| quality.
| TeMPOraL wrote:
| Aider does all of this too, and it has for quite a while. It
| just tends to ask you for explicit permission when e.g.
| adding files to the context (potentially token-expensive),
| creating new files, or running commands (potentially
| dangerous); AFAIR there's an option (both configuration and
| cli arg) to auto-approve those requests, though I never tried
| it.
|
| Aider has extensive code for computing "repository map", with
| specialized handling for many programming languages; that map
| is sent to LLM to give it an overview of the project
| structure and summary of files it might be interested in. It
| is indeed a very convenient feature.
|
| I never tried writing and launching unit tests via Aider, but
| from what I remember from the docs, it should work out of the
| box too.
| dakshgupta wrote:
| Love the demo video! Three quick questions:
|
| Any specific reason to choose the terminal as the interface? Do
| you plan to make it more extensible in the future? (sounds like
| this could be wrapped with an extension for any IDE, which is
| exciting)
|
| Also, do you see it being a problem that you can't point it to
| specific lines of code? In Cursor you can select some lines and
| CMD+K to instruct an edit. This takes away that fidelity, is it
| because you suspect models will get good enough to not require
| that level of handholding?
|
| Do you plan to benchmark this with swe-bench etc.?
| jahooma wrote:
| We thought about making a VSCode extension/fork like everyone
| else, but decided that the future is coding agents that do most
| of the work for you.
|
| The terminal is actually a great interface because it is so
| simple. It keeps the product focused to not have complex UI
| options. But also, we rarely thought we needed any options.
| It's enough to let the user say what they want in chat.
|
| You can't point to specific lines, but Codebuff is really good
| at finding the right spot.
|
| I actually still use Cursor to edit individual files because I
| feel it is better when you are manually coding and want to
| change just one thing there.
|
| We do plan to do the SWE bench. It's mostly the new Sonnet 3.5
| under the hood making the edits, so it should do about as well
| as Anthropic's benchmark for that, which is really high, 49%:
| https://www.anthropic.com/news/3-5-models-and-computer-use
|
| Fun fact is that the new Sonnet was given two tools to do code
| edits and run terminal commands to reach this high score.
| That's pretty much what Codebuff does.
| brandonchen wrote:
| To add on, I know a lot of people see the terminal as
| cruft/legacy from the mainframe days. But it is a funny thing
| to look at tons of people's IDE setup and see that the one
| _consistent_ thing between them all is that they have a
| terminal nearby. It makes sense, too, since programs run in
| the terminal and you can only abstract so much to developers.
| And like James said, this sets us up nicely to build for a
| future of coding agents running around. Feels like a unique
| insight, but I dunno. I guess time will tell.
| photonthug wrote:
| > I know a lot of people see the terminal as cruft/legacy
| from the mainframe days.
|
| Hah. If you encounter people that think like this, run away
| because as soon as they finish telling you that terminals
| are stupid they inevitably want help configuring their GUI
| for k8s or git. After that, with or without a GUI, it turns
| out they also don't understand version control /
| containers.
| ndyg wrote:
| Noting Codebuff is manicode renamed.
|
| It's become my go-to tool for handling fiddly refactors. Here's
| an example session from a Rust project where I used it to break a
| single file into a module directory.
|
| https://gist.github.com/cablehead/f235d61d3b646f2ec1794f656e...
|
| Notice how it can run tests, see the compile error, and then
| iterate until the task is done? Really impressive.
|
| For reference, this task used ~100 credits
| brandonchen wrote:
| Haha yes, here's the story of why we rebranded:
| https://manifold.markets/JamesGrugett/what-will-we-rename-ma...
|
| Thanks for sharing! haxton was asking about practical use
| cases, I'll link them here!
| hiatus wrote:
| Why is there stuff for Manifold Markets in the distributed
| package?
|
| /codebuff/dist/manifold-api.js
|
| https://www.npmjs.com/package/codebuff?activeTab=code
| jahooma wrote:
| Haha, it's because I used to work on Manifold Markets!
|
| Codebuff was originally called Manicode. We just renamed it
| this week actually.
|
| There was meant to be a universe of "Mani" products. My other
| cofounder made Manifund, and there's a conference we made
| called Manifest!
| drdrey wrote:
| but that doesn't answer the question? are these test files?
| jahooma wrote:
| Oh, it was a tool call I originally implemented so that
| Codebuff could look up the probabilities of markets to help
| it answer user questions.
|
| I thought it would be fun if you asked it about the chance
| of the election or maybe something about AI capabilities,
| it could back up the answer by citing a prediction market.
| pama wrote:
| Why not simply remove that dependency now?
| brandonchen wrote:
| Cruft that built up over time But you make a good point,
| I'll write a ticket to remove it soon.
| handfuloflight wrote:
| Codebuff waiting in the wings to implement.
| Lord_Zero wrote:
| Cause they wrote codebuff using codebuff.
| nathan_young wrote:
| They came out of Manifold. Though I acknowledge that doesn't
| really answer your question.
| ishbaid wrote:
| Looks awesome! Great work team.
| handfuloflight wrote:
| > One user racked up a $500 bill by building out two Flutter apps
| in parallel.
|
| Is that through the Enterprise plan?
| jahooma wrote:
| Nope, if you go over the allotted credits on the $99 plan, then
| you pay per usage (with a 5% discount).
|
| We actually ended up not charging this guy since there was a
| bug where we told him he got 50,000 credits instead of 10,000.
| Oops!
| handfuloflight wrote:
| Can you speak more to how efficiency towards context
| management works (to reduce token costs)? Or are you loading
| up context to the brim with each request?
| jahooma wrote:
| I think managing context is the most important aspect of
| today's coding agents. We pick only files we think would be
| relevant to the user request and add those. We generally
| pull more files than Cursor, which I think is an advantage.
|
| However, we also try to leverage prompt-caching as much as
| possible to lower costs and improve latency.
|
| So we basically only add files over time. Once context gets
| too large, it will purge them all and start again.
| handfuloflight wrote:
| > However, we also try to leverage prompt-caching as much
| as possible to lower costs and improve latency.
|
| Interesting! That does have 5 minute expiry on Claude,
| and your users can use Codebuff in an unoptimal way. Do
| you have plans in aligning your users towards using the
| tool in a way that makes the most use of prompt caches?
| jahooma wrote:
| That's a really great point. Since we manage the context,
| we should clear the old files if it's been > 5 minutes.
| Thanks for the idea!
| hsbauauvhabzb wrote:
| How did that bug occur? Was the code generated by your code
| generator?
| dnsbty wrote:
| I've been using Codebuff for the last few weeks, and it's been
| really nice for working in my Elixir repo. And as someone who
| uses Neovim in the terminal instead of VS Code, it's nice to
| actually be able to have it live in the tmux split beside Neovim
| instead of having to switch to a different editor.
|
| I have noticed some small oddities, like every now and then it
| will remove the existing contents of a module when adding a new
| function, but between a quick glance over the changes using the
| diff command and our standard CI suite, it's always pretty easy
| to catch and fix.
| jahooma wrote:
| Thanks for using Codebuff! Yeah, these edit issues are
| annoying, but I'm confident we can reduce the error rate a lot
| in the coming weeks.
| haxton wrote:
| The demos I see for these types of tools are always some toy
| project and doesn't reflect day to day work I do at all. Do you
| have any example PRs on larger more complex projects that have
| been written with codebuff and how much of that was human
| interactive?
|
| The real problem I want someone to solve is helping me with the
| real niche/challenging portion of a PR, ex: new tiptap extension
| that can do notebook code eval, migrate legacy auth service off
| auth0, record and replay API GET requests and replay a % of them
| as unit tests, etc.
|
| So many of these tools get stuck trying to help me "start" rather
| than help me "finish" or unblock the current problem I'm at.
| jahooma wrote:
| I hear you. This is actually a foundational idea for Codebuff.
| I made it to work within the large-ish codebase of my previous
| startup, Manifold Markets.
|
| I want the demos to be of real work, but somehow they never
| seem as cool unless it's a neat front end toy example.
|
| Here is the demo video I sent in my application to YC, which
| shows it doing real stuff:
| https://www.loom.com/share/fd4bced4eff94095a09c6a19b7f7f45c?...
| SpaghettiX wrote:
| Absolutely! Imaging setting a bunch of css styles through a
| long winded AI conversation, when you could have an IDE to do
| it in a few seconds. I don't need that.
|
| The long tail of niche engineering problems is the time
| consuming bit now. That's not being solved at all, IMHO.
| bung wrote:
| > ... setting a bunch of css styles through a long winded AI
| conversation
|
| Any links on this topic you rate/could share?
| amethystcookie wrote:
| It's pretty good for complex projects imo because codebuff can
| understand the structure of your codebase and which files to
| change to implement changes. It still struggles when there
| isn't good documentation, but it has helped me finish a number
| of projects
| handfuloflight wrote:
| > It still struggles when there isn't good documentation
|
| @Codebuff team, does it make sense to provide a
| documentation.md with exposition on the systems?
| amethystcookie wrote:
| you can put the information into knowledge.md or
| [description].knowledge.md, but sometimes I can't find
| documentation and we're both learning as we go lmao
| jahooma wrote:
| One cool thing you can do is a ask Codebuff to create these
| docs. In fact, we recommend it.
|
| Codebuff natively reads any files ending in "knowledge.md",
| so you can add any extra info you want it to know to these
| files.
|
| For example, to make sure Codebuff creates new endpoints
| properly, I wrote a short guide with an example on the
| three files you need to update, and put it in
| backend/api/knowledge.md. After that, Codebuff always
| create new endpoints correctly!
| craigds wrote:
| +1; Ideally I want a tool I don't have to specify the context
| for. If I can point it via config files at my medium-sized
| codebase once (~2000 py files; 300k LOC according to `cloc`)
| then it starts to get actually usable.
|
| Cursor Composer doesn't handle that and seems geared towards a
| small handful of handpicked files.
|
| Would codebuff be able to handle a proper sized codebase? Or do
| the models fundamentally not handle that much context?
| jahooma wrote:
| Yes. Natively, the models are limited to 200k tokens which is
| on the order of dozens of files, which is way too small.
|
| But Codebuff has a whole preliminary step where it searches
| your codebase to find relevant files to your query, and only
| those get added to the coding agent's context.
|
| That's why I think it should work up to medium-large
| codebases. If the codebase is too large, then our file-
| finding step will also start to fail.
|
| I would give it a shot on your codebase. I think it should
| work.
| cratermoon wrote:
| RAG is a well-known technique now, and to paraphrase Emily
| Bender[1], here are some reasons why it's not a solution.
|
| The code extruded from the LLM is still synthetic code, and
| likely to contain errors both in the form of extra tokens
| motivated by the pre-training data for the LLM rather than
| the input texts AND in the form of omission. It's difficult
| to detect when the summary you are relying on is actually
| missing critical information.
|
| Even if the set up includes the links to the retrieved
| documents, the presence of the generated code discourages
| users from actually drilling down and reading them.
|
| This is still a framing that says: Your question has an
| answer, and the computer can give it to you.
|
| 1 https://buttondown.com/maiht3k/archive/information-
| literacy-...
| jahooma wrote:
| We actually don't use RAG! It's not that good as you say.
|
| We build a description of the codebase including the file
| tree and parsed function names and class names, and then
| just ask Haiku which files are relevant!
|
| This works much better and doesn't require slowly
| creating an index. You can just run Codebuff in any
| directory and it works.
| michaelmior wrote:
| It sounds like it's arguably still a form of RAG, just
| where the retrieval is very different. I'm not saying
| that to knock your approach, just saying that it sounds
| like it's still the case where you're retrieving some
| context and then using that context to augment further
| generation. (I get that's definitely not what people
| think of when you say RAG though.)
| brandonchen wrote:
| Genuine question: at what point does the term RAG lose
| its meaning? Seems like LLMs work best when they have the
| right context, and that context must be pulled from
| somewhere for the LLM. But if that's RAG, then what
| isn't? Do you have a take on this? Been struggling to
| frame all this in my head, so would love some insight.
| petesergeant wrote:
| Not RAG: asking the LLM to generate using its internal
| weights only
|
| RAG: providing the LLM with contextual data you've pulled
| from outside its weights that you believe relate to a
| query
| brandonchen wrote:
| Nice, super simple. We're definitely fitting into this
| definition of RAG then!
| parsimo2010 wrote:
| RAG is a search step in an attempt to put relevant
| context into a prompt before performing inference. You
| are "augmenting" the prompt by "retrieving" information
| from a data set before giving it to an LLM to "generate"
| a response. The data set may be the internet, or a code
| base, or text files. The typical examples online uses an
| embedding model and a vector database for the search
| step, but doing a web query before inference is also RAG.
| Perplexity.ai is a RAG (but fairly good quality). I would
| argue that Codebuff's directory tree search to find
| relevant files is a search step. It's not the same as a
| similarity search on vector embeddings, and it's not
| PageRank, but it is a search step.
|
| Things that aren't RAG, but are also ways to get a LLM to
| "know" things that it didn't know prior:
|
| 1. Fine-tuning with your custom training data, since it
| modifies the model weights instead of adding context. 2.
| LoRA with your custom training data, since it adds a few
| layers on top of a foundation model. 3. Stuffing all your
| context into the prompt, since there is no search step
| being performed.
| brandonchen wrote:
| Gotcha - so broadly encompasses how we give external
| context to the LLM. Appreciate the extra note about
| vector databases, that's where I've heard this term used
| most, but I'm glad to know it extends beyond that. Thanks
| for explaining!
| petesergeant wrote:
| > We build a description of the codebase including the
| file tree and parsed function names and class names
|
| This sounds like RAG and also that you're building an
| index? Did you just mean that you're not using vector
| search over embeddings for the retrieval part, or have I
| missed something fundamental here?
| jahooma wrote:
| Ah yeah, that's what I mean! I thought RAG is synonymous
| with this vector search approach.
|
| Either way, we do the search step a little different and
| it works well.
| asattarmd wrote:
| What's the fundamental limitation to context size here? Why
| can't a model be fine-tuned per codebase, taking the entire
| code into context (and be continuously trained as it's
| updated)?
|
| Forgive my naivety, I don't now anything about LLMs.
| craigds wrote:
| I'll need to get approval to use this on that codebase.
| I've tried it out on a smaller open-source codebase as a
| first step.
|
| For anyone interested: - here's the
| Codebuff session: https://gist.github.com/craigds/b51bbd1aa
| 19f2725c8276c5ad36947e2 - The result was this PR:
| https://github.com/koordinates/kart/pull/1011
|
| It required a bit of back and forth to produce a relatively
| small change, and I think it was a bit too narrow with the
| files it selected (it missed updating the implementations
| of a method in some subclasses, since it didn't look at
| those files)
|
| So I'm not sure if this saved me time, but it's
| nevertheless promising! I'm looking forward to what it will
| be capable of in 6mo.
| brandonchen wrote:
| Great question - we struggled for a long time to put our demo
| together precisely for this reason. Codebuff is so useful in a
| practical setting, but we can't bore the audience with a ton of
| background on a codebase when we do demos, so we have to pick a
| toy project. Maybe in the future, we could start our demo with
| a half-built project?
|
| Hopefully the demo on our homepage shows a little bit more of
| your day-to-day workflows than other codegen tools show, but
| we're all ears on ways to improve this!
|
| To give a concrete example of usefulness, I was implementing a
| referrals feature in Drizzle a few weeks ago, and Codebuff was
| able to build out the cli app, frontend, backend, and set up db
| schema (under my supervision, of course!) because of its deep
| understanding of our codebase. Building the feature properly
| requires knowing how our systems intersect with one another and
| the right abstraction at each point. I was able to bounce back
| and forth with it to build this out. It felt akin to working
| with a great junior engineer, tbh!
|
| EDIT: another user shared their use cases here!
| https://news.ycombinator.com/item?id=42079914
| cratermoon wrote:
| Kind of like "please describe the solution and I will write
| code to do it". That's not how programming works. Writing code
| and testing it against expectations to get to the solution,
| that's programming.
| mathgeek wrote:
| Language is important here. Programming, at its basic
| definition, is just writing code that programs a machine.
| Software development or even design/engineering are closer to
| what you're referring to.
| brandonchen wrote:
| FWIW I don't find that I'm losing good engineering
| habits/thought processes. Codebuff is not at the stage where
| I'm comfortable accepting its work without reviewing, so I
| catch bugs it introduces or edge cases it's missed. The main
| difference for me is the speed at which I can build now.
| Instead of fussing over exact syntax or which package does
| what, I can keep my focus on the broader implications of a
| particular architecture or nuances of components, etc.
|
| I will admit, however, that my context switching has
| increased a ton, and that's probably not great. I often tell
| Codebuff to do something, inevitably get distracted with
| something else, and then come back later barely remembering
| the original task
| Aeolun wrote:
| > ex: new tiptap extension that can do notebook code eval
|
| Claude wrote me a prosemirror extension doing a bunch of stuff
| that I couldn't figure out how to do myself. It was very
| convenient.
| nseth wrote:
| I've been playing with Codebuff for a few days (building out some
| services with Node.js + Typescript) - been working beautifully!
| Feels like I'm watching a skilled surgeon at work.
| brandonchen wrote:
| A skilled surgeon is a great analogy! We actually instruct
| Codebuff to focus on making the most minimal edits, so that it
| does precisely what you want.
| boratanrikulu wrote:
| Allowing LLMs to execute unrestricted commands without human
| review is risky and insecure.
| codenamev wrote:
| You are really missing out: https://github.com/e2b-dev/e2b
| boratanrikulu wrote:
| I don't see any sandbox usage in the demo video.
| jahooma wrote:
| Yes, this is a good point. I think not asking to run commands
| is maybe the most controversial choice we've made so far.
|
| The reason we don't ask for human review is simply: we've found
| that it works fine to not ask.
|
| We've had a few hundred users so far and usually people are
| skeptical of this at first, but as they use it they find that
| they don't want it to ask for every command. It enables cool
| use cases where Codebuff and iterate by running tests, seeing
| the error, attempting a fix, and running them again.
|
| If you use source control like git, I also think that it's very
| hard for things to go wrong. Even if it ran rm -rf from your
| project directory, you should be able to undo that.
|
| But here's the other thing: it won't do that. Claude is trained
| to be careful about this stuff and we've further prompted it to
| be careful.
|
| I think not asking to run commands is the future of coding
| agents, so I hope you will at least entertain this idea. It's
| ok if you don't want to trust it, we're not asking you to do
| anything you are uncomfortable with.
| boratanrikulu wrote:
| > it won't do that. Claude is trained to be careful about
| this stuff and we've further prompted it to be careful.
|
| Could you please explain a bit how you are sure about it?
| jahooma wrote:
| It's mainly from experience. From when I set it up I didn't
| have the feature to ask whether to run commands. It has
| been rawdogging commands this whole time and has never been
| a problem for me.
|
| I think we have many other users who are similar. To be
| fair, sometimes after watching it install packages with
| npm, people are surprised and say that they would have
| preferred that it asked. But usually this is just the
| initial reaction. I'm pretty confident this is the way
| forward.
| boratanrikulu wrote:
| Do you have any sandbox-like restrictions in place to
| ensure that commands are limited to only touching the
| project folder not any other places in the system?
| jahooma wrote:
| We always reset the directory back to the project
| directory on each command, so that helps.
|
| But we're open to adding more restrictions so that it
| can't for example run `cd /usr && rm -rf .`
| ATechGuy wrote:
| How about executing commands in a VM (perhaps
| Firecracker)?
| israrkhan wrote:
| I am not afraid of rm -rf whole directory. I am afraid of
| other stuff that it can do to my machines. leak my ssh keys,
| cookies, persnal data, network devices, and making persistent
| modifications (malware) to my system. Or maybe inadvertently
| messing with my python version, or globally installing some
| library to mess up whole system.
| Retr0id wrote:
| I, as a well-intending human, have run commands that broke
| my local python install. At least I was vaguely aware of
| what I did, and was able to fix things. If I didn't know
| what had happened I'd be pretty lost.
| YetAnotherNick wrote:
| It's strange that all the closed models whose mentioned reasons
| for being closed is safety is allowing this, and banning the
| apps which allows for erotic roleplay all the time. Roleplay is
| significantly less dangerous than full shell control.
| gmaster1440 wrote:
| Does Codebuff / the tree sitter implementation support Svelte?
| jahooma wrote:
| Yes, at least, partially. It will work, but maybe not as well
| as we don't parse out the function names from .svelte files.
|
| I can add it if tree sitter adds support for Svelte. I haven't
| checked, maybe it already is supported?
| iLoveOncall wrote:
| Your website has a serious issue. Trying to play the YouTube
| video makes the page slow down to a crawl, even in 1080p, while
| playing it on YouTube directly has no issue, even in 4K.
|
| On the project itself, I don't really find it exciting at all,
| I'm sorry. It's just another wrapper for a 3rd party model, and
| the fact that you can 1) describe the entire workflow in 3
| paragraphs, and 2) built it and launched it in around 4 months,
| emphasizes that.
|
| Congrats on launch I guess.
| brandonchen wrote:
| Weird, thanks for flagging - we're just using a Youtube embed
| in an iframe but I'll take a look.
|
| No worries if this isn't a good fit for you. You're welcome to
| try it out for free anytime if you change your mind!
|
| FWIW I wasn't super excited when James first showed me the
| project. I had tried so many AI code editors before, but never
| found them to be _actually usable_. So when James asked me to
| try, I just thought I'd be humoring him. Once I gave it a real
| shot, I found Codebuff to be great because of its form factor
| and deep context awareness: CLI allows for portability and
| system integration that plugins or extensions really can't do.
| And when AI actually understands my codebase, I just get a lot
| more done.
|
| Not trying to convince you to change your mind, just sharing
| that I was in your shoes not too long ago!
| iLoveOncall wrote:
| I would really rethink your value proposition.
|
| > CLI allows for portability and system integration that
| plugins or extensions really can't do
|
| In the past 6 or 7 years I haven't written a single line of
| code outside of a JetBrains IDE. Same thing for all of my
| team (whether they use JetBrains IDEs or VS Code), and I
| imagine for the vast majority of developers.
|
| This is not a convincing argument for the vast majority of
| people. If anything, the fact that it requires a tool OUTSIDE
| of where they write code is an inconvenience.
|
| > And when AI actually understands my codebase, I just get a
| lot more done.
|
| But Amazon Q does this without me needing to type anything to
| instruct it, or to tell it which files to look at. And,
| again, without needing to go out of my IDE.
|
| Having to switch to a new tool to write code using AI is a
| huge deterrent and asking for it is a reckless choice for any
| company offering those tools. Integrating AI in tools already
| used to write code is how you win over the market.
| roopepal wrote:
| > Your website has a serious issue.
|
| I was thinking the same. My (admittedly old-ish) 2070 Super
| runs at 25-30% just looking at the landing page. Seems a bit
| crazy for a basic web page. I'm guessing it's the background
| animation.
| captaincrunch wrote:
| Couldn't get through the video, your keyboard sounds are very
| annoying.
| brandonchen wrote:
| Sorry Will try an external keyboard next time!
| toisanji wrote:
| Quality of code wise, is it worse or better than Cursor? I pay
| for Cursor now and it saves me a LOT of time to not copy files
| around. I actually still use the chatGPT/claude interfaces to
| code as well.
| jahooma wrote:
| Cool, it's probably about the same, since we're both using the
| new Sonnet 3.5 for coding.
|
| We might have a bit of an advantage because we pull more files
| as context so the edit can be more in the style of your
| existing code.
|
| One downside to use pulling more context is we burn more
| tokens. That's partly why we have to charge $99 whereas cursor
| is $20 per month.
| brandonchen wrote:
| It might sound small, but pulling in more context can make a
| huge difference - I remember one time Cursor completely
| hallucinated Prisma as part of our tech stack and created a
| whole new schema for us, whereas Codebuff knew we were
| already hooked up to Drizzle and just modified our existing
| schema. But like James said, we do use more tokens to do
| this, so pros & cons.
| toisanji wrote:
| If its the same, hard to justify $100/month vs $20/month . I
| code mostly from vim so I'm searching for my vim/cli
| replacement while I still use both vim and Cursor.
| jahooma wrote:
| Ah, but in Cursor you (mostly) have to manually choose
| files to edit and then approve all the changes.
|
| With Codebuff, you just chat from the terminal. After
| trying it, I think you might not want to go back to Cursor
| haha.
| toisanji wrote:
| That sounds cool and I like the idea, but definitely
| won't pay 5x. Maybe charge $30/month plus bring your own
| key. Let me know when you lower the price :)
| evntdrvn wrote:
| What if you have a microservice system with a repo-per-service
| setup, where to add functionality to a FE site you would have to
| edit code in three or four specific repos (FE site repo + backend
| service repo + API-client npm package repo + API gateway repo)
| out of hundreds of total repos?
| handfuloflight wrote:
| This does seem to be suited to monorepo.
| jahooma wrote:
| Yes, unfortunately, Codebuff will only read files within one
| directory (and sub-directories).
|
| If you have multiple repos, you could create a directory that
| contains them all, and that should work pretty well!
| brandonchen wrote:
| Codebuff works on a local directory level, so it technically
| doesn't have to be a monorepo (though full disclaimer: our
| codebase is a monorepo and that's where we use it most). Most
| important thing is to make sure you have the projects in the
| same root directory so you can access them together. I've used
| it in a setup with two different repos in the same folder. That
| said, it might warn you that there's not .git folder at the
| root level when this happens.
| kfajdsl wrote:
| Are there any plans to add a sandbox? This seems cool, but it
| seems susceptible to prompt injection attacks when for example
| asking questions about a not necessarily trusted open source
| codebase.
| jahooma wrote:
| We might! You could also set up your project within a docker
| container pretty simply (Codebuff would be great at setting
| that up :P).
| mitch7w wrote:
| Amazing stuff! The rebrand is great and it's cool to read the
| whole story!
| maldous wrote:
| brilliant - and thank you - so impressed with your work, i
| finally made an account to just comment - out of the box worked,
| a few minor glitches, but this is the start of awesome. keep
| doing what you are doing.
| jahooma wrote:
| Amazing, good to hear! What were the minor glitches you
| encountered? Would love to fix them up.
| tgtweak wrote:
| been using cline extension in vscode (which can execute commands
| and look at the output on terminal) and it's an incredibly adept
| sysadmin, cloud architect and data engineer. I like that cline
| lets you approve/decline execution requests and you can run it
| without sending the output which is safer from a data
| perspective.
|
| It's cool to have this natively on the remote system though. I
| think a safer approach would be to compile a small binary locally
| that is multi-platform, and which has the command plus the
| capture of output to relay back, and transmit that over ssh for
| execution (like how MGMT config management compiles golang to
| static binary and sends it over to the remote node vs having to
| have mgmt and all it's deps installed on every system it's
| managing).
|
| Could be low lift vs having a package, all it's dependencies and
| credentials running on the target system.
| dartos wrote:
| Are you an adept sysadmin, cloud architect, and/or data
| engineer?
|
| It's a weird catch-22 giving praise like that to LLMs.
|
| If you are, then you might be able to intuit and fill in the
| gaps left my the LLM and not even know it.
|
| And if you're not, then how could you judge?
|
| Not really much to do with that you were saying, really, just a
| thought I had.
| kajecounterhack wrote:
| I'd assume the person giving the praise is at least a bit of
| all 3.
|
| > It's a weird catch-22 giving praise like that to LLMs.
|
| It's a bit asymmetrical though isn't it -- judging quality is
| in fact much easier than producing it.
|
| > you might be able to intuit and fill in the gaps left my
| the LLM and not even know it
|
| Just because you are able to fill gaps with it doesn't mean
| it's not good. With all of these tools you basically have to
| fill gaps. There are still differences between Cline vs
| Cursor vs Aider vs Codebuff.
|
| Personally I've found Cline to be the best to date, followed
| by Cursor.
| dartos wrote:
| > judging quality is in fact much easier than producing it
|
| There's still a skill floor required to accurately judge
| something.
|
| A layman can't accurately judge the work of a surgeon.
|
| > Just because you are able to fill gaps with it doesn't
| mean it's not good.
|
| If I had to fill in my sysadmin's knowledge gaps I wouldn't
| call them a good sysadmin.
|
| Not saying the tool isn't useful, mind you, just playing
| semantics with calling a tool a "good sysadmin" or
| whatever.
| jahooma wrote:
| I have heard good things about Cline! I'm curious to learn
| more. I need to try it out myself.
|
| I see Codebuff as a premium version of Cline, assuming that we
| are in fact more expensive. We do a lot of work to find more
| relevant files to include in context.
| ali-netproject wrote:
| Extra context length looks valuable! Excited to try this out!
| iimaginary wrote:
| Really like the look of this interface. You're definitely onto
| something. Good work.
| israrkhan wrote:
| am I the only one who is scared of "it can run any command in
| your terminal"?
| brandonchen wrote:
| Hah no, you're not alone! Candidly, this is one of the top
| complaints from users. We're doing a lot of prompt engineering
| to be safe, but we can definitely do more. But the ones who
| take the leap of faith have found that it speeds up their
| workflows tremendously. Codebuff installs the right packages,
| sets up environments correctly, runs their scripts, etc. It
| feels magical because you can stay at a high level and focus on
| the real problems you're trying to solve.
|
| If you're nervous about this, I'd suggest throwing Codebuff in
| a Docker container or even a separate instance with just your
| codebase.
| CHERHU wrote:
| The product design is really thoughtful and thanks for sharing
| your story - Cannot wait to try this see you and see how you
| iterate on this!
| v3ss0n wrote:
| We already have AIDE, Continue, Cody , Aider, Cursor.. Why this?
| kahmeal wrote:
| first time?
| jahooma wrote:
| Codebuff is the easiest to use of all these, because you just
| chat, and it finds all the right files to edit. There's no
| clicking and you don't have to confirm edits.
|
| It is also a true agent. It can run terminal commands to aid
| the request. For one request it could: 1. Write a unit test 2.
| Run the test 3. Edit code to fix the error 4. Run it again and
| see it pass
|
| If you try out Codebuff, I think you'll see why it's unique!
| mediaman wrote:
| Can codebuff handle larger files (6,000 loc) and find the
| right classes/functions in that code, or if it finds the file
| with the necessary info does it load the entire file in?
| Finbarr wrote:
| I've been using Codebuff (formerly manicode) for a few weeks. I
| think they have nailed the editing paradigm and I'm using it
| multiple times a day.
|
| If you want to make a multi-file edit in cursor, you open
| composer, probably have to click to start a new composer session,
| type what you want, tell it which files it needs to include,
| watch it run through the change (seeing only an abbreviated
| version of the changes it makes), click apply all, then have to
| go and actually look at the real diff.
|
| With codebuff, you open codebuff in terminal and just type what
| you want, and it will scan the whole directory to figure out
| which files to include. Then you can see the whole diff. It's way
| cleaner and faster for making large changes. Because it can run
| terminal commands, it's also really good at cleaning up after
| itself, e.g., removing files, renaming files, installing
| dependencies, etc.
|
| Both tools need work in terms of reliability, but the workflow
| with Codebuff is 10x better.
| brandonchen wrote:
| Thanks for being an early user and supporter! You've helped us
| catch so many issues that have helped us get the product to
| where it is today!
| iandanforth wrote:
| Does this send code via your servers? If so, why? Nothing you've
| described couldn't be better implemented as a local service.
|
| Could this tool get a command from the LLM which would result in
| file-loss? How would you prevent that?
| draebek wrote:
| Congratulations on your launch! But I confess that I am really
| confused. This sounds exactly like Aider, but closed source and
| it's locked into a single LLM API? I just watched you use it, and
| looks a lot like Aider too? Why would I use this over Aider?
|
| I've seen people say "you don't have to add files to Codebuff",
| but Aider tells me when the LLM has requested to see files. I
| just have to approve it. If that bothers you, it's open source,
| so you could probably just add a config to always add files when
| requested.
|
| Aider can also run commands for you.
|
| What am I missing?
| carom wrote:
| How do you end up handling line numbers in patches? Counting has
| always been a sticking point for LLMs.
| sanketsaurav wrote:
| Congrats on the launch! I tried this on a migration project I'm
| working on (which involves a lot of rote refactoring) and it
| worked very well. I think you've nailed the ergonomics for
| terminal-based operations on the codebase.
|
| I've been using Zed editor as my primary workhorse, and I can see
| codebuff as a helper CLI when I need to work. I'm not sure if a
| CLI-only interface outside my editor is the right UX for me to
| generate/edit code -- but this is perfect for refactors.
___________________________________________________________________
(page generated 2024-11-07 23:00 UTC)