[HN Gopher] Show HN: GritQL, a Rust CLI for rewriting source code
___________________________________________________________________
Show HN: GritQL, a Rust CLI for rewriting source code
Hi everyone! I'm excited to open source GritQL, a Rust CLI for
searching and transforming source code. GritQL comes from my
experiences with conducting large scale refactors and migrations.
Usually, I would start exploring a codebase with grep. This is easy
to start with, but most migrations end up accumulating additional
requirements like ensuring the right packages are imported and
excluding cases which don't have a viable migration path.
Eventually, to build a complex migration, I usually ended up having
to write a full codemod program with a tool like jscodeshift. This
comes with its own problems: - Most of the exploratory work has to
be abandoned as you figure out how to represent your original regex
search as an AST. - Reading/writing a codemod requires mentally
translating from AST names back to what source code actually looks
like. - Performance is often an afterthought, so iterating on a
large codemod can be painfully slow. - Codemod frameworks are
language-specific, so if you're hopping between multiple languages
--or trying to migrate a shared API--you have to learn different
tools. GritQL is an attempt to develop a powerful middle ground: -
Exploratory analysis is easy: just put a code snippet in backticks
and use $metavariables for placeholders. - Incrementally add
complexity by introducing side conditions with where clauses. -
Reuse named patterns to avoid rebuilding queries, and use shared
patterns from our standard library for common tasks like ensuring
modules are imported. - Iterate on large codebases quickly: we use
Rust for maximum performance GritQL has already been used on
thousands of repositories for complex migrations[1] but we're
excited to collaborate more with the open source community. [1]
Ex. https://github.com/openai/openai-python/discussions/742
Author : morgante
Score : 79 points
Date : 2024-03-20 19:23 UTC (3 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mlejva wrote:
| Congrats to the launch!
|
| Managing tech debt is and will be an important issue in the world
| of AI software. I'm glad there's a company solving this.
| Additionally, this is something the current generation of LLMs
| are already helpful at solving. GritQL is very unique way of
| approaching this problem.
|
| OT: We've been working with Grit and Morgante for the past months
| and I have only positive things to say about them. If GritQL
| looks useful to you, you should reach out.
| psalv wrote:
| ++ congratulations to the team!
|
| We've tackled some pretty gnarly migrations with GritQL and it
| has saved us a lot of time.
|
| We have used ts-morph in the past for similar types of migrations
| but found the barrier to entry felt high, which prevented
| adoption. We were impressed by how complicated of migrations we
| could use GritQL for--we have been using it to migrate from mobx
| to react hooks.
| morgante wrote:
| Thanks Paul! It's been great getting your feedback as we've
| evolved Grit over time.
| jxnlco wrote:
| this definitly helped a tonne to migrate from openai <1.0 to 1.0
| ranman wrote:
| I'd love to see one of these for migrating from Claude's prompt
| API to their messaging API
| rattray wrote:
| Very happy Grit customer here. Wish I'd had this instead of
| jscodeshift et al back when I was doing oodles of migrations at
| Stripe, but glad I have it now, and very excited to see them go
| open-source!
| morgante wrote:
| Thanks Alex! Your feedback has been super helpful.
|
| Your suggestion to add the `--interactive` flag was hugely
| helpful, it's become very commonly used.
| anotherpaulg wrote:
| This looks great, thanks for building and sharing it.
|
| Interested folks may also want to check out ast-grep:
|
| https://github.com/ast-grep/ast-grep
| morgante wrote:
| ast-grep is great too, we've learned a lot from it. If you like
| yaml, you should definitely try ast-grep.
|
| The main area GritQL shines is in taking simpler transformation
| patterns and composing them into complex migrations. For
| example, the OpenAI migration was built by incrementally
| handling the different edge cases we've seen in the wild:
| https://github.com/getgrit/stdlib/blob/main/.grit/patterns/p...
| herrington_d wrote:
| thanks for mentioning ast-grep!
| practicalrs wrote:
| "a Rust CLI" - "C 96.5% Rust 1.4%"
|
| Not that I'm complaining ;)
|
| Is it a new way to market a C tool?
| morgante wrote:
| The C is all generated parsers, all our actual coding is done
| in Rust.
| mdaniel wrote:
| this audience will also enjoy(?) the tree-sitter discussion
| going on here: https://news.ycombinator.com/item?id=39762495
|
| there are very similar complaints about "yikes that thing
| just vomits a massive amount of loosey goosey C" so I hope
| you've not experienced the crashes they're talking about
| mdaniel wrote:
| _apologies if this should be a discussion /issue/whatever but:_
|
| Do you envision going up against CodeQL and/or
| <https://www.jetbrains.com/help/qodana/about-qodana.html> by
| making semantic information available to the ast nodes? OT1H, I
| can imagine it could be an overwhelming increase in project
| scope, but OTOH it could also truly lead to some stunning
| transformation patterns
|
| e.g.
| https://github.com/github/codeql/blob/v1.27.0/java/ql/exampl...
| or even more "textual" semantics such as var foo
| = "hello".substring(1); // knowing "foo" is a String
|
| Since you already have GitLab on your docs page, if I could pay
| for CodeQL over on GL that would be a game changer IMHO
| morgante wrote:
| One of the specific design goals we had was to make sure that
| the query engine isn't tightly coupled to AST nodes.
|
| We started with the AST, since that's ultimately the foundation
| for code but the goal is to add more graphs on top. So you
| could do a query like this to find all cases where a function
| called `unsafe` is queried with an unknown value:
|
| `unsafe($val)` where $val <: type `unknown`
|
| Is your interest in CodeQL primarily for security scans?
| mdaniel wrote:
| security is up there, but from reading the examples in CodeQL
| it just seemed like it would be possible to express some
| truly great versions of "don't do that" rules in it. I am a
| total JetBrains fanboi, and their introspections are world-
| class, but getting Qodana to run to completion before the
| heat death of the universe has proven to require more glucose
| than I have to offer it. Thus, I'm always interested in
| alternate implementations, even though I am _acutely_ aware
| of the computational complexity of what I 'm asking
|
| I recalled another link I wish I had included in my question
| from the SourceGraph folks
| https://github.com/sourcegraph/scip#scip-code-
| intelligence-p... which started out life as "Language Server
| Indexing Protocol" and seems to solve some similar project-
| wide introspection questions but TBH since their rug pull
| I've been a lot less willing to hitch my wagon to their train
| morgante wrote:
| Makes sense, we internally use GritQL as a super linter for
| some codebase-specific patterns. It's pretty easy to set
| up: https://docs.grit.io/guides/ci
|
| We've looked at both LSIF and SCIP from SourceGraph, though
| I expect we'll end up building our own index to allow for
| more complex queries. We're also incorporating some LLM
| functionality to express conditions you can't program
| deterministically.
|
| If you're interested in being a design partner for some of
| our auto-review features feel free to email
| morgante@grit.io.
| herrington_d wrote:
| Congrats from ast-grep team! Nice job Morgante!
|
| GritQL shares a lot of common ground with ast-grep, which is also
| a code rewriter based on tree-sitter.
|
| Interested folks may wonder about their difference. The key
| difference is that their surface APIs are quite different. GritQL
| is more like a DSL (or SQL). while ast-grep is more like pattern
| language + embedded configuration in YAML. Check it out here if
| you are interested https://github.com/ast-grep/ast-grep
___________________________________________________________________
(page generated 2024-03-20 23:00 UTC)