[HN Gopher] The evolution of a structural code editor
___________________________________________________________________
The evolution of a structural code editor
Author : crowdhailer
Score : 159 points
Date : 2025-01-06 09:00 UTC (14 hours ago)
(HTM) web link (crowdhailer.me)
(TXT) w3m dump (crowdhailer.me)
| naikrovek wrote:
| I have wanted something like this for so long that I can't
| remember when I started.
|
| Strongly coupling source code to plain text is a lost opportunity
| to improve tooling.
| jtwaleson wrote:
| Same! I think we need to free "code" from files, characters,
| tabs/spaces and variable names. Put millions of ASTs in a
| database. Imagine how much cheaper and better AI code
| assistants can be if they can only generate valid code with
| their "next token".
| zokier wrote:
| At some point the AST needs to be serialized as bytes on
| disk. In the big picture, the cost of having that
| serialization format be also human readable is relatively
| small as long as the syntax is designed to be sane (i.e. not
| c++/perl)
| norman784 wrote:
| > Put millions of ASTs in a database.
|
| That's basically what unison does, they store the source code
| in binary format in an sqlite database.
| umanwizard wrote:
| The emacs mode paredit is basically structural editing for lisp
| code (and some of its more modern competitors like Smartparens
| support other languages too, but not as well as Lisp). It's
| great.
| norir wrote:
| I disagree that plain text is an obstacle towards better
| tooling. You can do most of what the author describes with
| direct plain text edits pipelined into an analysis program.
|
| But I also think that the ux the author presents is
| undesirable. When writing new code, one must pass through a
| phase of broken code to reach the working code. I would not
| want an editor blocking me from typing a word that isn't yet
| defined but I do want it to notify me instantly that there is a
| problem.
|
| I also struggled to understand the use case for the mouse based
| ux. I am typing this comment on an iphone mini and would much
| rather write raw text on its keyboard than use the clicking
| interface shown in the fibonacci example.
| datadrivenangel wrote:
| In the data science space, people have been going back and forth
| on the benefits of Notebooks, which seem very similar, and were
| historically stored as json under the hood.
|
| There are many many benefits to plain text.
| jtwaleson wrote:
| I don't think notebooks are very similar to be honest. The
| notebooks still emit code (typically python) which is then
| interpreted as usual.
| norir wrote:
| I think structural editors are a great feature to add on top of a
| language with a definite textual representation. If you can
| pretty print the ast (as the fib example shows), you should
| generally be able to parse the output to generate the structured
| output.
| mebazaa wrote:
| My first thought: maybe competitive programmers could find it
| useful, if it's a way to reduce the amount of key presses needed
| to write their code? (At the very least, that's a crowd that
| would be motivated enough to get accustomed to that way of
| writing code.)
| poutsku wrote:
| Every year when I participate in advent of code, I have this
| thought: "wouldn't it be great to solve these problems directly
| from my phone?" (The puzzles unlock quite early morning in my
| timezone and opening my laptop at that time somehow annoys me.)
|
| I have wondered many times if some structural editor with a
| bunch of high-level library methods (Dijkstra's, for instance)
| would actually give a good/fast experience for most days. I
| haven't had the time or interest yet to try and build something
| yet. But perhaps I should.
| crowdhailer wrote:
| I'd love to get this to the point where add your own
| keybindings to structural edits, potentially very involved
| ones. It's superficially not very different to a combination
| of vim macros, auto completes or templates in other editors
| but I think the end result could be interesting as the units
| of change would be more meaningful. I think it's the same
| benefit of a hygienic macro system vs a source file
| manipulating version of meta programming.
| criddell wrote:
| Is it far enough along where you could work on the code editor
| _in_ the code editor?
| bennythomsson wrote:
| As the videos show, there are still a few usability issues.
| Full-screen switching somewhere else just to enter a name for
| example doesn't get you very far.
|
| But there is not much missing and I like the POC!
| crowdhailer wrote:
| The code editor is for the EYG language and implemented in
| Gleam. So that's a blocker. The editor is probably far enough
| along to have a crack at it by EYG would need some UI or web
| library so that EYG was able to build an editor.
| narag wrote:
| This is the way. Programs are not text, there's an impedance
| mismatch that shows at the seams.
|
| There're good reasons it hasn't been done more: text is good for
| troubleshooting, "every program should do one thing well" and
| maybe preventing lock-in.
|
| But with programming tools reaching maturity, we'll see more of
| this. Also there's an AI-related aspect: bots can understand
| structure just fine.
| taeric wrote:
| There are good reasons it hasn't been done more, and I don't
| think it is just "text is good for troubleshooting."
|
| At large, text wins because the job of a program text is to
| communicate the program to a user in a way that a computer can
| also be made to use. Things that help include that text can
| have multiple interpretations and meanings. This is why most
| variable names are nouns. You can literally "ascii art" parts
| of code to help convey some ideas.
|
| The most important part of why "text" wins, though, is that it
| is all visible. Yes it implies some structure underneath that
| the computer is going to need. The text, though, is fully
| visible to users. People think they want the ability to
| "structurally edit," and you can sometimes make that work
| rather nicely. Most of the time, though, you are working with
| structures that are not complete. And worse, completing them
| will _not_ be a process that goes in the same direction as the
| text. You will have to start a stack in how you are processing
| things to complete as you go.
|
| I'm torn on the bots comment, as I just don't know. LLMs,
| arguably, are leaning in on the structure that has been
| embedded in our symbolic text over many years. The same
| structure has not necessarily been developed in computer
| programming texts?
|
| (Also worth paying attention to how most of these goals were
| pursued with symbolic programming techniques back in the day.
| LISP, of course, but also term rewriting techniques that used
| to be rather magical.)
| throw-the-towel wrote:
| Hell, even text is not exactly text, it also has structure. I'd
| love to have refactoring tools for prose.
| ok_computer wrote:
| In my opinion, that's the magic of programming languages. I
| type a few high level words and characters, then there is a
| deep stack of interpreting the strings into machine code and
| returning data and side effects. It is stupid simple as the end
| user and immensely complex chain of machinery.
|
| SQL is an extreme example of strings parsed to make complex
| computer behavior. Shell scripting too.
|
| For enforcing complete object correctness and interfaces as
| they are explicitly designed then low code no code seems like
| it has a better proposition.
|
| Anyway, maybe structured editors fit a space between dumb
| strings and pointing and clicking blocks and connectors. I'm
| all for correct tool for the job and end user. I think that
| minimizing end user tooling with some LSP go-to definitions and
| linting in a text editor is my preference. Similar to how FIFA
| used to be against high tech slow mo replay for soccer matches
| to make the sport egalitarian / accessible for the whole world
| I think that analogously programming should focus on the
| simplicity and portability of authored documents.
| LeftHandPath wrote:
| It's fascinating because I came to a similar conclusion while
| working with _very_ old IBM mainframe code editors (AS400 /
| IBM i "Source Entry Utility")
|
| It's nearly impossible (if you read the docs) to write
| something with bad syntax on those machines. Columns are
| semantic in most of the languages (COBOL, RPG, DDL). When you
| write a program, if you don't know what you're doing, you can
| press "F4" and it will show you what each column means and
| when/how to use it [0]. It's not exactly a structured code
| editor, but it shares some similar concepts (see the link; you
| often write code in a "prompt" at the bottom of the screen, and
| the line won't insert until it's complete, though you can write
| it by hand if you wish).
|
| All of that said, IBM's old SEU is still vastly inferior to
| modern text editors and IDEs, mostly because of the limited
| view window and the lack of syntax highlighting / basic modern
| features [1].
|
| I think that, especially for enterprise code/development, there
| are some benefits to structured editing that are worth
| exploring. Not having a linter shout at a partially-complete
| line because you paused for a second or two would be nice, as
| well.
|
| [0]: https://youtu.be/iw_wk5elf3Q?si=m9zVeboTyw0T8_Wu&t=635
| [1]: https://youtu.be/Q3hxwcYB1Oo?si=WE9-rFdYo72A5yXz&t=404
| rzwitserloot wrote:
| I've been toying with the idea of making something very very much
| lik this, but, for an existing language. Which makes it far, far
| more complicated; I'm sure various aspects of the design of
| whatever language one would build this for are needlessly
| convoluted due to not having been designed with this kind of
| editing in mind.
|
| A few concerns:
|
| * As developers, our 'world' is so defined by text, we textify
| things that aren't: We write marked up documentation in plain
| text (HTML, markdown, etc) and we despise using tools like
| LibreOffice to write that sort of thing, and we even 'draw'
| helpful graphs in `.dot` files. We do this for a good reason: It
| 'works' with our other tools. git itself, and the cavalcade of
| git-understanding frontend version control systems, are excellent
| at showing you diffs between textual representations. They don't
| do 'diffs between these 2 png files' or 'diffs between these two
| .docx files' well or at all.
|
| Hence, these projects either balloon, or need a pragmatic take.
| Either you staple 'also reinvent git' onto the project (and in
| passing you are increasing the hurdle to adopt or try out this
| stuff for any interested folk), or you think about how to adapt
| the idea to fit existing practices.
|
| In this specific case the latter seems like the obvious answer:
| The way to store the program is as a file that, if opened in a
| text editor, looks just like it. But, I _can_ edit files in git
| and it looks like a text file even if it really isn't; so what
| happens if I edit an eyg file with a text editor? How 'perfect'
| must it be? What happens if in hand-editing the file I added some
| whitespace that shouldn't have been there as per the canonical
| storage rule?
|
| These are all easily solvable details, of course. But it's where
| my brain went. I'm pretty sure the correct answer is 'the source
| file is still text and is still readable; it's just that the
| language has some extreme style requirements (think about a
| language that has no line continuation facilities whatsoever,
| long lines will just have to be long lines, and dictates exactly
| how many tabs you can insert; indenting with spaces is not
| allowed at all, that sort of thing). This is fine because the
| language is meant to be read as plain text but only edited with
| the structural editor.
|
| * I often try to think how it would be if I was doing my
| programming work with a structural editor. From time to time I do
| things that seem blatantly annoying if I only had a structural
| editor. Where I really _want_ to dip into syntactically utterly
| invalid gobbledygook because I'm doing some text edits which will
| get back to valid code but I go through invalid code. For
| example, block-based copy/pasting (block =
| multiselect/multicursor). I'm having a hard time seeing how that
| would work in a structural editor but I use it from time to time.
| It feels like a structural editor can hopefully come up with
| alternatives for all the use cases but I'm not sure. Should there
| instead be a 'fallback' option where your source file turns into
| raw text mode and your edit functionality is like any plain jane
| text editor, you do your text-based transformations and then when
| done exit that mode, at which point the structural editor
| reparses what you have. I think that's right but it means the
| language does, actually, need all the bells and whistles after
| all: A Parser, an AST, good error handling for syntax errors, and
| so forth.
| PhilipRoman wrote:
| >They don't do 'diffs between these 2 png files' or 'diffs
| between these two .docx files' well or at all.
|
| I recently added an option to my gitconfig to allow diffing
| .docx files. My ugly 2-minute hack with Git is still miles
| ahead of the built-in Word history tracking. Ironically, to
| facilitate the diff, .docx is internally converted to text,
| once again showing how powerful it is to have a common
| denominator.
|
| On the topic of diffs, I think we still have not exhausted the
| full potential of text-based formats. Just because the inputs
| are text streams, doesn't mean the diff algorithm has to be
| dumb. As long as it emits readable output, it doesn't matter
| what it is doing internally (like parsing C code, detecting
| renames, etc.)
| hokkos wrote:
| I remember coding on my casio scientific calculator, you could
| only input full keyword by selecting from a menu, it was quick
| and you couldn't make syntax errors.
| nxobject wrote:
| Perhaps it was tokenising commands to save memory. Did your
| calculator have keystroke programming/some form of programming?
| zabzonk wrote:
| The Sinclair Spectrum Z80-based home computer had a keyboard
| that had all its BASIC keywords on the keyboard. I can't say I
| liked it, particularly as the keyboard was a very horrible
| rubber affair.
| anonzzzies wrote:
| People are now recreating them specifically to have the same
| feel out of nostalgia. I actually liked them as a kid; it was
| so easy to write impressive (for the time) graphical programs
| with BASIC without remembering all these pesky english words
| (I am not from an English speaking country).
| unshavedyak wrote:
| Awesome! Glad to see more exploration in this space. I too am
| exploring this, though i am perhaps making it yet even more
| complicated. I want to represent relationships between AST nodes
| in a visual way. Eg, i love Node editors in Blender and if it was
| given a good keyboard based UX i could imagine editing structural
| code in it.
|
| I still imagine the code would be visually text, but i'm
| imagining that you'd be able to see a series of nodes in a code
| flow graph for your current editing context.
|
| I figure AST editing (to some degree) is a generally good thing,
| but also representing code flow somehow is also a desired thing.
| It's what i do in my head, so can it be visualized in a
| beneficial way?
|
| Difficult to say. Keyboard first will also be a challenge as i'll
| need to have a lot of intelligent alignment and visual
| compression.
|
| Thanks for the post! Love to see work in this space.
| unshavedyak wrote:
| Oh, related in this space:
| https://mbuffett.com/posts/structured-editing-syntax/
|
| I thought this blog post had an interesting idea too. Basically
| that a structural editor which truly knew the underlying AST
| could then alter the visual representation of the code
| entirely. Eg changing it to look like Python, or whatever
| insane thing.
|
| Not advocating for that of course, but the idea that the editor
| could be effectively transpiling the code in real time is an
| interesting area to explore too. I imagine not for the purpose
| of "Write Python that becomes Rust" or w/e, but just in general
| customizing the experience. Perhaps reducing clutter, /shrug
| Slackwise wrote:
| > Glad to see more exploration in this space.
|
| I mean.... ParEdit [1] and ParInfer [2] have been around for a
| long time now. Structural editing is basically ancient. Lispers
| have been doing this for a long while, `slurp`ing and `barf`ing
| their parens.
|
| [1]: https://paredit.org/
|
| [2]: https://shaunlebron.github.io/parinfer/
| jerf wrote:
| I have no problem with anyone jousting with the windmills; I
| have a problem when they don't realize they are windmills.
| Hundreds and probably thousands of programmers have taken a
| crack at this problem. That doesn't prove it's a bad idea but
| it is effective proof until shown otherwise that if it is a
| good idea it is not such a good idea that it can instantly
| catch up to the decades of work in text editing. And the way
| some people talk about it, it's clear that a lot of
| programmers think that it should be that good.
|
| This is up there with "everything would be solved with visual
| programming" and "hey, what if we could put down different
| syntax on top of the ASTs?". If your mental model is that it
| must just be because nobody has put any effort into the
| question, update your model, because in fact _tons_ of effort
| has been expended and I 'd suggest anyone interested in
| adding to the pile and actually solving the problems (as
| opposed to playing about, anyone can play with anything they
| like) is best served by examining the large pile of previous
| efforts and figuring out what they will do better than
| before.
|
| Creating a language designed for being manipulated this way
| is at least a bit more rare than just declaring structural
| editing is the way forward in a general sense. Still, I'm
| skeptical. The existence of a textual serialization of
| programming language concepts is not the problem, nor is the
| usage of said serialization. Every serious tool already
| immediately deserializes the text into an AST, and when you
| really get down to the nitty-gritty of how one represents
| these things in memory, it is not at all clear that there is
| necessary a "better" way to serialize these structures, or
| one that is really very different in the ways that matter.
| There's a number of well-known (by those who look) pitfalls
| built into the idea of structural editing and it is not clear
| at all that the fundamental disadvantages can be overcome.
| Maybe they can. But the evidence at this point is effectively
| proof it's going to take more than waving "structural
| editing!" and some excitement at the problem.
| kagevf wrote:
| > Creating a language designed for being manipulated this
| way
|
| AFAIK, Lisp wasn't created for structural editing, it was
| just a "happy accident" that it lends itself so well to the
| technique. Even without the paredit and parinfer plugins,
| emacs already has good support for structural editing
| built-in. It reminds me of VIM text objects, but geared
| directly for Lisp syntax, ie. symbolic expressions.
| alexvitkov wrote:
| My biggest problem with node-based programming interfaces is
| the absurd number of nodes required for even fairly simple
| expressions, e.g `b*b-4*a*c` is 9 characters in most textual
| languages, but it would require 9 nodes in most visual
| scripting systems.
|
| I imagine you could have an arbitrary "expression" node with N
| inputs and a textfield, but I've never seen it done and it
| still feels like a bigger hassle than punching out the
| expression in a textual language.
| narag wrote:
| Indeed, a hybrid approach seems better.
| unshavedyak wrote:
| Well to be clear, my model (that i'm trying to make, it
| doesn't exist yet) is very mixed with traditional text
| editing.
|
| I agree with you, which is why mine is more of a text editor
| augmented by nodes. Notably each node is a variable scope
| window into text. I'm imaging a single function, more so than
| individual AST elements. The node can then have multiple
| inputs and outputs (or relationships of varying types, as i'm
| imagining) similar to how Blender can have many fields,
| inputs and outputs.
|
| The variable scoping would mean you can make a single node as
| large or as small as you need. Including a whole file, or a
| single expression within a single function, etc. The goal of
| this would be to visually reduce unrelated clutter, such that
| relationships should be clear and not dizzying.
|
| I want the nodes to visually represent how i normally work.
| Which is to say i have a text editor open and often i'm only
| looking at a single function in the file. Then i jump in and
| out of the function to related functions. Similarly those are
| text editor nodes as well, and so the chain continues.
|
| I should stress, i really enjoy my text editor (Helix). I'm
| trying to add onto that UX ultimately, rather than replace it
| entirely. Reduce the things i think/hope i don't care about
| -- ie unrelated functions in a file -- and add things like
| visual relationships. Imagine an aggressive traditional text
| editor setting which folds all code you're not using. But
| with a slightly different representation which hopefully adds
| to the experience.
|
| Sidenote, another motivation for me is to leave the Terminal.
| I've been in Terminal for 20 years now but _just_ leaving it
| is not alone worth it. I want to toy with graphical
| representations that would be difficult in a Terminal.
| Something to justify it 's existence when compared to
| something as easy and flexible as the Terminal is.
| penteract wrote:
| Having made my own forays into making a structural editor, I'd
| like to sing the praises of plain text: storing something that
| has been created by pressing buttons at a keyboard as something
| very close to the sequence of buttons that have been pressed is a
| great idea.
|
| The magical ability of plain text is to go back in time and ask
| "what if I pressed a different button a minute ago/yesterday/last
| year?". This lets you see and fix mistaken keypresses, and even
| shapes the kind of mistakes you can make in a moment of confusion
| into ones that are quickly fixable.
|
| The direct representation of buttons pressed also really helps
| with teaching/learning: With plain text, for mechanical questions
| on the level of "What buttons do I press to make a program that
| looks like that?", the answer is obvious for anyone who knows how
| to type. Try the editor discussed in the article to appreciate
| how different answering that question can be in a structural
| editor. Seeing and explaining how someone got where they did is
| also straightforward ("you missed a semicolon", "you misspelt a
| variable name"). Ideally, those are problems that don't happen in
| a structural editor, but people who press a slightly wrong
| sequence of buttons will probably still exist.
|
| These arguments get much weaker when you aren't using a physical
| keyboard; I think this is the reason the article starts by
| talking about writing programs on a phone and a TV. In those
| environments structural editing has much more to offer.
| daveloyall wrote:
| Since this is a post about a programming tool, I expected links
| to repositories. I like to pick up a tool and turn it around in
| my hands before I decide if it is a good tool or not.
| danielvaughn wrote:
| I'm obsessed with structural editors because it feels intuitively
| like the right approach to creating software. However I've yet to
| come across an interface that really clicks. Still love it
| though. Right now I'm building a structured editor for creating
| UIs, so can attest to how difficult it is to get it right.
|
| It's a super interesting problem space.
| crowdhailer wrote:
| I'd really like to get to the point where I have a library of
| transformations and you can make your own interface. Already
| the click ui and hot keys ui call the same `insert_function` or
| `assign_to` functions. Adding a ui with lots of dragging or
| voice control should be trivial
| danielvaughn wrote:
| I've actually moved away from GUI controls and towards
| something resembling a keyboard command language. Similar in
| spirit to vim motions, but instead of applied to text, it's
| applied to an AST.
|
| I think that could be the key to making structured editing
| feel less "clunky".
| xigency wrote:
| This is a topic I've been contemplating heavily and it's very
| instructive to see the iterative progress you've made here.
|
| I think development tools like this are necessary to expand
| coding outside of just the desktop experience. For example, I've
| seen kids program some pretty impressive robotics with just a
| touchscreen and Blockly.
|
| Also, from a compilers perspective, it feels almost absurd to
| rely on plaintext. Imagine how much harder code merging is using
| line diff compared to diffing syntax trees.
|
| Thanks for sharing!
| crowdhailer wrote:
| Diffing is really an interesting point. Git is quite (maybe
| very) good. I wonder how much better a structured approach
| could be but it's obviously a large amount of work that I don't
| yet have time for.
| kmoser wrote:
| A structured approach would be able to easily do things like
| ignore when a symbol was renamed (since that wouldn't affect
| the tree structure). So there are good reasons for wanting to
| do it that way, vs. a line-based diff.
| aiono wrote:
| Plain Text diffing has some obvious drawbacks:
|
| 1. If you rearrange your functions you will see a lot of
| additions and deletions while semantically there is no change
| in the program. It's just noise.
|
| 2. If you rename a variable you don't really have any actual
| change in places where it is referenced but text diff will
| again show a lot of noise. But the code references is still
| the same code.
| williamdclt wrote:
| Git (or whatever does the diff under the hood) is _adequate_.
| Every week I see diffs that are confusing, especially during
| merge conflicts, it's honestly a bit embarrassing that in
| this day and age we don't have anything better for something
| that is so central to our job
| ElevenLathe wrote:
| Modern SmallTalks have the option to save off a hierarchy of
| classes to text in order to use modern version control, and
| also to apply these exports to a running image. This should
| be even easier for traditional languages with no live running
| environment. There's no reason whatever structure editor you
| use can't export a normalized text representation so that you
| can still use it with Git
| amar1729 wrote:
| for diffing syntax trees, I've been trying out diffsitter, and
| it feels pretty good so far
|
| https://github.com/afnanenayet/diffsitter
| cbm-vic-20 wrote:
| This is neat.
|
| I wonder if this could be generalized to other languages with
| Tree-Sitter and LSPs. It could be helpful for programmers with
| accessibilty issues, among many other use cases.
| crowdhailer wrote:
| It could be in principle. EYG was developed along with the
| editor so that has effected the design of the AST. Ideally if a
| useful/powerful/pleasant/intuitive structured editing
| experience was found for EYG (and it's minimal AST) then it
| would be proven to work and so putting the effort into doing
| for a program with a more complex AST would be worth the
| effort.
| crabbone wrote:
| > Unlike traditional text-based editors, these tools treat code
| as a structured tree rather than a flat sequence of characters.
|
| This is such an obvious lie... Of course, even the disastrously
| bad editors like VSCode or IntelliJ tools understand code to have
| structure and allow interaction with that structure. Hell, even
| text editors not intended to deal with code will usually be able
| to deal with some structure, like identifying words, lines,
| perhaps even paragraphs and operating on them.
|
| > No syntax errors. Zero time is spent on missing quotes, semi-
| colons or other details.
|
| Usually paid for by inability to copy and paste chunks of code...
| not sure if it's worth the price.
|
| > Better type information. Because the program is always in a
| valid state the type checker can always run and give meaningful
| feedback.
|
| Why would you want to run typechecker _all the time?_ What 's the
| benefit? I run it when I'm interested in the results... otherwise
| it's just noise. By the time I'm interested in results, I'd get
| code in a decent enough shape to compile, so it won't be a
| problem.
|
| Other known downsides of this approach: having to access the code
| in places where the editor isn't available, s.a. reading diffs,
| debugging, running third-party tools unaware of structured format
| (which, given how our programming practices have developed, will
| be inconsistent between versions and platforms).
|
| All in all, I've seen these attempts multiple times before, and
| it always looks like the benefits simply aren't worth the effort.
| Or, maybe, too many stars need to align in order for this to
| actually become really useful in the day-to-day programming work,
| and so it doesn't happen because of some circular dependency
| where the tools won't be built, until the product is good enough
| to use, but the product is never good enough to use because there
| aren't any tools?
| lsaferite wrote:
| > disastrously bad editors like VSCode or IntelliJ tools
|
| I'm curious, what code editor(s) do you hold in high esteem?
| cfiggers wrote:
| I enjoy Lisp and Lisp-alike languages specifically because with
| the right editor support you can do a lot of this kind of
| structural editing stuff without giving up the niceties of source
| code as text.
| mtreis86 wrote:
| Yeah in emacs with paredit/smartparens/parinfer or whatever, it
| is both text and structure. With parinfer you change the text
| and the structure gets modified to match. With the other two
| you can change the structure directly with a command that has a
| silly but descriptive name like slurp or barf.
| perlcommunity wrote:
| https://www.jgrasp.org/
| EdiX wrote:
| I used a structural text editor a while back (technically a
| structural text editing mode for emacs). I went in completely
| buying the structural text editing propaganda (it's more logical,
| it will make you more efficient, it's the future...) but I
| eventually found the experience so excruciatingly bad and stupid
| that I now think the whole endeavour is a complete waste of time,
| the idea sounds good on paper but there are some fundamental
| unsolvable problems with it.
|
| The problem is that, when you want to make a change to piece of
| source code you know its current state and you know how you want
| it to look like at the end. With a normal text editor you will
| simply make a bunch of changes that turn the text as it into what
| it should be, it will normally go through a series of invalid
| states but you don't have to think about what those states will
| be or care about them. With a structural text editor this is
| impossible, invalid states _can not be represented_ so when you
| want to make a change to your source code you don 't just care
| about the beginning and end state, you also want to find a series
| of valid states that takes you from the beginning to the end
| where each one can be reached from the previous one with an
| editor command.
|
| For example, let's say you have a function call:
| (afunc 1 2 3)
|
| and you want to turn it into a conditional function call based on
| a flag: (if flag (afunc 1 2 3))
|
| with a normal text editor you: 1. type "(if
| flag " before the call 2. type ")" after the call
|
| It doesn't matter the order in which I do these operations or how
| I move the cursor.
|
| With a structural text editor, I have to: 1.
| create a new form before the function 2. type "if flag"
| 3. select the entire function call 4. cut it 5.
| paste it as the first branch of the if
|
| This is however only possible if the context where the function
| call currently is allows adding a new node before it (i.e. it's
| inside the equivalent of a lisp progn). If not I'd have to use a
| different strategy, for example: 1. select the
| entire function call 2. use a command to create a node
| around it 3. type "if flag"
|
| I did this for a few months, eventually I realized that I didn't
| want to think about the syntactical validity of the intermediate
| states my program goes through while I'm editing it, because they
| exist for mere seconds, and I didn't want to memorize the large
| palette of commands necessary to create those intermediate
| states. It was all additional cognitive load that accomplished
| nothing but satisfy someone's esthetic sense that source code
| should always be in a syntactically valid state.
|
| So of the benefits listed in the article:
|
| 1. "No syntax errors" is actually a major detriment of the
| structural editing model
|
| 2. "Better type information" is a lie, all that the structural
| text editor guarantees you is that the program is syntactically
| valid, not that it will pass type checking
|
| 3. "No keywords" is a lie, it's only true if you use a new
| programming language and don't store it on disk as text which
| opens a whole other can of worms (can't even read source code
| without a special text editor, won't work with github, won't work
| with any versioning system, won't work with grep, can't copy
| paste a chunk of it in an email...)
|
| 4. "Rich visualization" is an entirely different story from
| structural text editing but as far as I know nobody has ever
| shown visual programming to actually provide any kind of tangible
| benefit
|
| 5. "Reduced complexity" is the one I give them, yes if you are
| making a structured text editor you don't have to write a fault
| tolerant parser, it does make _your_ (aka the text editor 's
| implementor) life easier
|
| Fans of structural text editors usually mention that they will
| unlock "more powerful refactoring tools" but actually all you
| need for those to be "unlocked" is a dumb parser and an automatic
| code formatter, like "go fmt". Refactoring a function to be
| inlined into another is still a difficult operation to implement
| correctly but it isn't like the difficult part is running a
| parser on the source code, it's dealing with variable shadowing.
| herrington_d wrote:
| This is a pretty fair comment. I wonder how structural editing
| figures out things like formatting/searching/diffing and
| copy/paste across different editors.
| EdiX wrote:
| Those are all orthogonal concerns to the way the text editor
| behaves. You could have a structural text editor operate on
| plain text files and a dumb text editor operate on AST, you
| can do plain text diffs of ASTs or diff AST parsed from plain
| text files, etc etc...
|
| Some of these have existed historically (or even still
| exist).
|
| Paredit-mode is a structural text editor that saves and loads
| plain text files, Smalltalk was typically implemented as a
| dumb text editor but then code was saved as compiled binaries
| (which meant you couldn't save your functions if they weren't
| syntactically correct but you could have unsaved
| syntactically broken functions), Mathematica represents its
| code in a weird format that might as well be binary but
| copy/paste converts to plain text, there's one git plugin
| (don't remember the name right now) that does syntax aware
| diffs even though git deals with plain text...
| Twisol wrote:
| I agree with you on basically all points. But to play devil's
| advocate, I've often found myself doing something _like_
| structured editing when intentionally, carefully refactoring
| existing code. In those instances, it really matters to me that
| the programs at the start and at the end are equivalent, or at
| least equivalent up to certain semantic conditions (for
| instance: preconditions, things assumed by the domain, or cases
| not meant to be handled in the subject code). Since making a
| complex refactor in one step doesn 't make me very confident
| that I got it right, I usually carefully perform a series of
| small transformations such that, at every step, I have
| something that _manifestly_ behaves the same as the program
| before that step.
|
| This seems sufficiently like the structured programming model
| (small edits forming a path of "valid" programs from start to
| finish) that I wonder if something similar to structured
| programming might be genuinely useful in this niche. That said,
| these refactors aren't simple atomic tree edits; they're more
| like a small cluster of tree edits that together preserve
| meaning. So there's a difference in content, even if the "small
| edits that form a path through programs with X property" mental
| model is the same.
| EdiX wrote:
| For Go there's a program called rf
| (https://pkg.go.dev/rsc.io/rf) that's along those lines. It
| isn't worked on as much as I wish it was.
| herrington_d wrote:
| > Better type information. Because the program is always in a
| valid state the type checker can always run and give meaningful
| feedback.
|
| Note that programs can be syntactically well-formed but ill-
| typed. For example `let x = true + 1` has valid syntax but
| produces an "undefined" type for the variable `x`, if the type
| system does not support type error recovery.
|
| A quote from the great paper from POPL 2024,
| https://hazel.org/papers/marking-popl24.pdf
|
| > If a type error appears _anywhere_, the program is formally
| meaningless _everywhere_
| crowdhailer wrote:
| The EYG type system does support recovery, so you will get
| multiple type errors if that's the case in the program.
| herrington_d wrote:
| My point is that type error recovery is the property of the
| type system. Not the property of structural code editor.
| like_any_other wrote:
| For structural code editors to thrive, what is needed is a
| universal, simple, minimal, txt-like format for the AST, so that
| AST-editors are not locked to a single programming language each.
| crowdhailer wrote:
| What requires the format to be text-like? I'm not sure I agree
| with this assertion. I think structural editors need to be
| language specific so that they can offer higher value edits
| that might only make sense for specific languages.
| like_any_other wrote:
| "Text-like" meaning easy to parse and universally understood.
| With language-specific (or indeed, all) AST/generic tree
| attributes serialized in a uniform way that does not entirely
| break parsing if an editor does not understand one of them.
| I.e. I can edit code even if my editor doesn't understand the
| language and doesn't give me syntax highlighting and
| completion. More, that that 'understanding' can be factored
| out of the editor core, as is done with language servers
| today.
| zellyn wrote:
| You might enjoy reading up on EYG and/or listening to podcast
| appearances of the creator...
| crowdhailer wrote:
| Indeed. Here's a link to one episode
| https://pod.link/developer-
| voices/episode/e47720e8c6dc3d1660...
| andrewl wrote:
| I'm always glad to see new work on tools, and amplifications to
| our capabilities of any sort.
|
| I played with Smalltalk, and later Pharo, a bit, but it was years
| ago. As I recall the editor was always aware of the state of the
| system. And I _think_ it was at a level beyond what any current
| IDE can do. While I do not remember if it allowed you to enter
| invalid code, it was definitely an interesting approach.
|
| So I will be following Eyg for sure.
| xkriva11 wrote:
| In Pharo, the method needs to be syntactically valid to be
| saved.
| tcoff91 wrote:
| Haven't lisp coders been doing this forever with emacs?
___________________________________________________________________
(page generated 2025-01-06 23:01 UTC)