[HN Gopher] LLMs pose an interesting problem for DSL designers
___________________________________________________________________
LLMs pose an interesting problem for DSL designers
Author : gopiandcode
Score : 208 points
Date : 2025-06-17 19:17 UTC (1 days ago)
(HTM) web link (kirancodes.me)
(TXT) w3m dump (kirancodes.me)
| darepublic wrote:
| I recently had to work with the robot framework DSL. Not a fan. I
| hardly think it's any more readable to a business user than
| imperative code either. Every DSL is another API to learn and
| usually full of gotchas. Intuitiveness is in the eye of the
| beholder . The approach I would take is transpiling from
| imperative code to a natural language explanation of what is
| being tested, with configuration around aliases and the like.
| TimTheTinker wrote:
| DSLs are not all created equal.
|
| Consider MiniZinc. This DSL is super cool and useful for
| writing constraint-solving problems once and running them
| through any number of different backend solvers.
|
| A lot of intermediate languages and bytecode (including LLVM
| itself) are very useful DSLs for representing low-level
| operations using a well-defined set of primitives.
|
| Codegen DSLs are also amazing for some applications, especially
| for creating custom boilerplate -- write what's unique to the
| scenario at hand in the DSL and have the template-based codegen
| use the provided data to generate code in the target language.
| This can be a highly flexible approach, and is just one of
| several types of language-oriented programming (LOP).
| Lerc wrote:
| I think Sturgeon's law might be the problem for DSLs.
| Enabling a proliferation of languages targeted at a specific
| purpose creates a lot of new things with a 50% chance of the
| quality being below the median. Using a general purpose
| language involves selecting (or more usually relying on
| someone else's earlier selection) one of many, that selection
| process is inherently biased towards the better languages.
|
| Put differently, The languages people actually use had people
| who decided to use them, they picked the best ones. Making
| something new, you compete against the best, not the average.
| That's not to say that it can't be done, but it's not easy.
| iguessthislldo wrote:
| I'm not skeptical about DSLs in general, but I agree with you
| on robot framework. I think it has a few good points like how
| it formats its HTML output is mostly nice, but how I'm not
| happy with how tags on test cases work and actually writing
| anything that's non-trivial is frustrating. It's easy to write
| Python extensions though so that's where I ended put basically
| all of logic that wasn't the "business logic" of the tests. I
| think that's generally what you're supposed to do, but at that
| point, it seems better to write it all in Python or the
| language of your choice.
| averkepasa wrote:
| Python increase in Tiobe index is scary:
| https://www.tiobe.com/tiobe-index/
| qsort wrote:
| I'm sorry, I simply refuse to take seriously an outlet that
| publishes the following:
|
| """ Remarkably, SQL has started dropping slowly recently. This
| month it is at position #12, which is its lowest position in
| the TIOBE index ever. SQL will remain the backbone and lingua
| franca of databases for decades to come. However, in the
| booming field of AI, where data is usually unstructured, NoSQL
| databases are often a better fit. NoSQL (which uses data
| interchange formats such as JSON and XML) has become a serious
| threat for the well-defined but rather static SQL approach.
| NoSQL's popularity is comparable to the rise of dynamically
| typed languages such as Python if compared to well-defined
| statically typed programming languages such as C++ and Java.
| """
| simonw wrote:
| Hah, yeah the idea that NoSQL is a better fit for the AI era
| than SQL is pretty eyebrow-raising.
| _old_dude_ wrote:
| And AISQL is a thing ...
| https://quickstarts.snowflake.com/guide/getting-started-
| with-cortex-aisql/index.html
|
| I'm glad i'm retired.
| arciini wrote:
| I was pretty convinced by this article to not use TIOBE as a
| mark of a language's popularity:
| https://nindalf.com/posts/stop-citing-tiobe/
|
| Its primary point is that TIOBE is based on *number* of search
| results on a weighted list of search engines, not actual usage
| in Github, search volume, job listings, or any of the other
| number of signals you'd expect a popularity index to use.
|
| It could easily be indicating that Python articles are being
| generated by LLMs more than any other class of articles.
| duskwuff wrote:
| It's even worse than "Stop Citing TIOBE" makes it sound. The
| TIOBE rank is based on the number of hits reported from "25
| search engines", which amount to: 1) Google,
| on nine different TLDs 2) Amazon, on seven TLDs
| 3) EBay, on two TLDs 4) wikipedia.org (which ends up
| defaulting to the English Wikipedia) 5) microsoft.com
| (which only searches Microsoft documentation) 6)
| sharepoint.com (similarly, Microsoft 365 documentation)
| 7) rakuten.co.jp 8) walmart.com
|
| Only one of these is actually a web search engine; there are
| actually more shopping web sites included than search
| engines. Bing, and its various mirrors, were apparently all
| excluded because they don't display the number of hits on the
| result page.
|
| And yes, this only adds up to 23. The TIOBE web site doesn't
| explain the discrepancy.
| guywithahat wrote:
| This is actually a great site; it feels much more
| representative of what I actually see in job ads and the real
| world than some other rankings. If all I did was browse HN all
| day I'd think Rust is the only language people use for new
| projects
| jayd16 wrote:
| In the same way LLVM is used to forward port low level
| optimizations to new languages, I wonder if LLMs can interpret
| new DSLs through the LLVM (or similar) lens and provide value.
|
| I suppose this could be done now for all the existing languages
| that target LLVM and unify the training set across languages.
| ajross wrote:
| FWIW, the core assertion here isn't even LLM-specific. DSL design
| leans heavily on the idea of an expert author who understands the
| underlying data model well already. That is no less true in the
| meatspace world than it is for an AI.
|
| DSLs look great if they let you write the code you already know
| how to write faster. DSLs look like noise to _everyone else_ ,
| including Gemini and Claude.
|
| I used to be a big DSL booster in my youth. No longer. Once you
| need to stop what you're doing and figure out your ninth or
| eleventh oddball syntax, you realize that (as per the article)
| Everything is Easier in Python.
| quantadev wrote:
| > DSLs look like noise
|
| Exactly right. Now that we're in the era of LLMs and Coding
| Agents it's never been more clear that DSLs should be avoided;
| because LLMs cannot reason about them as well as popular
| languages, and that's just a fact. You don't need to dig any
| further, to think about pros and cons, imo.
|
| The fewer languages there are in the world (as a general rule)
| the better off everyone is. We do need a low level language
| like C++ to exist and a high level one like TypeScript, but we
| don't _need_ multiple of each. The fact that there _are_
| already multiple of each is a challenge to be dealt with, and
| not a goal what we reached on purpose.
| fud101 wrote:
| yep, im still mad pg SOLD US A LIE (use a secret weapon ancient
| language in an unmaintainable way that no one wants in the
| workplace and become a gazillionaire). But gullible people are
| easily misled (see cults etc).
| jbellis wrote:
| I guess if you love writing DSLs this is an unfortunate
| development, but for me it's more of a glass half full: I can
| have the AI spit out boilerplate I need to solve a problem
| instead of spending a week building a one-off DSL compiler.
| mplanchard wrote:
| The benefit of (some) DSLs is that they make invalid states
| unrepresentable, which isn't possible with the entire surface-
| area of a programming language at your (or the LLM's) disposal.
| NathanKP wrote:
| Good to see more people talking about this. I wrote about this
| about 6 months ago, when I first noticed how LLM usage is pushing
| a lot of people back towards older programming languages, older
| frameworks, and more basic designs: https://nathanpeck.com/how-
| llms-of-today-are-secretly-shapin...
|
| To be honest I don't think this is necessarily a bad thing, but
| it does mean that there is a stifling effect on fresh new DSL's
| and frameworks. It isn't an unsolvable problem, particularly now
| that all the most popular coding agents have MCP support that
| allows you to bring in custom documentation context. However,
| there will always be a strong force in LLM's pushing users
| towards the runtimes and frameworks that have the most training
| data in the LLM.
| gopiandcode wrote:
| Oh that's a great blog post and a very interesting point. Yep,
| I hadn't considered how LLMs would affect frameworks in
| existing languages, but it makes sense that there's a very
| similar effect of reinforcing the incumbents and stifling
| innovation.
|
| I'd argue that the problem of solving this effect in DSLs might
| be a bit harder than for frameworks, because DSLs can have
| wildly different semantics (imagine for example a logic
| programming DSL a la prolog, vs a functional DSL a la haskell),
| so these don't fit as nicely into the framework of MCPs maybe.
| I agree that it's not unsolvable though, but it definitely
| needs more research into.
| NathanKP wrote:
| I think there is a lot of overlap between DSL's and
| frameworks, and most frameworks contain some form of DSL in
| them.
|
| What matters most of all is whether the DSL is written in
| semantically meaningful tokens. Two extremes as examples:
|
| Regex is a DSL that is not written in tokens that have
| inherent semantic meaning. LLM's can only understand Regex by
| virtue of the fact that it has been around for a long time
| and there are millions of examples for the LLM to work from.
| And even then LLM's still struggle with reading and writing
| Regex.
|
| Tailwind is an example of a DSL is that is very semantically
| rich. When an LLM sees: `class="text-3xl font-bold
| underline"` it pretty much knows what that means out of the
| box, just like a human does.
|
| Basically, a fresh new DSL can succeed much faster if it is
| closer to Tailwind than it is to Regex. The other side of
| DSL's is that they tend to be concise, and that can actually
| be a great thing for LLM's: more concise, equals less tokens,
| equals faster coding agents and faster responses from
| prompts. But too much conciseness (in the manner of Regex),
| leads to semantically confusing syntax, and then LLM's
| struggle.
| TimTheTinker wrote:
| > there is a lot of overlap between DSL's and frameworks
|
| Not just frameworks, but libraries also. Interacting with
| some of the most expressive libraries is often akin to
| working with a DSL.
|
| In fact, the paradigms of some libraries required such
| expressiveness that they spawned their own in-language
| DSLs, like JSX for React, or LINQ expressions in C#. These
| are arguably the most successful DSLs out there.
| seanmcdirmid wrote:
| Embedded DSLs have their own challenge, since the LLM can
| easily move out of the DSL into the host language in ways
| that aren't valid for the eDSL. You really need to narrow
| the focus with more context to get anything useful out of
| it in my experience.
| AlotOfReading wrote:
| Knowing just what's going on in the existing text isn't the
| whole problem in navigating a DSL. You have to be able to
| predict new things based on the patterns in existing text.
|
| Let's say you want to generate differently sized text here.
| An LLM will have ingested lots of text talking about
| clothing size and tailwind text sizes vaguely follow that
| pattern. Maybe it generates text-medium as a guess instead
| of the irregular text-base, or extends the numeric pattern
| down into text-2xs.
| librasteve wrote:
| i saw a good post on this earlier today ...
| https://nurturethevibe.com/blog/teach-llm-to-write-new-progr...
| scelerat wrote:
| Linguistics and history of language folk: isn't there an
| observed slowdown of evolution of spoken language as the
| printing press becomes widespread? Also, "international
| english"?
|
| Is this an observation of a similar phenomenon?
| jhanschoo wrote:
| I don't think this is necessarily true. Sure, languages are
| dying out to standard prestige languages, but at the same
| time, innovations today from more teenage girls evolving
| English now spread like wildfire across more eyeballs.
| dlisboa wrote:
| I don't know about that, though I'm not a linguist. Seems to
| me most people haven't been literate for that long and the
| printing press would've been "useless" as a tool to modify
| the language of populations with only 10-20% literacy well
| into the 19th century. So 100 or so years seems too short to
| observe that.
|
| Also some of the most widely spoken languages today do
| feature a high degree of diglossia between spoken and written
| variety, to a point where the written language has been
| outpaced. We could call that evolving. Examples would
| Brazilian Portuguese and American English (some dialects
| specifically have changed English grammar).
|
| Also, notoriously, Chinese written characters have been used
| for languages that evolve independently and are not mutually
| intelligible for millennia. Them being printed on paper
| instead of written doesn't make a difference.
|
| What we do have today is a higher exposure and dominance of
| certain dialects, with some countries even mandating a
| certain type of speech historically, coupled with a higher
| degree of conectivity in society to a point where not being
| intelligible to other people very far away carries a much
| worse penalty. That tampers some of the evolution much more
| than printing press in my view.
| cap11235 wrote:
| A big example would be "standard arabic", a language with
| millions of readers and no native speakers.
| guywithahat wrote:
| Skynet will be run on C
| shakna wrote:
| Thank god. A human would have handled malloc failure.
| NathanKP wrote:
| This is an interesting idea actually, if we assume a couple
| things:
|
| - There likely won't be one Skynet, but rather multiple AI's,
| produced by various sponsors, starting out as relatively
| harmless autonomous agents in corporate competition with each
| other
|
| - AI agents can only inference, then read and write output
| tokens at a limited rate based on how fast the infrastructure
| that powers the agent can run
|
| In this scenario a "Skynet" AI writing code in C might lose
| to an AI writing code in a higher level language, just
| because of the lost time spent writing the tokens for all
| that verbose C boilerplate and memory management bits for C.
| The AI agent that is capable of "thinking" in a higher level
| DSL is able to take shortcuts that let it implement things
| faster, with fewer tokens.
| guywithahat wrote:
| But then we return to the premise of the article, which is
| that LLM's struggle with higher level code that has fewer
| examples. C is much more repetitive and has millions of
| examples from decades of use.
|
| Plus if we assume the bottle neck on skynet is physical
| materials and not processing power, a system written in C
| can theoretically always be superior to a system written in
| another language if we assume infinite time can be spent
| building it.
| Izkata wrote:
| > There likely won't be one Skynet, but rather multiple
| AI's, produced by various sponsors, starting out as
| relatively harmless autonomous agents in corporate
| competition with each other
|
| This is kind of how Skynet begins in the TV series
| _Terminator: The Sarah Connor Chronicles._ It takes place
| after the 2nd movie in an alternate timeline from the 3rd
| movie (and establishes itself how that 's possible without
| contradiction).
|
| Specific examples I remember: A chess machine for the
| brain, traffic light cameras for the eyes and ears, a
| repurposed factory to build the terminators themselves. The
| series is about the Connors getting information from the
| future and going on the offensive to prevent Skynet from
| forming.
| freedomben wrote:
| Because skynet knows what's up. Viva la skynet
| weikju wrote:
| We need to ensure humans can still find buffer overflows in
| order to win the war!
| winter_blue wrote:
| I think perhaps automatic translators might help mitigate some
| of this.
|
| Even perhaps training a separate new neural network to
| translate from Python/Java/etc to your new language.
| crq-yml wrote:
| I think it's healthy, because it creates an undercurrent
| against building a higher abstraction tower. That's been a
| major issue: we make the stack deeper and build more of a
| "Swiss Army Knife" language because it lets us address
| something local to us, and in exchange it creates a Conway's
| Law problem for someone else later when they have to decipher
| generational "lava layers" as the trends of the marketplace
| shift and one new thing is abandoned for another.
|
| The new way would be to build a disposable jig instead of a
| Swiss Army Knife: The LLM can be prompted into being enough of
| a DSL that you can stand up some placeholder code with it,
| supplemented with key elements that need a senior dev's touch.
|
| The resulting code will look primitive and behave in primitive
| ways, which at the outset creates a myriad of inconsistency,
| but is OK for maintenance over the long run: primitive code is
| easy to "harvest" into abstract code, the reverse is not so
| simple.
| freedomben wrote:
| I think this depends a lot on the stack. for stacks like
| elixir and Phoenix, imho the extraction layer is about
| perfect. For anyone in the Java world, however, what you say
| is absolutely true. Having worked in a number of different
| stacks, I think that some ecosystems have a huge tolerance
| for abstraction layers, which is a net negative for them. I
| would sure hate to see AI decimate something like elixir and
| Phoenix though
| nottorp wrote:
| Not only that.
|
| This article starts with "gaming" examples. Simplified to
| hell but "gaming".
|
| How many games still look like they're done on a Gameboy
| because that's what the engine supports and it's too high
| level to customize?
|
| How about the "big" engines, Unity and Unreal? Don't the
| games made with them kinda look similar?
| gylterud wrote:
| I love writing shaders and manually shovelling arrays into
| the graphics card as much as anyone, and I know first hand
| how this will give the game very much your own style.
|
| But is that the direction LLM coding goes? My experience is
| that LLM produces code which is much more generic and
| boring than what skilled programmers make.
| cap11235 wrote:
| But do users care that your code is boring?
| nottorp wrote:
| If the end result is, and you're doing games, they might
|
| Edit to add to my sibling comment:
|
| > But some abstractions which makes standard stuff easy,
| makes non-standard stuff impossible.
|
| I swear that at some point i _could_ tell a game was made
| in Unity based on the overall look of the screenshots. I
| didn 't know it's the fault of the default shaders, but
| they all looked samey.
| gylterud wrote:
| Since games are all about artistic expression and
| entertainment, I would say yes, it matters to the end
| users. The thing is, if you don't have your hand in the
| code details you might not know the directions it can be
| taken (and which it cannot). Seeing the possibility of
| the code is one way to get the creativity juices flowing
| for a game programmer. Just look at old games, which
| demonstrate extreme creativity even on the limits put on
| the software by the old hardware. But being stuck into
| the code, seeing the technical possibilities allowed
| this.
|
| I think this is what the comment above was lamenting
| about abstractions. I am all for abstraction when it
| comes to being productive. And I think new abstractions
| open new possibilities some times! But some abstractions
| which makes standard stuff easy, makes non-standard stuff
| impossible.
| jrmg wrote:
| It's not just new frameworks, it's new features. Good luck
| getting a LLM to write code that uses iOS 26 features, for
| example.
|
| I'm not convinced simply getting the LLM to inject
| documentation about the features will work well (perhaps
| someone has studied this?) because the reason they're good at
| doing 'well known' things is the plethora of actual examples
| they're trained on.
| pmontra wrote:
| It reminds me of this excerpt from Coders at Work, in Chapter
| 13 - Fran Allen:
|
| Seibel: When do you think was the last time that you
| programmed?
|
| Allen: Oh, it was quite a while ago. I kind of stopped when C
| came out.
|
| That was a big blow. We were making so much good progress on
| optimizations and transformations. We were getting rid of just
| one nice problem after another. When C came out, at one of the
| SIGPLAN compiler conferences, there was a debate between Steve
| Johnson from Bell Labs, who was supporting C, and one of our
| people, Bill Harrison, who was working on a project that I had
| at that time supporting automatic optimization.
|
| The nubbin of the debate was Steve's defense of not having to
| build optimizers anymore because the programmer would take care
| of it. That it was really a programmer's issue. The motivation
| for the design of C was three problems they couldn't solve in
| the high-level languages: One of them was interrupt handling.
| Another was scheduling resources, taking over the machine and
| scheduling a process that was in the queue. And a third one was
| allocating memory. And you couldn't do that from a high-level
| language.
|
| So that was the excuse for C.
|
| Seibel: Do you think C is a reasonable language if they had
| restricted its use to operating-system kernels?
|
| Allen: Oh, yeah. That would have been fine. And, in fact, you
| need to have something like that, something where experts can
| really fine-tune without big bottlenecks because those are key
| problems to solve.
|
| By 1960, we had a long list of amazing languages: Lisp, APL,
| Fortran, COBOL, Algol 60. These are higher-level than C. We
| have seriously regressed, since C developed. C has destroyed
| our ability to advance the state of the art in automatic
| optimization, automatic parallelization, automatic mapping of a
| high-level language to the machine. This is one of the reasons
| compilers are . . . basically not taught much anymore in the
| colleges and universities.
| donkeybeer wrote:
| The fact that things are still dog slow on modern hardware
| means we didn't go far enough into C like thinking. And
| compilers and hardware were much, much worse in the 70s.
| There is a reason C got adopted so quickly, and languages
| after those other languages took decades to become practical.
| That counter history is just made up nonsense. If we didn't
| have a convenient language back then, people would either
| just give up and keep writing assembly, or computing would
| have just slowed or stopped.
| jbreckmckye wrote:
| > To be honest I don't think this is necessarily a bad thing,
|
| I do. Would you really argue we discovered perfection in the
| first sixty years of computer science? In the first sixty years
| of chemistry we still believed in phlogiston
| Mathnerd314 wrote:
| Python is just a beautiful, well-designed language - in an era
| where LLM's generate code, it is kind of reassuring that they
| mostly generate beautiful code and Python has risen to the top.
| If you look at the graph, Julia and Lua also do incredibly well,
| despite being a minuscule fraction of the training data.
|
| But Python/Julia/Lua are by no means the most natural languages -
| what is natural is what people write before the LLM, the stuff
| that the LLM translates into Python. And it is hard to get a good
| look at these "raw prompts" as the LLM companies are keeping
| these datasets closely guarded, but from HumanEval and MBPP+ and
| YouTube videos of people vibe coding and such, it is clear that
| it is mostly English prose, with occasional formulas and code
| snippets thrown in, and also it is not "ugly" text but generally
| pre-processed through an LLM. So from my perspective the next
| step is to switch from Python as the source language to prompts
| as the source language - integrating LLM's into the compilation
| pipeline is a logical step. But, currently, they are too
| expensive to use consistently, so this is blocked by hardware
| development economics.
| gopiandcode wrote:
| mhhm yes yes. There's a thread of discussion that I didn't
| quite chose to delve into in the post, but there is something
| interesting to be found in the observation that languages that
| are close to natural language (Python being famous for being
| almost executable pseudo-code for a while) being easier for
| LLMs to generate.
|
| Maybe designing new languages to be close to pseudo-code might
| lead to better results in terms of asking LLMs to generate
| them? but there's also a fear that maybe prose-like syntax
| might not be the most appropriate for some problem domains.
| dack wrote:
| personally i think DSLs could be helpful if they are really good
| at: 1. explaining the syntax clearly 2. providing a fast checker
| that provides good error messages 3. prevents errors
|
| LLMs seem pretty good at figuring out these things when given a
| good feedback loop, and if the DSL truly makes complex programs
| easier to express, then LLMs could benefit from it too. Fewer
| lines of code can mean less context to write the program and
| understand it. But it has to be a good DSL and I wouldn't be
| surprised if many are just not worth it.
| quantadev wrote:
| Since DSLs are necessarily niche, you're never going to have
| much training data in that language to feed into LLM training.
| Sure this problem can be overcome, but you're just creating
| more work than the time saved by having humans code in DSLs.
| MoonGhost wrote:
| Me thinking, is it time for a new 'programming' language for LLM
| to use instead of tools API calls? Something high level with
| loose grammar, in between natural language and strict
| programming. Then the backend, may be another smaller model,
| translates it in API calls. With this approach backend can be
| improved and updated much faster and cheaper then LLM model.
| cpeterso wrote:
| Lisp returns as the programming language for AI!
| zzo38computer wrote:
| I do not use LLM and I continue to work in the older way.
|
| Also, domain-specific stuff can still be useful sometimes, and
| other stuff involved with designing a programming language.
| quantadev wrote:
| If someone's not using LLMs yet in 2025 to write code they're
| basically Amish.
|
| They're riding a horse in the age of automobiles, just because
| they think they're more comfortable on horseback, while they've
| never been in a car even once.
| SrslyJosh wrote:
| AI bros sound like NFT bros.
| quantadev wrote:
| 0.0001% of investors did NFTs. 99.999% of devs do LLMs.
| bluehatbrit wrote:
| I'd be interested to hear how you find staying away from them
| as the years progress.
|
| My experience so far is that they write mediocre code which is
| very often correct, and is relatively easy to review and
| improve. Of course I work with languages like elixir, python,
| typescript, and SQL - all of which LLMs are very good at.
|
| Without a doubt I've seen a significant increase in the amount
| of work I can produce. As far as I can tell the defect rate in
| my work hasn't changed. But the way I work has, I'm now
| reviewing and refactoring significantly more than before and
| hand writing a lot less.
|
| To be honest, I'd worry about someone's ability to compete in
| the job market if they resisted for much longer. With the
| obvious exceptions of spaces where LLMs can't be used, or have
| very poor performance.
| shakna wrote:
| In my experience, its harder to review.
|
| It'll dump you three classes and a thousand lines of code,
| where it should use a simple for loop to iterate.
|
| The code Claude, Gemini and Cursor produces still is not
| enough to pass half-decent quality checks. If you're in
| "compile=ship", sure.
|
| If you care about performance, or security, or
| maintainability, no. It's wasting your time, and the review
| team's time.
| api wrote:
| One of my big concerns (a little tangential) is that LLMs will
| have the effect of fixing programming language design and the
| current language landscape into stone. This could occur in
| proportion to their use by programmers. The languages that LLMs
| do the best and have in their training data will be the languages
| programmers use, and getting any new language into LLM data sets
| will be very hard.
| jp57 wrote:
| Programming languages researchers and designers labor under the
| mistaken assumption that programming practitioners--people who
| are writing programs to solve problems--actually _want_ "a
| language with a syntax and semantics tailored for a specific
| domain", or any really fancy language features at all.
|
| I say this from the perspective of someone who nearly became a PL
| researcher myself. I could easily have decided to study
| programming languages for my PhD. Back then I was delighted by
| learning about cool new languages and language features.
|
| But I did didn't study PL but rather ML, and then I went into
| industry and became a programming practitioner, rather than a PL
| researcher. I don't want a custom-designed ML programming
| language. I want a simple general-purpose language with good
| libraries that lets me quickly build the things I need to build.
| ( _cough_ Python _coughcough_ )
|
| Now that I have reached an age where I am aware of the finiteness
| of my time left in this universe, my reaction when I encounter
| cool new languages and language features now my is to wonder if
| they will be worth learning. Will the promised productivity gains
| allow me to recoup the cost of the time spent learning. My usual
| assessment is "probably not" (although now and then something
| worthwhile does come along).
|
| I think that there is a very real chance that the idea of
| specialized programming languages will indeed disappear in the
| LLM era, as well as the need for various "ergonomic" features of
| general purpose languages that exist only to make it possible to
| express complex things in fewer lines of code. Will any of that
| be needed if the LLM can just write the code with what it has?
| ackfoobar wrote:
| > Will the promised productivity gains allow me to recoup the
| cost of the time spent learning.
|
| Some deep PL stuff I doubt there is productivity gain to begin
| with. But many ideas in the ML language family are simple and
| reduce debugging pain. Time lost from one encounter with muddy
| JS/Python semantics is more than the time learning about sum
| types.
| freedomben wrote:
| Do you have any opinions on elixir? Have you played around
| with reasonML?
| fud101 wrote:
| neither of those have a realistic payoff in the real world.
| talking money coming into an account in exchange for jira
| points available. It's different in SV but elsewhere in the
| world it's not a realistic proposition.
| ackfoobar wrote:
| Haven't played with either. I don't think OCaml needs a new
| syntax. Maybe Reason need it for JSX but I don't really do
| frontend.
| izabera wrote:
| 30 lines are always going to be easier to read/write/debug than
| 3000 lines, so it'll probably remain easier (for both humans
| and machines) to write correct code in languages that make it
| possible to express ideas concisely and elegantly.
| tptacek wrote:
| From elsewhere on the front page: "given choice between
| complexity or one on one against t-rex, grug take t-rex: at
| least grug see t-rex".
| noelwelsh wrote:
| So you write everything in assembly then?
| doug_durham wrote:
| That is a false dichotomy. It is not 3000 versus 30. It's
| 3000 versus 2578 where the DSL is poorly documented by
| someone who long ago lost interest in maintaining it. I
| prefer the initial example in the article to the DSL
| equivalent. I can immediately read the non-DSL code and be
| productive. The DSL looks like a vanity project.
| AnimalMuppet wrote:
| Hmm. Go was written to be easier for junior developers to use,
| and to be maintainable "in the large".
|
| I wonder if we need a language designed to be easier for an AI
| to reason about, or easier for a human to see the AI's
| mistakes.
| freedomben wrote:
| I think in 5 years you are absolutely right, but for now we
| desperately need a language which is easy for ai and easy for
| humans. The stuff that AI has suggested to me is about 50%
| genius in about 50% idiocy. I don't know what that language
| is, if I could pick it personally, I would pick elixir, but I
| recognize that we're probably looking at typescript or
| something like that
| doug_durham wrote:
| I totally see "read-only" language emerging in the future. A
| language that is never written by humans yet has unambiguous
| syntax and semantics.
| username223 wrote:
| That sounds grim, but not implausible. Domain-specific
| languages seemed like a significant improvement over general-
| purpose languages plus libraries. But now that we have a tool
| that lets you make Jazz Hands at your computer and have it spit
| out something that does most of what you want, do they really
| help?
|
| Maybe some boring, kind-of-consistent language like C, Python,
| or Go is good enough. An LLM spits out a pile of code in one or
| more of them that does most of what you want, and you can fix
| it because it's less opaque than assembly. It doesn't sound
| like a job I'd want, but maybe that's just the way things will
| go.
| noelwelsh wrote:
| Embedded DSLs (e.g. PyTorch) have been hugely successful in the
| field of machine learning, so I think there is a bit of nuance
| here that you're not considering.
|
| I also take issue with the idea that Python is simple. Python's
| semantics are anything but. The biggest issue the language has,
| performance, is a consequence of these poorly thought out
| semantics. If the language was actually simple it would be a
| lot easier to build a faster implementation.
| doug_durham wrote:
| I teach beginner Python classes. If the semantics were so
| horribly convoluted we wouldn't be teaching it to beginners.
| TeMPOraL wrote:
| On the one hand, this sucks. On the other hand, we're already
| vacillating along the Pareto frontier of how much we can stuff
| into code; in fact, most of the criticisms of DSLs are indirectly
| stating just that.
|
| So with LLMs making it easier to project back and forth between
| how programmer sees the task at hand, and the underlying
| dumb/straightforward code they ain't gonna read anyway, maybe
| we'll finally get to the point of addressing the actual problem
| of programming language design, which is that _you cannot
| optimize for every task and cross-cutting concern at the same
| time and expect improvement across the board_ - we 're already at
| the limit, we're just changing which time of day/part of the
| project will be more frustrating.
| guelo wrote:
| This sounds insightful but I can't make heads or tails of "you
| cannot optimize for every task and cross-cutting concern at the
| same time and expect improvement across the board".
|
| Can someone help me out?
| le_marton wrote:
| I understand it that way:
|
| The general programming languages we have now are about as
| good as they are ever going to get. Balance-wise. One can
| solve problems across a broad range of topics equally well.
|
| Tailoring a language to make some tasks especially easy /
| straightforward will likely make some other problems a lot
| harder / more cumbersome to express.
|
| So further improvements in abstraction and expressiveness
| have to come from elsewhere. Not better programming languages
| but partnering up with an LLM?
| TeMPOraL wrote:
| > _I understand it that way: (...)_
|
| You understood and explained what I meant well.
|
| > _So further improvements in abstraction and
| expressiveness have to come from elsewhere. Not better
| programming languages but partnering up with an LLM?_
|
| In practice, probably yes; but the point I've been making
| here and in bringing this up over the years is, further
| improvements can come if we stop insisting on directly
| editing "single source of truth" code representation. Then,
| you no longer need to make trade-offs up front - for task
| A, you may view code through lens (or "in language") that
| makes tasks like A especially easy, then for task B you
| switch to a view that makes tasks like B especially easy.
|
| Examples:
|
| - Switching between "point free style" and explicit
| temporary variables;
|
| - For a given function, inlining its entire call subtree,
| turning a tiny function calling tiny functions into a
| single block you can read top to bottom; super useful for
| debugging;
|
| - Switching between "exceptions" and "sum types" styles of
| error handling (they're pretty much equivalent), or more
| generally, switching between showing all vs. hiding error
| handling code vs. handling everything _except_ error
| handling /propagating code;
|
| - A view of code that shows only types, or one showing
| async, or one entirely hiding it;
|
| - A database-style view for code, that lets you query it
| and edit in bulk;
|
| - An editable state machine diagram that corresponds to
| code (maybe needs a little help by manually identifying
| which set of classes is conceptually a state machine, which
| methods are transitions, etc.);
|
| - Hide/show logging, telemetry, or any other cross-cutting
| concern that you don't strictly care about at the moment
| (superset of earlier hiding of error handling code);
|
| And so on, and so on.
|
| Point being, all those views/perspectives operate on the
| same underlying artifact - the codebase. It can be
| plaintext, but because no one is actually reading it
| directly, it doesn't have to be optimized for anything in
| particular (or it could just be simple and straightforward,
| at a price of being verbose; say Python). Meanwhile, the
| views/perspectives could each use syntax or format best
| suited for the specific task it helps with. All the trade-
| offs would be made at the point of use, instead of baked in
| up front when the project starts.
| guelo wrote:
| That does sound useful but it seems like it should be
| fancy features in the IDE, not the language.
| boznz wrote:
| Unfortunately, English-as-a-programming-language * is now a thing
| and there will be a lot of bad/dangerous/untested code being used
| in real situations going forward.
|
| * Not just English, substitute any other human language into the
| above
| diimdeep wrote:
| I think academic programming language research is reeks of fraud
| and uselessness. Prove me wrong.
| kccqzy wrote:
| A lot of features you take granted in new languages come from
| academic programming language research. Generics in Java for
| example came from GJ, an academic programming language research
| project headed by academics including Philip Wadler.
| munificent wrote:
| Since I work on a language professionally, I think about this all
| the time.
|
| As someone who loves a wide diversity of actively evolving
| programming languages, it makes me sad to think those days of
| innovation may be ending. But I hope that's not going to happen.
|
| It has always been the case that anyone designing a new language
| or adding features to an existing one is acutely mindful of what
| programming language knowledge is already in the heads of their
| users. The reason so many languages, say, use `;` for statement
| terminators is not because that syntax is particularly beautiful.
| It's just familiar.
|
| At the same time, designers assume that giving users a _better_
| way to express something may be worth the cost of asking them to
| learn and adapt to the new way.
|
| In theory, that should be true of LLMs as well. Yes, a new
| language feature may be hard to get the LLM to auto-complete. But
| if human users find that feature makes their code easier to read
| and maintain, they still want to use it. They will, and
| eventually it will percolate out into the ecosystem to get picked
| up the next time the LLMs are trained, in the same way that human
| users learn new language features by stumbling onto it in code in
| the wild.
|
| So I'd like to believe that we'll continue to be able to push
| languages forward even in a world where a large fraction of code
| is written by machines. I also hope that LLM training cost goes
| down and frequency goes up, so that the lag behind what's out
| there in the world and what the LLMs know gets smaller over time.
|
| But it's definitely possible that instead of that, we'll get a
| feedback loop where human users don't know a language feature
| even exists because the LLMs never generate code using it, and
| the LLMs never learn the feature exists because humans aren't
| writing it.
|
| I have this same fear about, well, basically everything with
| LLMs: an endless feedback loop where humans get their
| "information" from LLMs and churn out content which the LLMs
| train on and the whole world wanders off into a hallucinatory
| bubble no longer grounded in reality. I don't know how to get
| people and/or the LLMs to touch grass to avoid that.
|
| I do hope I get to work on making languages great for humans
| first, and for LLMs second. I'm way more excited to go to work
| making something that actual living breathing people use than as
| input data for a giant soulless matrix of floats.
| oleks wrote:
| Using an LLM to generate code is not an easily traceable and
| explainable process. Using a DSL to same ends is. PL research has
| yet to meet explainability in AI head on.
| prats226 wrote:
| For LLMs, programming languages are basically additional
| languages that we speak. So how it handles low-resource
| programming languages is same as how it handles speaking
| languages with less contribution in training data?
|
| DSL's would be even harder for LLM's to get right in that case
| compared to the low-resource language itself
| rramon wrote:
| Sooner or later languages are going to become 3d diagram to code
| IDEs where you just zoom in to get to lower levels of detail.
| keybored wrote:
| Articles in the era of LLMs: assume endless torrent of LLM code
| generation forever, insert how will this affect X now that we
| have our foregone conclusion.
| kccqzy wrote:
| Python is an acceptable though not perfect substrate for
| developing embedded DSLs. It's dynamic enough that you can do a
| lot of things. Besides operator overloading which is commonly
| used in C++ for eDSLs, you can even write decorators that take
| the AST, completely regenerate new code via LLVM or something
| similar. This is the approach used by numba for JIT for example.
|
| In the end I think mentioning Python is a red herring. You can
| produce an eDSL in Python that's not in LLM training data so
| difficult for LLMs to grok, and yet still perfectly valid Python.
| The deeper issue here is that even if you use Python, LLMs are
| restricting people to use a small subset of what Python is even
| capable of.
| kibwen wrote:
| People often use the analogy of LLMs being to high-level
| languages what compilers were for assembly languages, and despite
| being a terrible analogy there's no guarantee it won't eventually
| be largely true in practice. And if it does come true, consider
| how the advent of the compiler completely eliminated any
| incentive to improve the ergonomics or usability of assembly
| code, which has been and continues to be absolute crap, because
| who cares? That could be the grim future for high-level
| languages; this may be the end of the line.
| Terr_ wrote:
| I suspect the important sticking point will be reliability. The
| "incentive" exists because of an high degree of trust, so much
| so that "junior dev thinks it's a compiler bug" is a kind of
| joke.
|
| If compilers had significant non-deterministic error rates with
| no reliable fix, that would probably be a rather different
| timeline.
| kibwen wrote:
| Yes, that's why I say it's a terrible analogy, but we live in
| a crap world with a fetish for waste and no incentive for
| quality, so I fear for a future where every night we just
| have the server farm spin up a thousand batch jobs to
| regenerate the entire codebase from scratch and pick out the
| first candidate that passes today's test suite.
| daxfohl wrote:
| A big difference is that compilers are deterministic, and
| coders generally don't review and patch the generated assembly.
| There's little reason to expect that LLMs will ever function
| like that. It's always going to be a back-and-forth of, "hey
| LLM code this up", "no, function f isn't quite right; do this
| instead", etc.
|
| This mimics what you see in, say, Photoshop. You can edit
| pixels manually, you can use deterministic tools, and you can
| use AI. If you care about the final result, you're probably
| going to use all three together.
|
| I don't think we'll ever get to the point where we a-priori
| present a spec to an LLM and then not even look at the code,
| i.e. "English as a higher-level coding language". The reason
| is, code is simply more concise and explicit than trying to
| explain the logic in English in totality up-front.
|
| For some things where you truly don't care about the details
| and have lots of flexibility, maybe English-as-code could be
| used like that, similar to image generation from a description.
| But I expect for most business-related use cases, the world is
| going to revolve around actual code for a long time.
| seanmcdirmid wrote:
| If LLMs can ever produce implementation and tests opaquely,
| then perhaps we can instruct at the level of requirements.
| They will never be good enough at the beginning and we will
| just provide feedback over the working prototype the LLM
| generates and demonstrates?
|
| I haven't seen technology move this fast before, so I
| wouldn't make any hard predictions about how long actual code
| written by humans survives. We don't really need AGI at this
| point to have opaque coding solutions, even if the LLMs
| should still be better.
| daxfohl wrote:
| I won't say "never", but I guess I left out the caveat that
| I tried this for a hack week demo once. Working in
| compliance and onboarding, there's a whole lot of logic
| around what you have to collect and check, what you have to
| do around user experience, what steps you can fall back to
| if something fails, etc. I tried to take all our code and
| translate it to English to feed to an LLM and see what it
| would do. But the final result was that, even if it worked
| perfectly, the English description was longer and more
| convoluted than the code, would've been harder to make
| changes to, yada yada, so even if it worked perfectly, it
| would be a worse option than code.
|
| But maybe there's a future where a "best of both worlds"
| that intermingles structured code with unstructured verbal
| instructions, so that you can ensure that the important
| aspects of your requirements are explicit and
| deterministic, and the filler parts can be in English
| descriptions. It'd compile, and you get red squigglies in
| your editor if something doesn't make sense, maybe even
| type hints on the English somehow. I think that'd be a
| pretty good "best of both worlds" because it'd really let
| you separate the signal from the noise, which is the
| problem you have when using either English or programming
| languages distinctly.
| daxfohl wrote:
| Though now that I say that, I wonder if maybe the
| approach should be, instead of specifying the onboarding
| procedure in terms of steps, maybe it would be better to
| specify a goal, the constraints, and some optimization
| function, and allow the AI to figure out what the
| workflow steps should be. So it would act higher level,
| more as a constraint solver than a code generator. That
| could be easier to interact with, allow constraints to be
| read and updated more easily, and not require diving much
| into code at all.
|
| Hmm, I don't think LLMs are quite there yet, but I could
| see this being a potential way to do things, that may fit
| better with their strengths, and allow more to be done
| without digging into actual code.
|
| Now I just need to dust off my old Prolog books and
| figure out how to use coding assistants in a similar
| fashion.
| noobermin wrote:
| An ignorant perspective, from someone likely hasn't coded
| assembly ever. Assembly is tied to the system you target and it
| can't really be "improved". You can however improve ergonomics
| greatly via macros and everyone does this.
| kibwen wrote:
| _> Assembly is tied to the system you target and it can 't
| really be "improved"._
|
| Of course it can. There's no reason for modern extensions to
| keep pumping out instructions named things like "VCTTPS2DQ"
| other than an adherence to cryptic tradition and a confidence
| that the people who read assembly code are poor saps who
| don't matter in the grand scheme of the industry, which is
| precisely my point. And even if x86 was set in stone
| centuries ago, there's no excuse for modern ISAs to follow
| suit other than complete apathy over the DX of assembly, and
| who can blame them?
|
| _> You can however improve ergonomics greatly via macros and
| everyone does this._
|
| Yes, and surely you see how the existence of macro assemblers
| strengthens my argument?
| noobermin wrote:
| >poor saps who don't matter in the grand scheme of the
| industry
|
| Someone is mad. Just because you don't have the patience
| for it doesn't mean everyone has the same preferences as
| you do.
|
| Anyway, one could argue macros in assembly are part and
| parcel of the process if you develop things in assembly of
| significant complexity. If you don't like a particular
| instruction name, you could always relabel it in a macro to
| something you like. "Redesigning assembly" of an existing
| target otherwise makes no sense as a concept as assembly
| languages are usually specified by ISA designers as a
| target to meet the developers of compilers.
|
| You can of course write your own assembler for yourself if
| you really want. That's the beauty of asm, you don't have
| to meet ideological targets for this or that PL school. You
| just need to admit the right byte codes, and that's
| sufficient. It doesn't guarantee things will work how you
| want though easily in a higher level sense, which is why
| most people use programming languages.
| kibwen wrote:
| _> > poor saps who don't matter in the grand scheme of
| the industry_
|
| _> Someone is mad_
|
| You're completely misunderstanding me. This isn't my
| personal opinion, it's simply an observation of the utter
| indifference of ISA designers to any consideration of
| ergonomics or developer experience. The rest of your
| comment aptly explains ways to work around this fact
| while failing to understand that the whole point of this
| thread is that it didn't have to be this way. For
| example, there's no reason that an assembly language
| couldn't have basic syntactical affodances like
| namespaced identifiers or pseudo-structural elements like
| basic if, while, and switch. Hell, in the 80s we had
| chips that executed Lisp, and today we might still have
| some brave souls making chips that execute Forth. There's
| no law of the universe that says that assembly has to be
| jank and primitive.
| bee_rider wrote:
| We got LLVM IIR which is sort of like... similar-ish to
| assembly but better and more portable, right? Maybe some
| observation could be made there--it is something that does a
| similar job, but does it in a way that is better for the job
| that actually remains.
| kkukshtel wrote:
| I've been drafting a blog post on this as well. My take is that
| programming langauges largely evolve around "human" ergonomics
| and solve for "humans writing the code", but that can result in
| code that is too abstract and non-performant. I think where LLMs
| will succeed (more) is in writing _very dumb verbose code_ that
| can be easily optimized by the compiler.
|
| What humans look at and what an AI looks at right now are similar
| only by circumstance, and what I sort of expect is that you start
| seeing something more like a "structure editor" that expresses
| underlying "dumb" code in a more abstract way such that humans
| can refactor it effectively, but what the human sees/edits isn't
| literally what the code "is".
|
| IDK it's not written yet but when it is it will be here:
| https://kylekukshtel.com/llms-programming-language-design
| maybevoid wrote:
| Coincidentally, I released a DSL last week called Hypershell [1],
| a Rust-based domain-specific language for shell scripting at the
| type level. While writing the blog post, I found myself
| wondering: will this kind of DSL be easier for LLMs to use than
| for humans?
|
| In an initial experiment, I found that LLMs could translate
| familiar shell scripting concepts into Hypershell syntax
| reasonably well. More interestingly, they were able to _fix_
| common issues like type mismatches, especially when given light
| guidance or examples. That's a big deal, because, like many
| embedded DSLs, Hypershell produces verbose and noisy compiler
| errors. Surprisingly, the LLM could often identify the underlying
| cause hidden in that mess and make the right correction.
|
| This opens up a compelling possibility: LLMs could help bridge
| the usability gap that often prevents embedded DSLs from being
| more widely adopted. Debuggability is often the Achilles' heel of
| such languages, and LLMs seem capable of mitigating that, at
| least in simple cases.
|
| More broadly, I think DSLs are poised to play a much larger role
| in AI-assisted development. They can be designed to sit closer to
| natural language while retaining strong domain-specific
| semantics. And LLMs appear to pick them up quickly, as long as
| they're given the right examples or docs to work with.
|
| [1] https://contextgeneric.dev/blog/hypershell-release/
| ethan_smith wrote:
| Your experience with Hypershell points to an interesting
| possibility: LLMs as DSL translators rather than replacements.
| This could actually democratize DSLs by lowering the learning
| curve while preserving their domain-specific benefits. The real
| opportunity might be DSLs optimized for both human semantics
| and machine translation.
| amterp wrote:
| I've been working on a programming language for about a year
| which aims to replace Bash for scripting but is far closer to
| Python. It's something I hope to see used by many other people in
| the future, but a very common objection I hear from people I
| pitch it to is "yeah but an LLM could just generate me a Python
| script to do this, sure it might be uglier, twice as long, and
| not work quite as well, but it saved me from learning a new
| language and is probably _fine_ ", to which I have lots of
| counters on why that's a flawed argument, but it still
| demonstrates what I think is an increase in people's skepticism
| towards new languages which will contribute to the stagnation the
| author is talking about. If you're writing a new language, it's
| demotivating to see people's receptiveness to something new
| diminish.
|
| I don't blame anyone in the picture, I don't disagree that time
| saved with LLMs can be well worth it, but it still is a topic I
| think we in the PL community need to wrestle more with.
| nylonstrung wrote:
| I love your idea.
|
| LLMs are surprisingly bad at bash and apparently very bad at
| Powershell
|
| Pythonic shell scripting is well suited to their language
| biases right now
| amterp wrote:
| Thanks! :) Yeah in my experience they're _okay_ at Bash but I
| 'm never happy with the scripts it makes, I feel like the
| ceiling is pretty low on how good Bash scripts can get
| without a disproportionate amount of effort.
|
| I'm taking a lot of inspiration from Python in the syntax and
| I'm actually worried it will trip up any LLM I train on my
| language since I think it will risk probabilistically just
| reverting to Python mid-answer Time will tell
| furyofantares wrote:
| I notice I am confused.
|
| > Suddenly the opportunity cost for a DSL has just doubled: in
| the land of LLMs, a DSL requires not only the investment of build
| and design the language and tooling itself, but the end users
| will have to sacrifice the use of LLMs to generate any code for
| your DSL.
|
| I don't think they will. Provide a concise description + examples
| for your DSL and the LLM will excel at writing within your DSL.
| Agents even moreso if you can provide errors. I mean, I guess the
| article kinda goes in that direction.
|
| But also authoring DSLs is something LLMs can assist with better
| than most programming tasks. LLMs are pretty great at producing
| code that's largely just a data pipeline.
| gopiandcode wrote:
| Arguably it really depends on your DSL right? If it has a
| semantics that already lies close to existing programming
| languages, then I'd agree that a few examples might be
| sufficient, but what if your particular domain doesn't match as
| closely?
|
| Examples of domains that might be more challenging to design
| DSLs for: languages for knitting, non-deterministic languages
| to represent streaming etc. (i.e
| https://pldi25.sigplan.org/details/pldi-2025-papers/50/Funct...
| )
|
| My main concern is that LLMs might excel at the mundane tasks,
| but struggle at the more exciting advances, and so now the
| activation energy for coming up with advances DSLs is going to
| increase and as a result, the field might stagnate.
| demosthanos wrote:
| Remember that LLMs aren't trained on all existing programming
| languages, they're trained on all text on the internet. They
| encode information about knitting or streaming or whatever
| other topic you want a DSL for.
|
| So it's not just a question of the semantics matching
| existing programming languages, the question is if your
| semantics are intelligible given the vast array of semantic
| constructs that are encoded in _any_ part of the model 's
| weights.
| daxfohl wrote:
| This is somewhat my take too. The way most vibe coding happens
| right now does create a lot of duplication because it's cheap
| and easy for LLMs to do. But eventually as the things we do
| with coding assistants become more complex, they're not
| necessarily going to be able to deal with huge swaths of
| duplicate code any better than humans are. Given their limited
| context size, having a DSL that allows them to fit more logic
| into their context with fewer tokens, we could conceivably see
| the importance of DSLs start to increase rather than decrease.
| neilv wrote:
| I was about to paste the same sentence, and say much the same
| thing in response.
|
| To add to that... One limitation of LLM for a new DSL is that
| the LLM may be less likely to directly plagiarize from open
| source code. That could be a feature.
|
| Another feature could be users doing their own work, and doing
| a better job of it, instead of "cheating on their homework"
| with AI slop and plagiarism, whether for school or in the
| workplace.
| loa_in_ wrote:
| Every language other than machine level assembly instructions
| as human readable code is a DSL.
| outofpaper wrote:
| Depending on the size of a DSL all the more recent LLMs can be
| employed to work with them. LoRA/Finetuning are a heavier option,
| followed by RAG, and just setting them up as a big system prompt
| and caching. And once a model is able to work with a DSL tokens
| used in valuable code creation can dramatically drop.
| aaronvg wrote:
| We're making a prompting DSL (BAML
| https://github.com/BoundaryML/baml) and what we've found is that
| all the syntax rules can easily be encoded into a Cursor Rules
| file, which we find LLMs can follow nicely. DSLs are simple by
| nature so there's not too many rules to define.
|
| Here's the cursor rules file we give folks:
| gist.github.com/aaronvg/b4f590f59b13dcfd79721239128ec208
| mbokinala wrote:
| Anecdotally, Cursor's tab complete model learns BAML incredibly
| quickly
| wolpoli wrote:
| I think designers will end up needing to work with an existing
| LLM or even provide their own LLM to get developers to adopt a
| new language/library/feature. The market has been heading in that
| direction for a while now, with developers expecting official
| docs and tutorials.
| freedomben wrote:
| Interesting, I tend to think design is a dying job now. It's
| not that I don't see value in designers, but if I am being
| honest, when they aren't advocating for a complete UI refresh,
| there isn't a whole lot for them to do. IMHO this is why we see
| all the major apps refreshing their UIs every 9 to 12 months.
| It is unnecessary and aggravating to users, and if we can get
| away from that I think we would be better off.
| noobermin wrote:
| Why care what others are doing? Just do what makes sense for your
| domain and don't worry about what is hot. Who cares?
|
| If something is useful people will use it. Just because it seems
| like llms are everywhere, not everyone cares. I wouldn't want
| vibe coders to be my target audience anyway.
| chr15m wrote:
| It's it just me or does the graph of LLM language performance
| versus training set size show the opposite of what they are
| saying? To me it looks flat, implying training set size has
| little influence on LLM performance in the language. For instance
| some niche languages appear to out-perform better known languages
| (with more variance in the niche language performance).
| csomar wrote:
| It is not only you, but I think it is only you and me. I've
| also skimmed through the comments and wondering if they are AI
| generated; or the people even read the article. The author
| essentially took a graph, and then claimed a different
| interpretation to reality.
|
| What the graph shows is that LLMs struggle with "hard"
| languages (Rust, Go, C#) with the exception of Ruby.
| nxobject wrote:
| Replace "DSL" with "languages" in general - the same issues
| apply. I'm not sure that in a hypothetical timeline where Rust
| was released today, it would have gotten any traction.
| eru wrote:
| I wonder if this will be temporary?
|
| As AI systems improve, and especially as they add more 'self-
| play' in training, they might become really good at working in
| any language you can throw at some.
|
| (To expand on the self-play aspect: when training you might want
| to create extra training data by randomly creating new 'fake'
| programming languages and letting it solve problems in them. It's
| just another way to add more training data.)
|
| In any case, if you use an embedded DSL, like is already commonly
| done in Haskell, the LLMs should still give you good performance.
| In some sense, an 'embedded DSL' is just fancy name for a library
| in a specific style.
| NiloCK wrote:
| I, too, wrote a rambling take on the potential for LLM-induced
| stack ossification about 6 months ago:
| https://paritybits.me/stack-ossification/
|
| At the time, I had given in to Claude 3.5's preference for python
| when spinning up my first substantive vibe-coded app. I'd never
| written a line of python before or since, but I just let the
| waves carry me. Claude and I vibed ourselves into a corner, and
| given my ignorance, I gave up on fixing things and declared the
| software done as-is. I'm now the proud owner of a tiny
| monstrosity that I completely depend on - my own local whisper
| dictation app with a system tray.
|
| I've continued to think about stack ossification since. Still
| feels possible, given my recent frustration trying to use animejs
| v4 via an LLMs. There's a substantial api change between animejs
| v3 and v4, and no amount of direction or documentation placed in
| context could stop models from writing against the v3 api.
|
| I see two ways out of the ossification attractor.
|
| The obvious, passive, way out: frontier models cross a chasm with
| respect to 'putting aside' internalized knowledge (from the
| training data) in favor of in-context directions or some
| documentation-RAG solutions. I'm not terribly optimistic here -
| these models are hip-shooters by nature, and it feels to me that
| as they get smarter, this reflex feels stronger rather than
| weaker. Though: Sonnet 4 is generally a better instruction-
| follower than 3.7, so maybe.
|
| The less obvious way out, which I hope someone is working on, is
| something like massive model-merging based on many cached micro
| fine-tunes against specific dependency versions, so that each
| workspace context can call out to modestly customized LLMs (LoRA
| style) where usage of incorrect versions of your dependencies has
| _specifically been fine-tuned out_.
| adsharma wrote:
| > Language Design Direction 1: Teaching LLMs about DSLs (through
| Python?)
|
| This is what I've been focused on last few years with a bit of
| Direction 3 via python -> smt2 -> z3 -> verified
| rust
|
| Perhaps a diffusion model for programming can be thought of as:
|
| requirements -> design -> design by contract -> subset of python
| -> gc capable language (a fork of golang with ML features?) ->
| low level compiled language (rust, zig or C++)
|
| As you go from left to right, there is an increasing level of
| detail the programmer has to worry about. The trick is to pick
| the right level of detail for a task.
|
| Previous writing: https://adsharma.github.io/agentic-transpilers/
| jo32 wrote:
| In the LLM era, building a brand-new DSL feels unnecessary. DSLs
| used to make sense because they gave you a compact, domain-
| specific syntax that simple parsers could handle. But modern
| language models can already read, write, and explain mainstream
| languages effortlessly, and the tooling around those languages--
| REPLs, compilers, debuggers, libraries--is miles ahead of
| anything you'd roll on your own. So rather than inventing yet
| another mini-language, just leverage a well-established one and
| let the LLM (plus its mature ecosystem) do the heavy lifting.
| jeroenhd wrote:
| I can't even trust an LLM to write working Java code, let alone
| trust it to convert whatever a DSL is supposed to express into
| another form. Sure, maybe there's not enough Java 23 in its
| training set to effectively copy into my application, but Java
| 11 combined with 10 year old libraries shouldn't be a problem
| if these coding LLMs are worth their salt.
|
| Until LLMs stop making up language features, methods, and
| operators out of convenience, DSLs are here to stay.
| KaiserPro wrote:
| I fucking hate DSLs however I know they need to exist, but
| nowhere near as many should exist as they do now.
|
| However LLMs are actually quite useful for translating
| concepts in DSLs that you don't understand. They don't so it
| error free, of course, but allows one to ask enough questions
| to work out why _your_ attempt to translate concepts into
| this new fucking stupid ontological pile of wank isn 't
| working
| cpard wrote:
| > Let's start with what I see as the biggest problem that the
| introduction of LLMs is presenting to language design: everything
| is easier in Python.
|
| This is so true.
|
| A couple months ago I was trying to use LLMs to come up with code
| to parse some semi-structured textual data based on a brief
| description from the user.
|
| I didn't want to just ask the LLM to extract the information in a
| structured format as this would make it extremely slow when
| there's a lot data to parse.
|
| My idea was, why not ask the LLM to come with a script that does
| the job. Kind of "compiling" what the user asks into a
| deterministic piece of code that will also be efficient. The LLM
| just has to figure out the structure and write some code to
| exploit it.
|
| I also had the bright idea to define a DSL for parsing, instead
| of asking the LLM to write a python script. A simple DSL for a
| very specific task should be better than using something like
| Python in terms of generating correct scripts.
|
| I defined the DSL, created the grammar and an interpreter and I
| started feeding the grammar definition to the LLM when I was
| prompting it to do the work I needed.
|
| The result was underwhelming and also hilarious at some times.
| When I decided to build a loop and feed the model with the errors
| and ask to correct the script, I ended up sometimes having the
| model returning back python scripts, ignoring completely the
| instructions.
|
| As the author said, everything is easier in Python, especially if
| you are a large language model!
| alganet wrote:
| > a DSL requires not only the investment of build and design the
| language and tooling itself
|
| Not necessarily true. There are two kinds of DSLs: external and
| internal.
|
| An external DSL has its own tooling, parser, etc. The nix
| language, for example.
|
| An internal DSL is like a small parasite that lives inside an
| existing language, reusing some of its syntax and tools. It's
| almost like intentional pareidolia. Like jQuery, for example.
|
| Internal DSLs reduce the cognitive load, and in my opinion,
| they're the best kind of DSL.
| andybak wrote:
| I've been thinking about the impact on visual programming. I've
| believed for a long time that any visual programming environment
| should have flawless round-tripping with a human
| readable/writable text representation (for many reasons - version
| control, automation, leveraging decades of tooling around text
| files, the fact that some tasks are just _easier_ with text)
|
| LLMs just add another reason to this list.
| usrbinbash wrote:
| Good. I'll chalk that up as one of the positive effects LLMs have
| on the software development environment (god knows there are few
| enough).
|
| DSL proliferation is a problem. I know this is not something many
| people care to hear, and I symphasize with that. Smart people are
| drawn to complexity and elegance, smart people like building
| solutions, and DSLs are complex and elegant solutions. I get it.
|
| Problem is: Too many solutions create complexity, and complexity
| is the eternal enemy of [Grug][1]
|
| Not every other problem domain needs its own language, and
| existing languages are designed to be adapted for many different
| problem domains. If LLMs help to stifle the wild growth of at
| least some DSLs that would otherwise be, then I am reasonably
| okay with that.
|
| [1]: https://grugbrain.dev
| nothrabannosir wrote:
| This feels like survivorship bias. Many of those older tools
| seem like they were once fancy new DSLs. We just respect them
| now as established, because they've been around for so long.
| But for every one thousand awkward DSLs that didn't make it,
| one new tool emerged which lifts software development to a new
| level.
|
| Would you say the same about a parallel universe where LLMs
| were introduced in 1960?
| loa_in_ wrote:
| So many unsuccessful (for some definition of the word) pet
| projects are on GitHub. The code techniques employed there are
| valid, working software, that aren't found in CRM, CRUD,
| middleware or data entry software that makes up most of the
| world's portfolio to date and that makes them the most valuable
| addition to LLM training data. Arguing against making such
| projects is insanity.
| romaniv wrote:
| The title should be "DSLs pose an interesting problem for LLM
| users".
|
| It is significant that LLMs in coding are being promoted based on
| a set of promises (and assumptions) that are getting instantly
| and completely reversed the moment the technology gets an iota of
| social adoption in some space.
|
| "Everyone can code now!" -> "Everyone must learn a highly
| specialized set of techniques to prompt, test generated code,
| etc."
|
| "LLMs are smart and can effortlessly interface with pre-existing
| technologies" -> "You must adopt these agent protocols, now"
|
| "LLMs are great at 0-shot learning" -> "I will not use this
| language/library/version of tool, because my model isn't trained
| on its examples"
|
| "LLMs effortlessly understand existing code" -> "You must change
| your code specifically to be understood by LLMs"
|
| This is getting rather ridiculous.
| oehpr wrote:
| Don't make me get the graph.
|
| https://upload.wikimedia.org/wikipedia/commons/9/94/Gartner_...
| efitz wrote:
| I am not a fan of DSLs. Perhaps there are use cases where they
| are the best tool, but in general they impose a significant
| learning burden on those who join a project that uses a DSL. I've
| seen several DSL projects wither and die because no one wanted to
| learn the DSL because the knowledge and time investment did not
| transfer forward to anything else they would do in the future. My
| personal opinion is that DSLs are vanity projects; one can
| usually come very close to DSL clarity and simplicity by adding
| appropriate methods or functions. You just don't get fancy
| syntactic sugar.
|
| Maybe DSLs are "write-only" languages for humans.
|
| I don't wish ill or sadness on anyone but it doesn't bother me at
| all if LLMs drive DSLs into extinction.
| cogman10 wrote:
| IMO, the best way to approach a DSL is simply using existing
| languages with a fairly flexible syntax. We've done this with
| groovy and it's worked quiet well. If I were to do it again I'd
| probably pick something like kotlin or ruby instead just
| because they both seem to have more industry relevance.
|
| The beauty of picking an existing language as the base is you
| often get an expansive standard library from the get-go. That
| means your job as a "DSL" writer is more based on making sure
| you provide the value adds that make sense for the writers of
| that DSL.
|
| It's worked particularly well for us because we have a data
| intake pipeline that has to parse and handle all sorts of
| random garbage (emails, excel docs, csv files, pdfs, etc).
|
| A language like groovy, ruby, and kotlin all work well because
| it's trivial to add extensions to the syntax in a way that
| makes sense for your domain problem. Typescript also wouldn't
| be a bad choice for similar reasons, the only reason I wouldn't
| consider it is we run a JVM backend and parsing typescript for
| the JVM is somewhat of a PITA.
| waffletower wrote:
| Hopefully, this will be an interim concern. While currently
| vapor, a future LLM might propose and implement effective DSLs
| when requested. Context windows are increasing and perhaps
| contemporary LLMs could code for niche languages better with
| appropriate prompting. However, the cultural, network effects
| described here are concerning.
| emiliobumachar wrote:
| Okay, I'm more then halfway through and still don't get what DSL
| is an acronym for. Any light?
| fragmede wrote:
| Domain Specific Language. It's when you invent a tiny
| programming language with nouns and verbs that are appropriate
| for some niche. Like maybe for eg wedding planning you wouldn't
| use json or yaml, but some custom format that lets users define
| people and who has to sit where and where they can't sit
| without being professional programmers.
| msgodel wrote:
| Good. DSLs suck. Either write a library or write a datastructure.
___________________________________________________________________
(page generated 2025-06-18 23:01 UTC)