[HN Gopher] An AI-first program synthesis framework built around...
___________________________________________________________________
An AI-first program synthesis framework built around a new
programming language
Author : tosh
Score : 100 points
Date : 2025-08-09 15:36 UTC (1 days ago)
(HTM) web link (queue.acm.org)
(TXT) w3m dump (queue.acm.org)
| sys13 wrote:
| Glad to see focus being put on keeping humans in the drivers
| seat, democratizing coding with the help of AI. The syntax is
| probably still too verbose to be easily accessible, but I like
| the overall approach.
| AIPedant wrote:
| I know ACM Queue is a non-peer-reviewed magazine for
| practitioners but this still feels like too much of an
| advertisement, without any attempt whatsoever to discuss
| downsides or limitations. This really doesn't inspire confidence:
| While this may seem like a whimsical example, it is not
| intrinsically easier or harder for an AI model compared to
| solving a real-world problem from a human perspective. The model
| processes both simple and complex problems using the same
| underlying mechanism. To lessen the cognitive load for the human
| reader, however, we will stick to simple targeted examples in
| this article.
|
| For LLMs this is blatantly false - in fact asking about "used
| textbooks" instead of "apples" is measurably more likely to
| result in an error! Maybe the (deterministic, Prolog-style)
| Universalis language mitigates this. But since Automind (an LLM,
| I think) is responsible for pre/post validation, naively I would
| expect it to sometimes output incorrect Universalis code and
| incorrectly claim an assertion holds when it does not.
|
| Maybe I am making a mountain out of a molehill but this bit about
| "lessen the cognitive load of the human reader" is kind of
| obnoxious. Show me how this handles a slightly nontrivial
| problem, don't assume I'm too stupid to understand it by trying
| to impress me with the happy path.
| tannhaeuser wrote:
| Prolog works indeed very well as target for generation by an
| LLM, for input problems limited and similar enough in nature to
| given classes of templated in-context examples, so well indeed
| that the lack of a succinct, exhaustive text description of
| your problem is becoming the issue. At which point you can
| specify your problem in Prolog directly considering Prolog was
| also invented to model natural language parsing and not just
| for solving constraint/logic problems, or could employ ILP
| techniques to learn or optimize Prolog solvers from existing
| problem solutions rather than text descriptions. See [1].
|
| [1]: https://quantumprolog.sgml.net
| YeGoblynQueenne wrote:
| >> Since we're not doing original research, but rather intend
| to demonstrate a port of the Aleph ILP package to ISO Prolog
| running on Quantum Prolog, we cite the problem's definition
| in full from the original paper (ilp09):
|
| Aleph? In 2025. That's just lazy, now. At the very least they
| should try Metagol or Popper, both with dozens of recent
| publications (and I'm not even promoting my own work).
| tannhaeuser wrote:
| You're not wrong, and alternatives were considered, but
| those were really not fit to be ported to ISO Prolog in
| bounded time with complete lack of tests or even a basic
| reproducible demo, uncontrolled usage of non-ISO libs and
| features only available on the originally targetted Prolog
| implementation, and other issues typical of "academic
| codes."
| YeGoblynQueenne wrote:
| The lack of unit tests is something I'm guilty of too and
| you're very right about it. The community is dimly aware
| of the fact that systems are more on the "academic
| prototype" side of things and less on the "enterprise
| software" but there's so little interest from industry
| that nobody is going to spend significant effort to
| change that. Kind of a catch 22 maybe.
|
| How about ISO? Why was this a requirement, out of
| curiosity?
| tannhaeuser wrote:
| Quantum Prolog is pure ISO and was the primary target for
| porting here. The idea is really just putting effort
| behind libs that can be shared by other non-legacy
| Prologs and thus get more people involved.
| YeGoblynQueenne wrote:
| Look, I'll be blunt. If ISO is keeping you from running
| any ILP system newer than Aleph and you have a good
| reason to want to use an ILP system then you should
| reconsider the benefits of ISO. There's no ISO for LLMs
| for example, yet you'll have no trouble at all convincing
| people to get involved, there.
|
| That said, I think Vanilla should be easy to ISO-fy. It's
| just a meta-interpreter really. It uses tabling and I'm
| guessing that's not ISO (it's more like the Wild West)
| but it is not indispensible.
|
| https://github.com/stassa/vanilla/blob/master/src/vanilla
| .pl
|
| I could have a look, but I have no idea what's in the ISO
| standard (and that's another bugbear- the ISO docs are
| not free. What?). I guess I could try to run it on your
| engine and see what the errors say though.
|
| P.S. Re: "academic codes" I got documentation, logging,
| and custom errors and that's more than you can say for
| 100% of academic prototypes plus ~80% of industry
| software also. I do have unit tests, even, just not many.
| AIPedant wrote:
| Right but my point was that "given classes of templated in-
| context examples" is either
|
| a) a game of roulette where you hope the LLM provider has
| RLHFed something very close to your use case, or
|
| b) trying to few-shot it with in-context examples requires
| more engineering (and is still less reliable) than simply
| doing it yourself
|
| In particular it's not just "the lack of a succinct,
| exhaustive text description," it also a lack of
| English->Prolog "translations."
|
| It seems like the LLM-Prolog community is well aware of all
| this (https://swi-prolog.discourse.group/t/llm-and-prolog-a-
| marria...) but I don't see anything in Universalis that
| solves the problem. Instead it's just magically invoking the
| LLM.
| tannhaeuser wrote:
| That link you're citing is old news and also
| contained/discussed in the Quantum Prolog article. Those
| observations were made with respect to translating problem
| descriptions into PDDL, a frame-based, LISPish
| specification language for AI competitions encoding
| planning "domains" in a tight a priori taxonomical
| framework rather than logic or any other Turing-complete
| language. As such, contrary to what's speculated in that
| link, the expectation is that results do _not_ carry over
| to the case of Prolog, which is much more expressive. I
| actually considered customizing an LLM using an English
| /Prolog corpus, which should be relatively straightforward
| given Prolog's NLP roots, but the in-context techniques
| turned out so impressive already (using 2025 SoTA open
| weight models) that indeed the bottleneck was the lack of
| text descriptions for really challenging real-world
| problems, as mentioned in the article. The reason may lie
| in the fact that English-to-Prolog mapping examples and/or
| English Prolog code documentation examples are sufficiently
| common in the latent space/in foundation training data.
|
| I can ensure you Prolog prompting for at least the class of
| robotic planning problems (and similar discrete problems,
| plus potentially more advanced classes such as scheduling
| and financial/investment allocation planning requiring
| objective function optimization) works well, and you can
| easily check it out yourself with the prompting guide or
| even online if you have a capable endpoint you're willing
| to enter [1].
|
| [1]: https://quantumprolog.sgml.net/llm-demo/part2.html
| duskwuff wrote:
| This article feels extremely imprecise. The syntax of the
| "language" changes from example to example, control structures
| like conditionals are expressed in English prose, some examples
| are solved by "do all the work for me" functions like the
| "toPdf()" example...
|
| This whole thing feels like an elaborate LLM fantasy. Is there
| any real, usable language behind these examples, or is the author
| just role-playing with ChatGPT?
| YeGoblynQueenne wrote:
| More to the point, is ChatGPT role-playing the author?
| mannycalavera42 wrote:
| lol
| ethanwillis wrote:
| > Universalis ensures that even those with minimal experience in
| programming can perform advanced data manipulations.
|
| Is it a good thing to make this easier? We're drowning in garbage
| already.
| rvasa wrote:
| Great to start off ... then we will end-up reinventing/re-
| specifying functions for reusability, module/packages for higher-
| level grouping, types/classes, state machines, control-flows
| [with the nuances for edge cases and exit conditions], then we
| will need error control, exceptions; sooner or later concurrency,
| parallelism, data structures, recursion [lets throw in monads for
| the Haskellians amoung us]; who knows .. we may even end up with
| GOTOs peppered all over the English sentences [with global
| labels] & wake up to the scoping, parameter passing. We can have
| a whole lot of new fights if we need object-oriented programming;
| figure out new design patterns with special "Token Factory
| Factories".
|
| We took a few decades to figure out how to specify & evolve
| current code to solve a certain class of problems [nothing is
| perfect .. but it seems to work at scale with trade-offs]. Shall
| watch this from a distance with pop-corn.
| pjmlp wrote:
| So after Haskell, and helping with LINQ, this is what Erik Meijer
| has been focusing on.
| gavinray wrote:
| > "The actual implementation of Universalis uses Kotlin
| DataFrames"
|
| Neat. My favorite talk at the 2025 Kotlin Conf was the demo of
| the new compiler plugin that allows ad-hoc type inference from
| DataFrame expressions.
|
| Essentially, say that you have an input type like:
| type User { name: String, age: Int }
|
| And you do something like ".map(it => { doubledAge: it.age * 2
| })"
|
| The inferred type of that intermediate operation is now:
| type User_synthetic { doubledAge: Int }
|
| Which is wild, since you essentially have TypeScript-like
| inference in a JVM language.
|
| Timestamp to talk:
|
| https://www.youtube.com/watch?v=F5NaqGF9oT4&t=543s
___________________________________________________________________
(page generated 2025-08-10 23:02 UTC)