[HN Gopher] The Neat Programming Language
___________________________________________________________________
The Neat Programming Language
Author : FeepingCreature
Score : 65 points
Date : 2023-12-25 17:38 UTC (5 hours ago)
(HTM) web link (neat-lang.github.io)
(TXT) w3m dump (neat-lang.github.io)
| FeepingCreature wrote:
| I too have made a language! Hope you find Neat interesting. It's
| heavily inspired by D, except with reference counting instead of
| garbage collector. Also, it has macros.
|
| My DConf talk on the language is now up too!
| https://www.youtube.com/watch?v=nDqlYnS-K2c
| jqpabc123 wrote:
| Looks fun ... and neat.
|
| But the mere fact that the primary ascribed goal is "fun"
| suggests it is not be taken seriously for actual work projects.
| FeepingCreature wrote:
| Oh no good god, Neat is definitely beta. This language has
| one user at the moment. Do _not_ use it for actual work
| projects.
|
| (Though I will be proper chuffed if you do. I just can not
| recommend it.)
|
| To be honest, my goal is to have maybe 10 to 50 users right
| now. That'd be quite enough for this christmas.
| dr_kiszonka wrote:
| RoR was optimized for developer happiness. Maybe a language
| optimized for fun could be used for serious work too?
|
| https://rubyonrails.org/doctrine
| 7thaccount wrote:
| There's a few languages that claim to be optimized for fun
| (among other things). One example is Raku. I think -o Fun,
| is written somewhere.
| brabel wrote:
| Haven't had the time to watch that talk yet... but what's your
| motivation to create a new language? Any drawbacks in D that
| you're trying to fix (besides the GC being replaced with RC -
| which many consider to also be GC by the way)?
| FeepingCreature wrote:
| Yep, here's the big ones:
|
| - proper macros
|
| - proper sumtypes with implicit conversion
|
| - more normal lambdas (seriously, D's lambdas are wild)
|
| - faster compiler with more caching and (maybe) eventually
| live reload?
|
| - proper packages instead of include path.
|
| And a bunch of small fry like format strings and named
| parameters.
|
| And yeah, um, GC can be good but the D GC kind of isn't. We
| use D at work, and we run into GC issues frequently. Neat's
| RC is a much thinner wrapper around C memory allocation,
| which is just better imo, much as GC and RC are logically
| equivalent in some ways.
| dataangel wrote:
| For the uninitiated what are wild about D lambdas? I'm
| familiar with C++ lambdas and Python lambdas.
| FeepingCreature wrote:
| Okay so if you're calling a D std.algorithm function, for
| instance int factor = 2;
| assert([2, 3].map!(a => a * factor).array == [4, 6]);
|
| Then the `!` indicates that you're actually passing the
| lambda as a compiletime parameter to `map`. But the
| lambda can access the surrounding context! How does it
| pass a runtime value at compile time?
|
| So what you're passing is actually purely a function
| symbol. The way that it gets the stack reference to the
| surrounding function is that it's actually a nested
| function. And the way that `map` gets the stack reference
| to pass to the lambda is that, effectively, that instance
| of `map` is _also_ a nested function of the calling
| function.
|
| That's also why you cannot pass a lambda as a template
| parameter to a class method in D: it already has a
| context parameter, ie. the class reference.
|
| In Neat, the _value_ of the lambda is the stackframe
| reference, and it 's just passed as a regular parameter:
| int factor = 2; assert([2, 3].map(a => a *
| factor).array == [4, 6]);
|
| Which avoids this whole issue at the cost of requiring
| some cleverness with refcounting.
| matheusmoreira wrote:
| > proper packages instead of include path
|
| Please elaborate on this. It's something I'm trying to get
| right in my own language since day one. I see in your
| language's manual that you have chosen to implement modules
| as files and packages as directories. I came up with
| something very similar but special cased the main module
| file so that the entire module could be contained in its
| directory. my-module/ my-module.ln
| (import (my-module)); reads my-module/my-module.ln
|
| How do you represent modules in your implementation? How do
| you handle module loading? What paths do you search? Do you
| support Linux distribution packaging? This last feature is
| something I'm interested in supporting, I added system
| directories to my search path for this reason but I wonder
| if there's anything else that I need to do.
| FeepingCreature wrote:
| Yes-ish. Basically, it's sort of like D in the sense that
| a `module a.b.c;` always corresponds to a file `a/b/c.nt`
| somewhere. But the places where it looks follow a
| recursive hierarchy, not an include path: you can define
| paths to search, but you have to explicitly define
| _import relations_ between those paths. And that 's
| because it's not really a search path, it's a full-on
| package graph, where each package is only allowed to see
| imports from packages that it explicitly lists as a
| dependency. In D, if you're importing a Dub package,
| because it's just using a search path, the files from
| that Dub package can actually import modules from your
| main project. In Neat, the lookup order is "current
| package", then "dependency packages", then stop.
|
| So Neat's actual hierarchy is "file [> folder]* >
| package", where `package` itself is optional in the
| import declaration: you can write `import
| package(gtk).gtk`, but you don't have to. This is
| occasionally useful during bootstrap builds when you want
| to clarify which version of the compiler an import is
| coming from: the current _running_ compiler is always
| `package(compiler)`.
|
| This is all because I've been writing it with something
| like a package manager in mind from essentially day one.
| JonChesterfield wrote:
| What do you mean by implicit conversion in the context of
| sum types? I don't see it under https://neat-
| lang.github.io/manual.html#types
| FeepingCreature wrote:
| Oh yeah I didn't actually write this down in the manual:
| if you expect `(A | B)`, then both A and B implicitly
| convert to that type.
| JonChesterfield wrote:
| Type A|B is implicitly constructible from A or from B
| sounds reasonable. A|A from A presumably an error, or
| possibly A|A itself is an error. Thanks
| logicprog wrote:
| Hey, I've been trying to not interact on social media much
| anymore but I just wanted to let you know that this looks like
| a pretty great (neat :)) little language! I'm very comfortable
| using Rust so I'm probably not going to end up using it, but my
| girlfriend is a data scientist who only knows Python and
| recently tried to start programming a game involving nbody
| simulation in Python, only to find that for the type of games
| she was trying to make Python was far too slow, and she's been
| really struggling to find a language that's relatively close to
| systems programming level (since she doesn't want to use an
| established engine) that also uses concepts and paradigms she
| understands and has reasonable syntax and stuff, and this looks
| like exactly what she's been looking for! When I sent her the
| link she got extremely excited, even more so when she saw that
| you had examples that were making games and using sdl. I hope
| you keep working on this language because I think it really
| does hit a nice (if niche) sweet spot!
|
| As a funny side note, I read the part on your website where you
| talk about why you chose reference counting over garbage
| collection, and despite your light-hearted resentment of the
| fact that garbage collection is such a turn off for people, the
| fact that your language doesn't use garbage collection _is_ one
| of the major factors in why she might end up using it lol. We
| decided not to do C# (my first suggestion actually) because
| since she doesn 't want to use an engine, it wouldn't be used
| just as a scripting language, but for the entire object model
| and update loop and so on, so GC would be a dealbreaker. Sorry-
|
| (tiny rant mode as a hobbyist gamedev myself) there is actually
| a good reason for this -- it is much easier to do
| multithreading when you don't have a separate thread going over
| all shared memory, and it's much easier to do a soft real time
| thing like game development if your memory allocation and
| deallocation stays relatively consistent and predictable, even
| if it is slower overall. Also, yes malloc _may_ cause variable
| slowdowns too, but definitely to a smaller degree /varience
| than the average garbage collector and more predictably since
| you can control when allocations happen; otherwise there
| wouldn't be a noticeable difference between using garbage
| collected and non garbage collected languages for writing
| large-scale games. Plus, in any case, that's the reason why
| game developers tend to try to limit dynamic allocation at
| runtime in the first place.
| FeepingCreature wrote:
| Did she find the built-in vector types yet? :) Small-scale
| gamedev is one place where I think this language could shine.
| Anyway, please also tell her to hit up the Discord (or IRC)
| for any questions!
| nsajko wrote:
| How is it implemented? Via transpilation to C?
| FeepingCreature wrote:
| Two backends:
|
| - LLVM is a proper SSA backend, it just uses the llvm-c API to
| generate bc modules, like any other LLVM-based compiler. It
| technically links with clang, but that's just cause I didn't
| want to set up my own link driver and optimization pipeline.
|
| - GCC is technically "transpiling" to C, but the generated C
| files are unreadable, because it's still SSA.
|
| The GCC backend is also used to build the releases. I just
| build the compiler with the GCC backend, then zip up all the .c
| files that it produced.
|
| That's why the release build script starts by building a bunch
| of C files.
| zem wrote:
| I perked up at "like D but with built in sum types". is C interop
| intended to be as good as D's?
| FeepingCreature wrote:
| It runs on plain C ABI, so you can just define C functions as
| `extern(C)`, just as you would in D. But you can also use
| `std.macro.cimport` to import C headers directly. Check out the
| Dragon demo, https://github.com/Neat-
| Lang/neat/blob/master/demos/dragon.n... :
| macro import std.macro.cimport; import
| c_header("raylib.h");
|
| And then you can just use (most of) the Raylib functions and
| types.
|
| cimport is a massive hack: it runs gcc on the header in
| preprocessor expansion mode, then parses the result. Its tactic
| for C syntax it doesn't understand is "just skip it and hope
| for the best." :) Works surprisingly well.
|
| On x86, there was a lot to be gained from having a language
| specific ABI, because the cdecl ABI was so slow. But the x86-64
| ABI, much as I may dislike its complexity, is genuinely plenty
| fast already.
| sillysaurusx wrote:
| feep's been working on this for like a decade. Happy to see it
| get some recognition.
| FeepingCreature wrote:
| Ah! Technically, this version of the compiler has no continuity
| with the first one. It's just named Neat too, because I spent
| like a day trying to think of another name after I got sniped
| on C*, and couldn't come up with anything, and went "hey, I
| already have a name."
|
| Neat as in the current iteration of the language, started in
| 2020.
| sillysaurusx wrote:
| Is it fair to discount all the false starts and early work
| prior to 2020? A bit like saying Picasso has been working on
| a painting for a month instead of his whole life.
|
| I guess it's more accurate to say that you've been an
| inveterate language designer for a very long time, and it's
| nice to see your latest one go public.
| FeepingCreature wrote:
| Yay!
| cornholio wrote:
| Could a Rust like borrow checker automatically and transparently
| insert the refcount logic only when it's needed, slowing down
| only access to variables with multiple owners, which would be a
| minority?
|
| And perhaps declare some special variables as "resource" to
| inform the compiler to really enforce the single write owner
| rule, for "special" external things like file descriptors etc.
|
| IMHO, memory as a tightly managed resource is adequate for system
| programming, but a hindrance in most other general programming
| tasks. But the approach has its point for actual resources any
| program needs to manage, allowing powerful compile time checks.
|
| So make it optional, with the default being automatic memory
| management.
| FeepingCreature wrote:
| I'm coming from a D perspective, where you really don't want to
| enforce anything single-owner-like, because it makes algorithms
| awkward. Neat does some commonsense optimizations to avoid
| pointless refcount changes, for instance it internally assumes
| that all function parameters are managed by the caller and
| doesn't take an additional reference for them. But at the end
| of the day, it mostly just refcounts and calls it good enough.
|
| You _can_ define types in Neat that are noncopyable, but it
| really limits what you can do with them. For instance, nested
| function references are noncopyable by default to avoid forcing
| closure allocations.
| beagle3 wrote:
| You've basically described Nim's memory management.
| dang wrote:
| Related:
|
| _Breakelse: When Compiler Developers Get Bored_ -
| https://news.ycombinator.com/item?id=37887426 - Oct 2023 (78
| comments)
| FeepingCreature wrote:
| I've been working myself up to this post for a while.
| IshKebab wrote:
| > I've been in the D community for a decade and a half. People
| keep telling us they don't like the GC. They keep saying GC is a
| dealbreaker. They want predictable memory usage and cleanup
| times. Somehow, none of this is ever an issue with C# and Java.
| Somehow, the fact that any memory allocator, including glibc's,
| can incur arbitrary delays never matters. Well, fine! Fine.
| Whatever. I disagree with the choice, but as an offshoot of an
| offshoot, I really can't afford alienating folks. I'm tired of
| arguing this point. Nobody has ever said that reference counting
| was a dealbreaker. So reference counting it is.
|
| I think this is a bit off. It _is_ an issue with Java and C#, but
| they don 't position themselves as a C++ replacement in the same
| way D does. Even so you still get lots of complaints about
| missing RAII and GC pauses (witness how well received Go's low
| latency pauses have been received).
|
| And yes glibc could introduce arbitrary delays, but it generally
| _doesn 't_. Allocation is way faster than GC.
|
| Maybe he was talking about long pauses when you destroy a big C++
| object (e.g. a big `std::map`)? That can definitely cause
| annoying big pauses due to deallocation. But he already
| identified the critical factor - it's predictable. You can fix it
| deterministically.
|
| Anyway reference counting is a decent choice. It can be very fast
| (especially if you are only referencing counting big objects and
| not little integers etc.)
| FeepingCreature wrote:
| I've come around more to "yeah GC is a problem actually" since
| I wrote that snippet. It really depends on your usecase though.
| IshKebab wrote:
| Yeah I agree. There are plenty of use cases where it _isn 't_
| a problem, but in those cases you would probably not pick
| D/C++/Rust/Zig anyway. Or you don't have to at least (I still
| use Rust in cases where a GC is fine because it's such a
| great language).
| dataangel wrote:
| > glibc could introduce arbitrary delays, but it generally
| doesn't.
|
| Yeah it does, that's why gamedevs and hfts don't use any malloc
| in the fast path, glibc or otherwise.
| IshKebab wrote:
| Incorrect. They don't use malloc in the fast path because
| allocation is slow, not because it introduces arbitrary
| delays like GC pauses do. To be clear:
|
| * Not allocating (or stack/bump allocation): extremely fast,
| completely deterministic.
|
| * malloc: pretty fast, in theory arbitrarily slow but in
| practice it's well bounded
|
| * GC pauses: very slow (until Go anyway)
| dadoum wrote:
| If you can avoid allocations where speed matters, then the
| GC won't slow you down either, as (at least in D) it cannot
| be triggered if you don't allocate.
| IshKebab wrote:
| The difference is that it's _deterministic_. If you avoid
| allocation in C /C++/Zig/Rust then you _know_ nothing is
| going to slow you down, and at worst you have a couple of
| allocations that won 't cause a big spike.
|
| With GC you can try to avoid allocations, but you might
| miss a couple and get occasional frame stutters anyway
| every now and then. Also avoiding allocation is a whole
| lot harder in languages that use a GC for everything than
| languages that provide an alternative.
|
| The real solution for games is explicit control over GC
| runs - tell it not to GC under any circumstances until
| the frame is finished rendering, then it can GC until the
| next frame starts. I assume Unity does this for example.
| Still, games are only one application where you don't
| want big pauses - one that happens to have convenient
| regular times when you probably aren't doing anything.
| neonsunset wrote:
| Go's GC throttles on allocation if it can't keep up and causes
| bad tail latencies just as much as GC in .NET or Java or even
| worse if you have high memory traffic (because Go's GC has far
| lower throughput).
| dianeb wrote:
| Once upon a time, NCR's mainframe language was called Neat (the
| version I briefly used was called Neat/3 which was a low-to-mid-
| level language if memory serves. I was writing compilers.) My
| memories of a language with that name are unpleasant.
|
| This is probably the least useful of all comments, but please
| think of another name. My quibble with the language has to do
| with the use of '[' .. ']' pairs. I'm not confident that
| refactoring will be straight-forward. I could be wrong.
| FeepingCreature wrote:
| I tried, I failed, now it's too late. Just gonna hope not many
| people remember that other language.
|
| Not sure what you mean with `[]`?
| dianeb wrote:
| This construct: string longestLine = [
| argmax(line.strip.length) line for line in text.split("\n")];
| print(longestLine);
|
| looks problematic to me -- are the brackets indicating scope?
| an array? something else?
|
| As far as the naming is concerned, you'll probably have to
| put up with remarks such as "neat code is messy" because
| that's the way people are with something new. Don't let that
| dishearten you!
| AeroNotix wrote:
| It looks like what python calls a list comprehension. An
| expression which builds up each element of a sequence,
| potentially from another sequence.
| FeepingCreature wrote:
| Yeah that's a macro (`std.macro.listcomprehension`), you
| can copy that file and change it to be whatever syntax you
| like. It's indeed inspired by Python's syntax.
| fyzix wrote:
| Is it ready for benchmarking? D currently sits at the top of
| https://github.com/jinyus/related_post_gen and it would be
| interesting to see how neat stacks up.
| FeepingCreature wrote:
| Ah, I remember I looked at that.
| https://gist.github.com/FeepingCreature/cd935a209fa2eebd5928...
| Here's a Neat port I had lying around, I think it's pretty much
| a copypaste from D? I think it was a bit slower, can't remember
| why though.
___________________________________________________________________
(page generated 2023-12-25 23:00 UTC)