[HN Gopher] Rewriting Rust: A Response
___________________________________________________________________
Rewriting Rust: A Response
Author : gavinhoward
Score : 105 points
Date : 2024-09-27 14:36 UTC (8 hours ago)
(HTM) web link (gavinhoward.com)
(TXT) w3m dump (gavinhoward.com)
| az09mugen wrote:
| Wow, about Yao:
|
| "In fact, it might be better said that Yao is meant to be:
|
| As powerful as C, As flexible as Lisp, As easy as Python, As
| provable as Ada/SPARK, More reliable than Rust, With the time
| semantics of HAL/S."
|
| Even if I don't really believe in this incredible statement, I'm
| really curious to see what can be the result.
| gavinhoward wrote:
| Author here.
|
| Yeah, you shouldn't believe it yet, but here is the status of
| each:
|
| * "As powerful as C": unproven. I might need something like
| Rust's `unsafe`, and even though I have something in mind for
| that, it requires that _any_ program be expressible in Dynamic
| Restricted Structured Concurrency, an open problem. I am
| working on a proof, though.
|
| * "As flexible as Lisp": I think this is proven. Yao has
| equivalents for both macros (keywords) and reader macros
| (lexing modes). My build system uses its own keywords to
| implement a build DSL [1]. The shell sublanguage uses a lexing
| mode, and another makes it possible to embed JSON.
|
| * "As easy as Python": in retrospect, this isn't quite
| possible, but I hope to get 90% of the way.
|
| * "As provable as Ada/SPARK": I'll let you read the design in
| [2] and decide for yourself. But Yao will also have contracts.
|
| * "More reliable than Rust": unproven, but I think the lack of
| async will go a long way.
|
| * "With the time semantics of HAL/S": unproven. HAL/S was what
| the Shuttle software was written in [3]. It was hard real-time.
| Because Yao will be distributed in IR form [4], the back end
| could leverage knowledge of worst-case instruction latencies to
| calculate worst-case response times. This same thing will also
| allow using only constant-time instructions for cryptography,
| using a `constant_time` function trait.
|
| [1]: https://rigbuild.dev/build.rig5/#keywords
|
| [2]: https://gavinhoward.com/2024/05/what-rust-got-wrong-on-
| forma...
|
| [3]: https://www.fastcompany.com/28121/they-write-right-stuff
|
| [4]: https://gavinhoward.com/2024/09/rewriting-rust-a-
| response/#d...
| az09mugen wrote:
| Thank you very much for your detailed answer. Will look into
| your links right now.
| az09mugen wrote:
| Hats off, you really did in fact a deep analysis of Rust
| conceptual errors by searching alternative ways to fix Rust
| async. Also by understanding and describing the decisions you
| made and what are their architectural implications (I liked
| the good, the bad and the ugly part).
|
| While I can't appreciate the entirety of this language
| because I can't grasp all the technical subtilities, I really
| like the core ideas at the basis of Yao and the spirit you
| put into it. I hope I can play with this language some day.
| And wish you good luck with this nice project.
| gavinhoward wrote:
| This is one of the nicest, yet honest, compliments I have
| ever received. Thank you!
| AlotOfReading wrote:
| Worst case execution times are generally unknowable, even at
| codegen. It depends on things like the specific
| microarchitectural details, what other instructions are
| currently executing, the memory hierarchy, what instructions
| have been executed before, how the power supply is
| responding, what the thermal environment is like, etc.
|
| HAL/S dealt with these problems by a combination of
| programmer/validation discipline, custom hardware designed
| for hard real-time usecases, and generous support from the
| underlying RTOS (FCOS). Cryptography code _doesn 't_ deal
| with it, it just avoids data dependent nondeterminism, which
| is good enough for those use cases.
| gavinhoward wrote:
| Yes, you are correct.
|
| My idea is to cause a compile error if the backend can't
| use all instructions for which it knows the worst case
| latency or if it can't use purely constant-time
| instructions.
| AlotOfReading wrote:
| That's essentially every instruction though. You can't
| write a program that doesn't access memory for example,
| and every memory access is nondeterministic in wall clock
| time.
| gavinhoward wrote:
| Yes.
|
| In the crypto example, the backend would have to be
| sophisticated enough to know when the same addresses
| would be accessed every time, like bitslicing in AES. [1]
|
| The devil _absolutely is_ in the details. This is why I
| say Yao 's capabilities on this front are unproven.
|
| [1]: https://www.bearssl.org/constanttime.html#constant-
| time-in-b...
| xedrac wrote:
| > As flexible as Lisp": I think this is proven.
|
| Given the development model of Lisp is modifying a live image
| of a running program, I have to assume you limited this
| statement to the availability of macros.
| gavinhoward wrote:
| Yes, sorry. I should clarify that.
| lispm wrote:
| > Given the development model of Lisp is modifying a live
| image of a running program
|
| I would say it is modifying the running program itself. An
| image is the saved version of the heap. Saving/loading
| images is optional.
|
| But with Lisp one can also compile source files to compiled
| files. In rare occasions it is also possible to compile
| whole programs.
| xavxav wrote:
| > * "As provable as Ada/SPARK": I'll let you read the design
| in [2] and decide for yourself. But Yao will also have
| contracts.
|
| Without being too self-indulgent, I'm not sure there is that
| big of a gap between the two in provability, there are now a
| huge array of verifiers for Rust code which are being used to
| verify real code: SAT/SMT solvers, kernels, memory allocators
| etc...
| gavinhoward wrote:
| I agree. I think getting provability right would bring
| reliability along with it.
| BoingBoomTschak wrote:
| > "As flexible as Lisp": I think this is proven. Yao has
| equivalents for both macros (keywords) and reader macros
| (lexing modes).
|
| First-class symbols (incl. uninterned symbols)? Classes and
| functions can be redefined at runtime? User-available parsing
| (read) and runtime compilation (compile)? Restarts? CLOS and
| MOP? Communication with the compiler (declarations and ClTl2
| *-information)?
|
| I don't mean to sound too abrasive, but reducing Lisp
| flexibility to macros is a bit much. In any case, good luck
| with your project!
| gavinhoward wrote:
| > First-class symbols (incl. uninterned symbols)?
|
| Yes.
|
| > Classes and functions can be redefined at runtime?
|
| No.
|
| > User-available parsing (read) and runtime compilation
| (compile)?
|
| Yes, though runtime compilation is not implemented yet.
|
| > Restarts?
|
| Yes, though they are only implemented in the runtime; the
| keywords don't exist yet.
|
| > CLOS and MOP?
|
| No, the object system will be more recognizable to non-
| Lispers.
|
| > Communication with the compiler (declarations and ClTl2
| *-information)?
|
| I wasn't able to find info on these. I will keep looking,
| and I would appreciate pointers, but the answer is probably
| no.
|
| Edit: after skimming [1], the answer on declarations is
| probably yes, for the most part. Still searching for
| *-information stuff.
|
| Edit 2: After skimming [2], Yao originally didn't have any
| concept of stuff like the *-information functions, but
| those are good ideas, and I will add them.
|
| [1]:
| https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node105.html
|
| [2]:
| https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html
| BoingBoomTschak wrote:
| Huh, thanks for replying seriously to my snark, that's
| more than what I expected!
|
| About CLOS, I suggest at least making method combination
| more flexible than C++'s override. A way to run both the
| parent and child methods is often needed (maybe in a
| specific order, maybe with a way to collect all the
| return values).
|
| Declarations are basically C/C++'s pragmas and
| __attribute__, but standardized and using a seamless
| syntax. The ClTl2 additions I'm talking about are ways
| for the compiler to give lexical information to macros:
| see https://github.com/Zulu-Inuoe/trivial-cltl2 and
| https://github.com/alex-gutev/cl-environments
| gavinhoward wrote:
| > Huh, thanks for replying seriously to my snark, that's
| more than what I expected!
|
| Quite frankly, the snark was warranted. I'm not a Lisper,
| but I do know more, and I should have been more precise.
|
| > About CLOS, I suggest at least making method
| combination more flexible than C++'s override. At least a
| way to run both the parent and child methods is often
| needed (maybe in a specific order, maybe with a way to
| collect all the return values).
|
| I agree with this. However, Yao will not have
| inheritance; instead, it will be more like Go:
| composition over inheritance. And interfaces. So I could
| be wrong, but without inheritance, I'm not sure the
| entire complexity of the CLOS is needed. Do correct me,
| though.
|
| > Declarations are basically C/C++'s pragmas and
| __attribute__, but standardized and using a seamless
| syntax.
|
| That's such a nicer description than in CLtL2. And yes,
| Yao will have them.
|
| > The ClTl2 additions I'm talking about are ways for the
| compiler to give lexical information to macros: see
| https://github.com/Zulu-Inuoe/trivial-cltl2 and
| https://github.com/alex-gutev/cl-environments
|
| If those are just for giving info to macros, Yao already
| has that; the lexer will tell keywords if the given token
| is a package, a keyword, an operator, a function, a type,
| etc., along with other info.
| BoingBoomTschak wrote:
| > However, Yao will not have inheritance; instead, it
| will be more like Go: composition over inheritance. And
| interfaces. So I could be wrong, but without inheritance,
| I'm not sure the entire complexity of the CLOS is needed.
| Do correct me, though.
|
| I'm sadly not very well versed with this alternative to
| OOP. But something tells me that a more "basic" and less
| dynamic scheme like that is probably best for performance
| reasons, since you'd either need a Julia-like JIT or a
| granular "dynamism knob" like
| https://github.com/marcoheisig/fast-generic-functions to
| truly amortize the cost of CLOS-like dispatch and
| polymorphism; though magic can be done, as the "Fast
| generic dispatch for Common Lisp" paper shows.
|
| > if those are just for giving info to macros, Yao
| already has that
|
| Huh! If this includes types deduced by compiler inference
| and not just user declarations (since Yao seems gradually
| typed, to my eyes), that'd be pretty impressive.
|
| Hope you'll be able to keep your spirits high, this is a
| lifetime's work you seem to be looking forward to ;)
| gavinhoward wrote:
| > I'm sadly not very well versed with this alternative to
| OOP. But something tells me that a more "basic" and less
| dynamic scheme like that is probably best for performance
| reasons, since you'd either need a Julia-like JIT or a
| granular "dynamism knob" like
| https://github.com/marcoheisig/fast-generic-functions to
| truly amortize the cost of CLOS-like dispatch and
| polymorphism; though magic can be done, as the "Fast
| generic dispatch for Common Lisp" paper shows.
|
| You know, for someone who isn't "well versed" with it,
| you certainly nailed exactly why I chose it. Also,
| simplicity. The "as easy as Python" is only possible if I
| remove complecting [1] things like inheritance.
|
| > Huh! If this includes types deduced by compiler
| inference and not just user declarations (since Yao seems
| gradually typed, to my eyes), that'd be pretty
| impressive.
|
| It does, actually. But Yao is not gradually typed,
| unfortunately. Too complecting.
|
| [1]: https://www.youtube.com/watch?v=LKtk3HCgTa8
| ubj wrote:
| Interesting response. I'm as curious as anyone else as to how Yao
| will improve upon Rust, but here is what I've observed from
| following several (relatively) recently created languages (Julia,
| Nim, Mojo, JAX):
|
| * Language creators start with high hopes and big promises.
| "We've learned from the past! This will solve all the problems of
| previous X,Y,Z languages! It will be so much easier to use!"
|
| * The language begins to be designed, implemented, and developed.
| Problems arise. Tradeoffs must be made. Invariably, the new
| language develops its own set of limitations, shortcomings, and
| weaknesses. These are inevitable with any new programming
| language.
|
| * Ultimately the language either finds a userbase who are willing
| to overlook or work around the limitations and pitfalls, or it
| fades away.
|
| * If it remains, eventually another new language arises saying
| "We've learned from the past! This will solve all the problems of
| previous X,Y,Z languages! It will be so much easier to use!" And
| the cycle repeats.
|
| Now, is this a bad thing? I would argue no, not necessarily.
| Every time a new language is created, our vision of what is
| possible is expanded. We learn new lessons about what works and
| what doesn't work in programming languages.
|
| So in short I'm looking forward to see how the state of the art
| is advanced by Yao. If it gets to a decent working state, I might
| even try it out if I have time. However, in light of the above
| statements forgive me for being skeptical of the following claim
| from Yao's website [1]:
|
| > As powerful as C, As flexible as Lisp, As easy as Python, As
| provable as Ada/SPARK, More reliable than Rust, With the time
| semantics of HAL/S.
|
| [1]:
| https://git.yzena.com/Yzena/Yc/src/branch/master/docs/yao/ma...
| gavinhoward wrote:
| Author here.
|
| You are absolutely right to be skeptical. I wrote that a long
| time ago.
|
| I did detail the current status of those claims in a comment:
| https://news.ycombinator.com/item?id=41672316 .
| jadbox wrote:
| Fwiw, Zig is one of the few languages on my radar that's on the
| uptick. At least in my circle, I know people from firmware to
| VM development that moved from Rust over to Zig for its
| flexibility. There' also Bun that managed to nearly be a drop-
| in replacement for Node in a very short development cycle,
| which they claim is due to Zig. (I am not associated with Zig
| nor have a lot of experience with it yet)
| slekker wrote:
| Also Zig wants to have better memory safety before v1 -
| there's an issue tracking that in the zig org but I couldn't
| find it
| ubj wrote:
| I've heard great things about Zig as well. The one difference
| between it and Rust is the amount of industry and financial
| investment to date. Rust has had literally millions of
| dollars of buy-in from some of the biggest tech corporations
| on Earth, which is why I'm more confident that it will likely
| stick around.
|
| This is one area where, for example, Julia has struggled. In
| many ways it's a fantastic language for scientific research,
| but it simply has never received nearly as much funding or
| investment as, e.g., Python.
| throwawaymaths wrote:
| At 8 years (1999) python did not have any big investment or
| even big tech backers. IIRC the first real python success
| story was dropbox
|
| At 8 years there's a major JavaScript bundler and Uber
| using zig.
| aidenn0 wrote:
| I also appreciate the way that Zig focuses on tooling,
| particularly around cross-compilation. That's something that
| is usually a "nice to have" rather than "must have"
| particularly in young projects, and for users that need it,
| it's a huge win.
| 0cf8612b2e1e wrote:
| On the gripping hand, Rust is the first language to popularize
| (not invent, hold your tomatoes) many terrific programming
| ideas under one wrapper. It has demonstrated how much the
| compiler can do to prove code is correct. Yet...it has some
| warts. I am quite interested in what lessons can be learned to
| smooth out some of the Rust edges for a language which does not
| have "replace C++" as the guiding principle.
| steveklabnik wrote:
| I am always glad to see people pursuing creating their own
| programming language. More people should give it a try, IMHO.
|
| A tricky thing that comes up with Rust comparisons is that often,
| Rust has a feature that's weird or hard to use, but it's because
| that's the only solution that makes sense within the constraints
| Rust has placed upon itself. Clicking some links gets me to
| https://git.yzena.com/Yzena/Yc/src/branch/master/docs/yao/de...
|
| > Yao's biggest goals are: Correctness, Convenience, Performance,
| in that order.
|
| Having clear goals is a great thing when making a language,
| because they give you guidance on what is appropriate to include
| and what is not.
|
| Rust has certainly demonstrated having similar goals, but in a
| slightly different order: Correctness, Performance, and then
| Convenience. So it wouldn't shock me if Yao could do some things
| better than Rust, in accordance with its goals. But that also
| means that sometimes, Rust would be useful where Yao cannot be.
| Everything is tradeoffs.
|
| Incidentally, I actually think that figuring out what _your_
| values are, and what your needs are, is a great way to pick a
| programming language. Decide what matters to you as an engineer,
| and then find a language that shares similar values. I gave a
| conference talk a few years back on this idea, and how I viewed
| Rust 's values at the time
| https://www.infoq.com/presentations/rust-tradeoffs/
|
| This was based off of bcantrill's Platform as a Reflection of
| Values, which was very influential on me.
| https://www.youtube.com/watch?v=Xhx970_JKX4
|
| If you've ever heard about Oxide's focus on values, this is some
| of the older background on that.
| gavinhoward wrote:
| I wish I could upvote you twice.
|
| I decided on my values because of bcantrill's talk. Everyone
| should watch it.
|
| A more complete list of my values is at
| https://git.yzena.com/Yzena/Yc/src/branch/master/docs/adl/00...
| .
| andai wrote:
| Could you elaborate on value #0 being users (and what it
| means to prioritize users over programmers)?
| gavinhoward wrote:
| Good question. I will add that.
|
| It means that users' convenience comes before my
| convenience. (I am the programmer in question.)
|
| This means that if there is something I could do to make
| things easier on users, I do it, even if it is a lot more
| work for me.
|
| An example is https://git.yzena.com/Yzena/Yc/commit/c9086af
| 225d42b2f125959... . Comments in Yao do _not_ use the `#`
| character, but I added the shebang to the parser specially
| so that scripts could be run just like any others scripts.
| timeon wrote:
| This is nice to see. I often came across opposite
| sentiment here. "Dev time is precious, user is OK with
| un-performant/etc. software."
| gavinhoward wrote:
| Thank you for your kind words!
| whateveracct wrote:
| > A tricky thing that comes up with Rust comparisons is that
| often, Rust has a feature that's weird or hard to use, but it's
| because that's the only solution that makes sense within the
| constraints Rust has placed upon itself.
|
| Hah I find the same thing about Haskell. People are a bit less
| charitable about Haskell's quirks though I find..
|
| The place I work definitely has a culture of whining :P
| _0ffh wrote:
| > I am always glad to see people pursuing creating their own
| programming language. More people should give it a try, IMHO.
|
| I must agree that it's certainly lots of fun and a learning
| experience, but I must also always think of this amusing quote
| I once read somewhere: "What is a gentleman? A computer
| scientist who doesn't invent his own programming language."
| steveklabnik wrote:
| I've never heard that, and that's very funny.
| ivanjermakov wrote:
| This is what I want more people participating in "language
| wars" to understand. There is not much sense to compare
| projects that pursue completely different goals.
|
| And having a background in programming language design surely
| makes understanding of solutions and compromises a lot easier.
| steveklabnik wrote:
| I have said this recently a few times: I have found that most
| of the time, various people involved in working on languages
| themselves have far kinder attitudes towards other languages
| than their respective communities tend to. I've always
| chalked it up to some sort of collegial professionalism, but
| maybe understanding the tradeoffs more deeply is an aspect of
| it too.
| sgt wrote:
| Glad I waited before starting with Rust. Now I can just jump
| straight to Yao. Rust is so early 2020s.
| gavinhoward wrote:
| Please don't jump to Yao! The only thing it's good for right
| now is to replace shell scripts.
| bregma wrote:
| You're still using Yao? Pfff.
| chipdart wrote:
| > You're still using Yao? Pfff.
|
| One-up flex master right here. Unbeatable.
| sgt wrote:
| Hip crowd's already moved on to Tcl, I know (with the
| advancements in v9.0 and all).
| Ygg2 wrote:
| Tcl is yesterday's news. Hip crowd moved to artisanal
| ASM.
| falcor84 wrote:
| Excuse me, but real programmers use butterflies
|
| https://xkcd.com/378/
| epage wrote:
| > Pfff
|
| Now I want a language named that.
| wizzwizz4 wrote:
| > Instead of marking asynchronous functions, languages should
| have had us mark synchronous functions [...] Although that
| wouldn't remove the possibility of bugs when calling a blocking
| function in async code,
|
| So mark "synchronous" (i.e., non-yielding) and "asynchronous"
| (i.e., non-blocking). Then we can have the synchronous colour,
| the asynchronous colour, functions that are both (e.g. pure CPU-
| bound tasks, like arithmetic), and functions that are neither
| (e.g. buggy code).
| Animats wrote:
| Mostly an ad for Yao.
|
| - Is there really a major use case for function traits? Is more
| elaboration of the type system actually useful?
|
| - The original "Rewriting Rust" article discussed replacing
| Rust's macro system. That's not mentioned here. If you had to
| write your macros in Rust, to be run in compile time, would that
| be a win? I doubt it. Each macro would start with a call to a
| parser, and then you generate code as strings. This would pay off
| only for complex cases such as a regular expression compiler.
| Rust already has compile-time build programs, where you can run
| Rust during the build process. (I use that to take in an API
| definition from another source and generate Rust.)
|
| - "Runtime capabilities" seems to be an attempt to build a
| container system into the language. It's not clear that helps
| much. That's really the OS's job. Protecting programs against
| themselves is nice, but not usually the problem. I'd like to see
| something in Android that gives an app file access with less
| access than "all files". Like "Can read what was installed for
| this app, can write working subdirectory for this app, and
| nothing else."
| gavinhoward wrote:
| Fair criticisms, John. I just want to address macros.
|
| Yao has something like Rust's macros. They are called
| "keywords," because even the built-in keywords are implemented
| that way. (That is why Yao can just not have a `while`
| keyword.)
|
| My Yao-based build system, Rig, uses keywords to implement a
| build DSL. [1]
|
| And keywords are fast to compile.
|
| [1]: https://rigbuild.dev/build.rig5/#keywords
| nicoburns wrote:
| > Is there really a major use case for function traits?
|
| Absolutely. They can be used for function overloading, and for
| manually creating closures from structs, which can be more
| ergonomic sometimes. Whether you want those features is another
| matter. The rust compiler supports them but doesn't expose them
| to stable rust.
| ruthmarx wrote:
| > 'd like to see something in Android that gives an app file
| access with less access than "all files".
|
| SELinux has been able to solve that problem in Android for a
| long, long time now. Which is more appropriate, as you say it
| should be the job of the OS.
|
| The problem is the age old one where users want convenience at
| the cost of security.
| osigurdson wrote:
| Without sounding too much like a Cathy Woods, I'm wondering if AI
| can be leveraged to help with all of the non-core aspects of
| creating new languages? For example, really why does every
| language really need a json parser, or http requests library. If
| one were to examine the machine code instructions involved in
| each, they would likely be rather similar after all. Writing all
| of that stuff is an enormous lift for the community. Tooling is
| similarly "undifferentiated heavy lifting" in a way. Also, moving
| existing code bases to new languages is still hard.
|
| If there were better ways to achieve these things, there would be
| much less inertia in the language space.
| tazjin wrote:
| Odd post, and it landed me on the Yzena website which is very
| surreal. It seems to be intentionally hard from the website to
| figure out what Yzena _is_ or why anyone would want to buy a
| license for it (or the things it makes?). There was a link to a
| repo full of perl scripts, though.
|
| According to the website I'm not allowed to ask about this though
| ;)
| gavinhoward wrote:
| I only have one Perl script in that repo...
|
| Anyway, Yzena is my business. I am not a very good marketer,
| and the website is bad.
|
| I also don't think I will get any business, but eh.
| dang wrote:
| Recent and related:
|
| _Rewriting Rust_ - https://news.ycombinator.com/item?id=41654871
| - Sept 2024 (385 comments)
| andai wrote:
| >Yao can't be compiled in the traditional way.
|
| >Instead, Yao will be compiled to an LLVM-like IR (which already
| exists), and that's how it will be distributed. Users will have
| to do the final compile step on their local machines.
|
| Is this referring to the compiler itself, or programs written in
| Yao?
| gavinhoward wrote:
| Programs written in Yao.
| kibwen wrote:
| If the goal is portability, Google pursued the idea of
| distributing cross-platform LLVM IR via PNaCl and ultimately
| abandoned it, because LLVM IR isn't really designed for
| portability.
| gavinhoward wrote:
| I designed my own, uncreatively called Yvm. Yao already
| compiles to it.
| andai wrote:
| Is there something like this but without concurrency? I don't
| need concurrency or multithreading, and most of my pain with Rust
| has come from the fact that it prevents me from doing things that
| I know are correct because it thinks my code will run in a
| multithreaded environment (it will not).
|
| It seems like Yao's whole project is solving the concurrency
| problem.
|
| I'm like, why not drop it entirely (if performance isn't one of
| the language's top priorities anyway)? What does keeping it gain
| that's so important? (Context: I don't understand concurrency and
| multithreading.)
| gavinhoward wrote:
| Good question.
|
| Yao's concurrency will be explicit. You will not get
| concurrency unless you open a threadset and spawn threads. Yao
| scripts run in a single thread by default.
|
| I focused on solving the concurrency problem because if it is
| solved, single thread code is solved as well.
| skitter wrote:
| > it prevents me from doing things that I know are correct
| because it thinks my code will run in a multithreaded
| environment
|
| Could you elaborate on that? In your case you don't need to
| worry about Send/Sync, you can use Cell/RefCell, etc. The
| borrow checker isn't about threading, it's useful in a purely
| single-threaded context too.
| umanwizard wrote:
| Not OP but don't a lot of async libraries require things to
| be Send because they assume they'll run on a multithreaded
| executor?
| ruthmarx wrote:
| > I don't need concurrency or multithreading, > ... > Context:
| I don't understand concurrency and multithreading.
|
| I get not needing it, but if you don't understand concurrency
| or multithreading how are you so certain you don't need
| concurrency or multithreading?
| j1elo wrote:
| > that language is coming into existence right now! It is called
| Yao.
|
| Wasn't it called _Mojo_? It 's been a while since I saw traction
| of that new language over here.
|
| Anyhow, it is interesting how many new languages are appearing
| every day. The bar is indeed constantly moving upwards.
| cryptonector wrote:
| > This is actually the root of the function color problem: async
| is exactly backwards! Instead of marking asynchronous functions,
| languages should have had us mark synchronous functions, and then
| async should have been the default!
|
| What happens when you need to add a new trait that all existing
| functions must be changed to have? That's why async is generally
| an attribute.
|
| The answer is that we'd have to say that all functions have some
| default trait set that can be extended automatically as the
| language evolves, except those anti-traits they have. Then we
| could say that a function is async meaning "not sync", or even we
| could write !sync instead of async.
|
| I rather like the idea that you have to write `!sync` instead of
| `async`.
| gavinhoward wrote:
| The post answers that.
|
| Every unmarked function can do anything. If you add a new thing
| that functions can do, you assume unmarked functions can do it
| (even if they don't), and you add a trait for the negative.
| cryptonector wrote:
| Thanks, I got interrupted in my reading.
| klabb3 wrote:
| > The Rust async book mentions three more methods: [...]
|
| Hah, I wrote this section and was deeply aware of these problems
| at the time. Let's just say it didn't fit in the introduction. In
| fact, a lot of the understanding of this issue has been expressed
| later, blog post by blog post, largely without coordination. It's
| almost eerie to see again and again people encountering the same
| issue, and spending huge effort to address it (this one takes the
| prize though - making your own language).
|
| Let me try a to narrativize it: we have a fundamental problem
| with unrestricted concurrent control flow (very similar to how we
| used to have unrestricted _sequential_ control flow when we had
| goto). This problem exists in most programming languages, but we
| mostly pretend it's not there (and live with the subtle bugs
| instead). However, with Rust's unique ownership- and safety
| model, the lack of structured concurrency was an unavoidable
| problem, at collision course with Rust itself. In the Rust we
| have today (without structured concurrency), a bunch of expected
| and reasonable behaviors cannot be achieved (such as borrowing
| from a parent task).
| throwaway17_17 wrote:
| Do you know of any longer form content that walks through this
| area, or maybe just a listing of blogs. I have a feeling the
| resources are relatively scattered over various sites and
| across time, so any tips on search terms to use if a deep dive
| into this problem (both in Rust and generically in programming
| language semantics) intrigues me?
| genter wrote:
| https://vorpus.org/blog/notes-on-structured-concurrency-
| or-g...
|
| https://dubroy.com/blog/handling-user-input-with-
| structured-...
|
| The first one is particularly good. Both of those have links
| to more topics.
| divs1210 wrote:
| This discussion is seriously lacking in references to Koka
| language[0].
|
| Koka is memory safe without using traditional GC, has effects,
| and is pretty cool over all.
|
| [0] https://koka-lang.github.io/koka/doc/index.html
| SkiFire13 wrote:
| Reference counting is ultimately a GC, although not a tracing
| GC which is the most common kind. I also don't really see the
| appeal of not having a GC in a language like that. If it
| doesn't compete with C/C++ for performance and low level
| support then not having a GC is no longer an advantage.
|
| That said Koka still remains very cool for the effect system
| though, and I would really like to see it in a mainstream
| language!
| tmtvl wrote:
| Serious mode: it's good that people are exploring the programming
| language space, both tiny and big changes can mean a world of
| difference in reducing bugs and making programs nicer both for
| the user to run and the developer to create and maintain.
|
| Serious mode off. When will we get Yao in the Linux kernel?
| gavinhoward wrote:
| > Serious mode off. When will we get Yao in the Linux kernel?
|
| As soon as someone pays me long enough to get it there. :)
|
| I only have 3 hours max a day (usually only 2) to work on Yao.
| And I have other projects I want to do as well.
|
| But if someone paid, I sure could find the desire to work on
| it.
| fuzztester wrote:
| >Serious mode off. When will we get Yao in the Linux kernel?
|
| Yaol can get involved and help with that, yaol know?
|
| Serious_mode_still_off? true
| cryptonector wrote:
| Slightly off-topic, but TFA links to
| https://gavinhoward.com/2024/05/what-rust-got-wrong-on-forma...
| which says:
|
| > But there is an even harder problem: the C10M Problem. Is it
| possible to use RSC to handle 10 million connections, of which 1
| million need some processing every second?
|
| (RSC being restricted structured concurrency.)
|
| IMO the C10M problem requires: a) user-land network stacks or
| unikernel servers, b) NPROC (or fewer) threads or processes, c)
| evented, continuation-passing style code in each of those threads
| or processes.
|
| I've written one such program (though without (a)), a HTTP read-
| only file server that does (b) and (c) and which supports
| "tailing" files by doing `GET`s with `Range: bytes=0-` (or some
| other starting offset, but no ending offset), and it scales to
| tens of thousands of concurrent clients trivially. To get to
| millions of clients I would need to use H/3 and QUIC, and also do
| (a). (That server also supports ETags and conditional requests.
| Tailing `GET`s end when the file is unlinked or renamed away.
| Essentially it is a poor person's Kafka.) This program stores
| state in a per-client data structure and in the form of a pointer
| to a continuation closure that is `{data structure, handler
| closure}` that is registered with the event system, and each
| handler does only non-blocking operations (if you can think of
| filesystem operations as non-blocking anyways, though one could
| also use aio).
|
| I bet this works well with RSC if all the state is kept in
| statically allocated objects. In my program all state is kept in
| dynamically allocated heap objects, but I could have statically
| allocated them instead, perhaps at program startup, or else at
| compile time.
|
| The point here is to absolutely minimize the representation of
| state. In a thread-per-client implementation the state is smeared
| on each thread's stack and also any necessary heap objects. In a
| CPS implementation the state is fully explicit, with none of it
| in any stack frames -- the programmer is in charge of producing
| the most compact state representation, and it's easy because the
| state has to be fully explicit.
|
| I view evented CPS and thread-per-client as the ends of a
| concurrency spectrum where green threads and co-routines are
| somewhere in the middle. If you want C10M you will probably end
| up wanting evented CPS. Green threads / fibers can get close to
| CPS if you work to minimize state anyways.
___________________________________________________________________
(page generated 2024-09-27 23:01 UTC)