[HN Gopher] k on pdp11
___________________________________________________________________
k on pdp11
Author : tosh
Score : 97 points
Date : 2024-01-08 14:31 UTC (8 hours ago)
(HTM) web link (ktye.github.io)
(TXT) w3m dump (ktye.github.io)
| xelxebar wrote:
| I've been knee deep in APL for the past year, working on a YAML
| parser: https://github.com/xelxebar/dayaml. It's still very
| prototypy, but for ~30 lines of code it's surprisingly complete.
|
| The process of getting that project to where it is now really
| revolutionized how I think of software development in general and
| how I manage my dev teams at work. Our best practices and wisdom
| around software dev really are much more strongly tied to
| historical habits and tooling whims than we'd like to believe. I
| believe we as a community have a lot of unpicked fruits laying
| around if we shift focus onto the human side of Human-Computer
| Interfacing for a while.
|
| About K, though, Arthur Whitney sounds like such a beast, and I
| would love to mingle with the K community. Within the array
| programming circles, APL/J/BQN form one cluster and K looks like
| it's off on its own a bit with different ideas and a laser focus
| on high performance. Extremely enticing.
| haolez wrote:
| Please do tell more. I'm a CTO struggling with managing my
| devs. What do you mean?
| 7thaccount wrote:
| Not OP, but I can guess a bit
|
| With modern software development there is a ton of
| boilerplate and focus on tooling (how many hours do we waste
| on IDEs or text editors), compilers, build tools... whatever.
|
| With APL or K, you don't really have a lot of that. Once
| someone is proficient, they can do a ton with just a tiny
| amount of code. Aaron Hsu has a bunch of videos online and
| chats on HN (user arcfide I think) on the strengths of APL in
| general and how he built a compiler to run APL on the GPU and
| got it down to like a page of the alien glyphs. His point is
| that with APL you can eliminate huge amounts of things like
| helper functions and see the entire code at a more macro
| level. He literally uses windows notepad to code in as well,
| so a lot less time is wasted on learning/managing/configuring
| all the tools in the ecosystem.
|
| Supposedly they've done some studies where people can be
| taught APL pretty easily if they go in with an open mind. I
| can code a bit in it after only playing with it a bit. The
| symbols are a LOT more intuitive than you think (a lot of
| effort went into making them make sense in representing the
| operations they do). The big problem is professional devs
| don't want to get locked into a niche technology or admit
| there are other ways to do things that don't involve Java-
| like hell. This is less of a problem with finance workers
| using Kdb+ (a very fast array database that you access with
| the k or q languages) that get paid $$$$$$$ to write stock
| ticker analysis software.
|
| As a CTO, I'm not sure what the lessons are except keep code
| more reasonable. Don't write stuff you don't need...etc. I'm
| guessing OP has gotten use to the expressive power of APL and
| the interactive nature of the software and how to keep things
| lightweight and now sees a lot of modern software as being
| overly bloated. It reminds me of a vendor that shipped us
| what should have been a page long application (pretty simple
| data transformations), but instead it was like 50 class files
| and all sorts of other junk. I was blown away by the
| inefficiency and extreme complexity for what we later wrote
| ourselves in a short script with few functions. Assuming it
| wasn't outright on purpose, I think certain industry tools
| encourage over architecture.
| steveBK123 wrote:
| I think the problem is that most "big data" problems aren't
| really that big, or the latency requirements aren't that
| stringent. So people are OK with much slower solutions they
| can throw midlevel python devs & parallelize AWS compute
| at. With true time series problems you do get many "big
| data" problems that you can't actually parallelize (think
| sequential event based operations where the previous
| results are needed for deciding what to do with the next
| event).
|
| With data frames you get some of the "close to the data"
| interactivity you get with a APL/J/K/Q stack. Of course, K
| is ~100x faster than Polars which is what.. ~10x faster
| than Pandas. Meanwhile most people are still using Pandas
| in a Jupyter notebook despite Polars being right there.
|
| You'd think as people look at their AWS bills they'd be
| considering solutions that use 1/10th, 1/100th and even
| 1/1000th the compute time.. and yet.
| 7thaccount wrote:
| They may be breaking even after firing all their internal
| server people...idk.
|
| Other solutions like Kdb+ may be way more efficient, but
| are also crazy expensive from what I've heard.
|
| Is there a free lunch in this context?
| BMarkmann wrote:
| If you mean "free" as in open-source/free, there is J,
| which has its own builtin database. I'm assuming it's
| similar to k/kdb, but that's just a guess.
|
| Learning an APL variant... not a free lunch. Takes a
| while and commitment to grasp.
| steveBK123 wrote:
| Arguably "crazy expensive" software that minimizes crazy
| expensive hardware is something work considering. Firms
| pay 10s of times more for things like DataDog than they
| do for a KDB+ license.
|
| Also I found the argument that KDB+ devs are expensive to
| be laughable once I saw how much (same, or even more) we
| started to pay AWS/Python devs.
| mhuffman wrote:
| If you like podcasts you might check out the Array Cast podcast
| if you haven't.
|
| >https://www.arraycast.com/
| Duanemclemore wrote:
| Wanna second the Array Cast. Really, really solid stuff even
| if you're not specifically interested in the array languages.
| mhuffman wrote:
| Yes, I didn't mention that but they have a rotating
| schedule of some of the most elite programmers you have
| ever heard. I will also say that array programming is very
| seductive if you have the "puzzle solving" mentality that a
| lot of us have. It seems so elegant. I have never found a
| legitimate use for it in my paying work, but I am
| captivated by it privately.
| gitonthescene wrote:
| What bars do you use to measure "legitimate use"?
| andai wrote:
| I laughed when I saw the source: "just" 30 lines, to be sure,
| but 30 lines of dense, alien glyphs! This is probably 300 (or
| 3000?) in a less expressive language, right?
|
| What's a good place to start for languages like this? k is
| proprietary, isn't it? I found unrelated languages with the
| same name on GitHub.
| RodgerTheGreat wrote:
| there are at least three reasonably comprehensive open-source
| k implementations:
|
| https://github.com/kevinlawler/kona (k3)
|
| https://github.com/JohnEarnest/ok (k6, js-based)
|
| https://codeberg.org/ngn/k (k6, c-based)
|
| even today, many k programmers learn from the k2 reference
| manual. k2/k3 are extremely similar:
|
| http://web.archive.org/web/20050504070651/http://www.kx.com/.
| ..
|
| oK has a briefer manual which describes k5/k6:
|
| https://github.com/JohnEarnest/ok/blob/gh-
| pages/docs/Manual....
| mhuffman wrote:
| >What's a good place to start for languages like this?
|
| I recommend:
|
| https://xpqz.github.io/learnapl/intro.html#but-it-s-
| unreadab... (for apl)
|
| and
|
| https://xpqz.github.io/kbook/Introduction.html (for k)
|
| ... the author admits they are not an expert in either, but
| the pace and tone are very beginner friendly, which I have
| found is necessary if you do not have any experience in array
| languages.
| skruger wrote:
| I'm happy to have contributed in some small way to growing
| the APL community. Thanks for the shoutout.
| ithkuil wrote:
| you may also enjoy uiua (https://www.uiua.org/) which uses
| these alien glyphs but is even more alien because it's a
| concatenative language (stack oriented), like forth or
| postscript, but to make it even more alien it's written right
| to left. For example 1+2 is written "+ 1 2" (in forth it
| would be "1 2 +")
|
| The language and the site are brilliant and I think worth 30m
| of your time skimming through and trying out the examples in
| the online editor / tutorial.
| uticus wrote:
| The only reason stack-based languages don't seem like alien
| tech to me was because of the HP48G+ graphing calculator I
| had in high school.
|
| > Uiua can be embedded in Rust programs as a library.
|
| Cool! I was wondering if there was any viability beyond
| short scripts.
| ithkuil wrote:
| I still use reverse polish calculator (on my phone) but
| I'm not used typing it left to right but executed right
| to left.
|
| For example in uiua: * + 1 2 5
|
| Pushes 5 on the stack, then pushed 2, then pushes 1. Then
| pops 1 and 2 adds them together and pushes 3, then pops 3
| and 5 and multiplies then together
|
| Reverse polish: 5 2 1 + *
|
| EDIT: it does makes sense in the context of an array
| language
| plagiarist wrote:
| The uiua version makes more sense to me than the other
| even without arrays. It's closer to how my brain wants to
| understand what is happening: "multiply (the result
| of...) and 5" instead of "5 (2 and 1 added) multiplied."
|
| The latter is kinda like "the old man the boat" where
| information occurring later in time changes the meaning
| of what comes prior.
|
| I'm sure once things get complicated enough I would
| understand neither RtL or LtR with equal confusion, but
| the smaller cases seem very readable.
| ithkuil wrote:
| Interestingly the uiua order is the same as lisp (if you
| add some parenthesis) (* (+ 1 2) 5)
| jd3 wrote:
| From experience, my take is that most laymen devs are
| unwilling and/or uninterested in taking the dive into
| apl/k/array langs due to the, in their own words,
| deliberately abstruse and obscurantist syntax/glyphs/typing;
| it's just too theoretical and/or cognitively/neurologically
| daunting for average engineers that just want to get paid.
| 7thaccount wrote:
| I think it's a combination of that and being taught for so
| long the "right way" to do things with languages like APL
| appearing "too clever".
| jd3 wrote:
| Afaik most undergrad education is still explicitly
| procedural/imperative/OOO with a dash of lisp/scheme in
| theory of comp.
|
| Barring the exceptionally driven and programs that are
| unusually theoretical and/or historically based (i.e.,
| SICP), many new grads enter industry having only ever
| used js/python/java/c++ it seems.
| uticus wrote:
| > abstruse and obscurantist syntax/glyphs/typing
|
| I've often wondered what are the real blocks to having it
| both ways? Have the source saved as some lowest common
| denominator - IL, tagged AST, or somesuch - and then read
| and written in whatever pleases the individual. It would be
| hard to make it work between languages with drastically
| different semantics that don't map to anything in another
| language, I suppose would be the real reason... but I
| always think about the fact that it makes it down to
| machine code eventually, so it seems intuitive that the
| process could be reversed.
|
| Or less drastically, an IDE for APL beginners: symbols have
| human language helpers or UI overlays, but the source is
| still APL.
| RodgerTheGreat wrote:
| The concise notation of vector languages is not just an
| affectation; it's a crucial property. Obscuring the
| notation with long names and "more familiar" syntax
| diminishes its value. Translating it into something
| "easier" leaves beginners unable to communicate clearly
| with APL experts or engage with resources from the
| broader community.
|
| Imagine suggesting to a Prolog programmer that they make
| the language more accessible to beginners with tools that
| dynamically translate it into the equivalent Fortran,
| which is, after all, a Turing-complete language...
| plagiarist wrote:
| I'd need some stronger evidence to believe the claim that
| in-place translation of individual glyphs into words is
| equivalent to translating between languages with wildly
| different paradigms.
| gitonthescene wrote:
| I'd need some stronger evidence to believe the claim that
| in-place translation of individual glyphs into words is
| equivalent to translating between languages with wildly
| different paradigms.
| uticus wrote:
| Ah, that makes sense. Okay, you've convinced me my
| original idea doesn't make much sense after all!
| adregan wrote:
| > Or less drastically, an IDE for APL beginners: symbols
| have human language helpers or UI overlays
|
| Dyalog, which maintains a commercial APL, provides a free
| development environment which does exactly what you
| want[0]. You can hover over all of the available glyphs
| for documentation.
|
| It's a really fun development environment in that it does
| something that I've never really experienced in any
| another: you can modify prior inputs to experiment with
| new ideas.
|
| Playing with APL (it often does feel like play, or even
| sculpting with clay) is really fun. Be careful, you might
| get attached!
|
| 0: https://www.dyalog.com/
| uticus wrote:
| > you can modify prior inputs to experiment with new
| ideas
|
| having trouble understanding what is meant. like,
| changing things like in excel?
| abrudz wrote:
| Try that experience at https://tryapl.org
|
| Enter 1+2 where the cursor is below the copyright notice.
|
| Now click or use arrow keys to navigate up and change the
| 2 to a 4 and hit Enter again.
| uticus wrote:
| i mean that's cool, but from my background i would say
| since the input is "variable", assign it to a named
| variable and run the function with the variable assigned
| to 2, then again with it assigned to 4. that's doable in
| most any REPL.
|
| or even take the function, and map a range of values to
| the function, to get a range of outputs. again, doable in
| most REPLs where the language supports some sort of
| mapping syntax.
|
| can you help me understand what makes this unique? i'm
| not an APL user, so maybe it's just a different mindset
| altogether that i'm missing.
| adregan wrote:
| You know how in a typical REPL, you'll make use of the up
| arrow to repeat a previously executed expression?
|
| Dyalog allows you to click on the previously printed
| expression, modify it in place and when you hit enter,
| it'll evaluate as if entered on the read line.
|
| It's a simple little feature but it makes everything feel
| so malleable.
| 082349872349872 wrote:
| > _for average engineers that just want to get paid_
|
| Eukleides' (fl 300 BC) hot take: https://mathshistory.st-
| andrews.ac.uk/Biographies/Euclid/#:~...
| Exolon wrote:
| Imagine if someone wanted to play music in an orchestra,
| but couldn't read music because it was "too confusing", but
| they'd never even tried to learn. They would be laughed out
| the door.
|
| I'm not sure why so many laymen devs you mention (who,
| incidentally, tend to have an affinity for learning music
| and other "esoteric" skills) seem to apply that same logic.
| Like, why not learn and try the basics before throwing your
| hands up and saying it's too hard. It isn't... although it
| is very, very different.
| msteffen wrote:
| I'll second the request for elaboration! I'm interested in APLs
| but have never tried writing more than a tutorial.
|
| Any particular examples of things that people {are,aren't}
| doing, that they {shouldn't,should} be doing that you felt like
| sharing, or just examples of things you realized while writing
| this, would be really interesting to hear!
|
| I (and I know lots and lots of others) in tech feel like we
| could be doing better but don't know how, and APL is
| interesting because it attracts such veneration despite being
| so weird
| gitonthescene wrote:
| I've found the notion of a parent vector to be useful for
| parsing problems. I like to call them Apter trees after this
| article http://archive.vector.org.uk/art10500340, but it's also
| the core idea in Aaron Hsu's thesis.
| gitonthescene wrote:
| [?]\ is an awesome trick. You might also like {2 |
| (1+[?][?][?]) ([?] - [?]\[?]x) ~[?]}` for `[?]IO-0 for
| detecting which escape characters are themselves escaped.
| This is translated from BQN:
| https://github.com/mlochbaum/BQN/blob/master/md.bqn#L53
|
| This last is more compactly expressed as <\ when you have
| left to right folds which (unfortunately) APL does not.
| hhyndman wrote:
| Back in the early 70s, I wrote software for the PDP11/20 (in ASM)
| to collect data from counters for accelerated particles. I recall
| handcrafting interrupt service routines to ensure they stayed
| ahead of the counters. And using paper tape to load the software
| after I hand-toggled the bootstrap loader.
|
| For the next 15 years, I was an APL programmer, and later a Q/K
| developer. It would have been a hoot to use K was back in the 70s
| to analyze the collected data!
|
| What a fun project! Bravo
| vok wrote:
| Beautiful. Try typing:
|
| 2 3 + 4 5
|
| in the upper-left, under "kyte/k".
| Duanemclemore wrote:
| This is so perfect. Even just the PDP-11 color scheme hits right.
| The documentation is incredibly helpful in showing how both the
| language and the machine work. It looks like a great learning
| tool, but also you could play with it for hours.
|
| My personal favorite piece of K lore is that the -entire
| language- supposedly fits in processor cache, which is why it's
| so mind-bendingly fast. Apparently it runs something like 85 -
| 90% of the algorithmic trading industry for this reason. I could
| have the details wrong, but...
| steveBK123 wrote:
| The entire binary would still fit on a 3.5" floppy and still
| have room for your application.
| Duanemclemore wrote:
| If I could afford a career change it I'd be soooo tempted.
| The idea of being on the ground floor of implementing K on
| edge computing / "IoT" for construction automation is really
| interesting.
| Pompidou wrote:
| I've just discovered array/ matrix languages, and especially J.
| It's so a pleasure to use it. I struggle since a long time to use
| python or r for their mixed scalar/array style. K/J/apl are so
| clean and well designed in this regard.
|
| For instance: Eratosthene's sieve to get all prime under 100 : "
| (-.] * /])2}.i.100 " Another great property of J is his natural
| language like structure :
|
| without =: -.
|
| multiplied_by =: */
|
| upto =: 4 : 'x}.i.1+y'
|
| Sieve_100 =: (2 upto 100) without (2 upto 100) multiplied_by (2
| upto 100)
| abrudz wrote:
| That's not Eratosthene's sieve, but rather a naive O(n2)
| algorithm, implementing "a prime never occurs in the
| multiplication table".
| nerdponx wrote:
| > mixed scalar/array style
|
| For whatever it's worth, R specifically is almost entirely
| arrays, with the exception of NULL.
|
| Even language constructs like functions, including source code
| itself, are represented as heterogeneous arrays (called
| "lists"). I'd go so far as to claim that R is an homoiconic
| array language.
| moonchild wrote:
| > upto =: 4 : 'x}.i.1+y'
|
| Fyi: for a while now, j has supported the more convenient
| 'direct definition' syntax. You can now write that definition
| like:
|
| upto =: {{ x}.i.1+y }}
|
| It will automatically make a monad or a dyad depending on
| whether you use x or not (or an adverb or a conjunction if you
| use u or m or v or n).
| jd3 wrote:
| What's the deal with Arthur Whitney's Shakti these days? The
| download has been gone for a year or two at this point.
|
| https://shakti.com/
| 7thaccount wrote:
| I took wonder about this. I guess you have to get in touch with
| them.
| eismcc wrote:
| Judging by the google group, they are in early development
| phase and are onboarding their first customer.
| i_don_t_know wrote:
| No idea what the current state is. The other day I found an
| older version that runs in your browser at: https://kparc.io/k/
| Exolon wrote:
| That's fantastic. It's a shame there isn't more stuff written
| (and especially, videos/streams) about K, but I get the feeling
| that will improve since there are a good few open-source K-like
| implementations (at least ngn/k, oK, ktye/i, kona, kuc and Goal).
| Recently I stumbled upon iKe
| (http://johnearnest.github.io/ok/ike/ike.html) which is pretty
| amazing.
___________________________________________________________________
(page generated 2024-01-08 23:00 UTC)