[HN Gopher] Glamorize your problem domain (2022)
___________________________________________________________________
Glamorize your problem domain (2022)
Author : zdw
Score : 71 points
Date : 2023-03-26 16:09 UTC (1 days ago)
(HTM) web link (twitchard.github.io)
(TXT) w3m dump (twitchard.github.io)
| ftxbro wrote:
| To me this reads like satire. But no one in comments is reading
| it that way so maybe not?
| ftxbro wrote:
| Glamorize your problem domain
|
| by ftxbro - March 27, 2023
|
| I gave a conference talk last year, tldr: "we build apps. Here
| are all the lessons we've learned from framing this as a
| blockchain problem."
|
| My introductory joke was "how to become a blockchain expert in
| three easy steps". Work in software
| Work on a project Decide that whatever you're working on
| is a blockchain
|
| But was I truly joking? I genuinely believe teams should try
| really really hard to draw analogies between what they are
| building and "classic", well-understood, formally studied systems
| and ideas in computing. Is your web API a blockchain if you
| squint? Is your authentication system really a smart contract if
| you squint? Squint, I tell you, squint!
|
| The reason is not so you will somehow be able to apply advanced
| techniques from research journals to your web apps. You won't.
| But there are other advantages.
|
| First, you get a vocabulary for free. If you pretend your app is
| actually a blockchain, you can talk about "blocks" and
| "transactions" and "smart contracts" and "consensus algorithms" -
| things that you would otherwise have to invent bespoke names for,
| or be unable to talk about at all.
|
| Second, analogies can be a good source of interesting ideas. If
| your web app is a blockchain, HTTP is your transaction protocol,
| and user authentication is your smart contract, could you build a
| "mining pool"? What would that mean? Could you attribute the
| performance issues in your app to particular parts of the
| blockchain? Also what are your consensus algorithms? Is it worth
| introducing a new one? Should your app be a proof of work
| blockchain or a proof of stake blockchain? Does that distinction
| even make sense? These won't all be good ideas, it is plain to
| see. But you might strike gold.
|
| Third, morale. It's fun to be able to think of yourself as a
| blockchain* expert, even if the asterisk is big and the analogy
| is a little hazy.
|
| Last, I'll observe it's quite within the realm of possibility
| you'll encounter a problem that truly belongs to the domain of
| e.g. blockchain development, no squinting required. Satoshi
| Nakamoto wrote The root problem with
| conventional currency is all the trust that's required to make it
| work. The central bank must be trusted not to debase the
| currency, but the history of fiat currencies is full of breaches
| of that trust. Banks must be trusted to hold our money and
| transfer it electronically, but they lend it out in waves of
| credit bubbles with barely a fraction in reserve. We have to
| trust them with our privacy, trust them not to let identity
| thieves drain our accounts.
|
| I agree something seems to be awry with the financial industry's
| popular "fiat currencies". You can hardly spit without hitting a
| currency that, frankly, deserves it. Cryptocurrency these days is
| "blockchain heaven". I was on a team years ago that sunk weeks
| into debugging our app's flaky payment system, entirely thanks to
| the traditional banking system's poor transparency and terrible
| security. In theory, cryptocurrencies that are based on
| blockchain technology - i.e. Bitcoin, Ethereum, and others -
| should be better than fiat currencies. In practice, though, the
| blockchains of these cryptocurrencies fail to inherit most of the
| UX benefits of traditional currencies, i.e. it is not natural to
| use them for everyday transactions, they are not widely accepted,
| and they are subject to extreme volatility. (Searching the
| Internet indicates that some of this is possible, I have never
| seen it done.)
|
| Nakamoto, a blockchain developer, unsurprisingly prescribes
| blockchain theory as a remedy. Our cryptocurrencies would be
| better if their authors knew how to write smart contracts,
| formally define their blockchain's consensus algorithms, etc.
|
| Maybe so, but I'll settle for far less than formal semantics. I
| just want the authors of cryptocurrencies to think of themselves
| more glamorously. Don't be a payment system designer - your job
| isn't just to facilitate transactions in a rigid way. Be a
| blockchain developer - your job is to give your users a way to
| express themselves, to send and receive money, elegantly, in a
| decentralized way - to guide them interactively, help them
| discover what is possible, and steer them away from scams.
| Provide the trappings that you would demand of a modern, widely
| accepted currency - a user-friendly wallet, fast transaction
| times, low fees, and so on - or at least as many as you can
| afford.
|
| Doing this correctly without reinventing the wheel (please don't
| reinvent the wheel) almost always involves building your
| cryptocurrency on a larger, established blockchain. I find the
| "Ethereum" blockchain by Vitalik Buterin to be an excellent
| example of what I think is a great UX for a cryptocurrency.
| Ethereum is just a blockchain, and so you can use all the
| features that the blockchain ecosystem provides without any fuss.
| The Dogecoin blockchain by Billy Markus is a less stellar
| example. It is based on the "Litecoin" blockchain, and so gains
| the security of that blockchain - however it obscures the entry
| point and execution path from the user, and so has a poor user
| experience. (It's a similar story for many other
| cryptocurrencies.) Of course, still miles better than traditional
| banking.
|
| Even if you don't work on a finance-related product, the internal
| tools and systems of a large-ish organization probably have a
| "blockchain" or two that would benefit from being recognized as
| such. I often think about a talk I saw that describes IBM's
| approach to supply chain management and "Hyperledger Fabric", a
| blockchain IBM invented for describing supply chain transactions
| and cross-organization requests. I can't speak personally to how
| effective their tool is, but I'm comfortable predicting that it
| is better than the counterfactual where developers just do
| everything in traditional databases.
|
| Go forth, the blog post is ended.
| ucarion wrote:
| There are useful mental analogies between pedestrian line-of-
| business CRUD apps and a compilers. I don't disagree with that
| thesis at all. But I have seen it taken too far. Many engineers
| abstract and generalize business requirements until they find
| them interesting, and then implement against those requirements.
|
| Programming languages are typically designed to be relatively
| simple and easy to compile. Line-of-business software tends to be
| as complicated and arbitrary as the organization requires. Trying
| too hard to make your mundane app into a compiler can push the
| software in a direction that makes it difficult to change in the
| messy ways you'll need to later on.
|
| Note, for instance, that the author worked on a system that
| generates SDKs in multiple languages. This is pretty obviously a
| sort of compiler. Are your requirements really so fancy? Or are
| you making a CRUD app hooked up to a couple external integration
| APIs?
|
| It's useful to see the analogies between one's domain and
| compilers, but see if you can't keep that analogy in your head
| and out of the code whenever possible.
| mattgreenrocks wrote:
| I dislike picking on people like this, but I feel like anything
| that might involve CS theory inevitably elicits this type of
| comment. I'm not sure what drives it, but I don't see how it
| helps, as it essentially amounts to, "don't write boneheaded
| code."
|
| I don't think many people would argue with that. I'm just not
| sure what it is contributing to _this_ discussion.
|
| The number of people who know compilers well enough to know how
| to jam them somewhere inappropriate is probably...startingly
| low, too.
| ElectricalUnion wrote:
| > The number of people who know compilers well enough to know
| how to jam them somewhere inappropriate is
| probably...startingly low, too.
|
| Nah, (almost) everyone that don't have a clue what they're
| doing does this for _every app_ , be it required or not; it's
| however called "Babel" or "Webpack" because "compiler" is
| "scary".
| marcosdumay wrote:
| The discussion is really not about people using compilers
| on their build systems.
| ModernMech wrote:
| > The number of people who know compilers well enough to know
| how to jam them somewhere inappropriate is
| probably...startingly low, too.
|
| Agreed, I had a number of students complain to me in my
| programming languages class that I was focusing too much on
| compilers, and who would ever really need to write a compiler
| except compiler writers? They would rather I had taught them
| Go or Python. But I had to tell them, guys, compilers are
| awesome. There are so many uses! You can use them everywhere!
|
| No one seems to want to :(
| twitchard wrote:
| Any stories to tell about compiler analogies being taken too
| far in web apps?
| mattgreenrocks wrote:
| Interpreters/compilers are one of the most fundamental patterns
| of computer science. They recur _everywhere_ , but you need the
| understanding and the eyes to see them. Ignore them at your
| peril.
| mprovost wrote:
| Reminds me of this classic Steve Yegge post: "If you don't know
| how compilers work, then you don't know how computers work. If
| you're not 100% sure whether you know how compilers work, then
| you don't know how they work."
|
| http://steve-yegge.blogspot.com/2007/06/rich-programmer-food...
| musingsole wrote:
| No True Scotsman.
|
| For years I struggled to grok the difference between regular
| expression and parsing. The conversations I would hear about
| the topic always left me feeling there must be some
| interesting meat to the difference...and so I would go read
| up on it only to find myself with the same problem a few
| months down the line.
|
| The only difference is layers. Enough regular expression
| becomes a parser. A slim enough compiler is just a templating
| engine.
|
| Insisting it's a deep, dark magic and not the advanced
| application of `str.replace` does the industry a disservice.
| serjeem wrote:
| > Insisting it's a deep, dark magic and not the advanced
| application of `str.replace` does the industry a
| disservice.
|
| Respectfully, this perspective contradicts the computer
| science underlying our work--there are in fact different
| categories of string "languages" that can be parsed using
| different technologies, and regular expressions are not
| enough for most "languages".
|
| See https://en.wikipedia.org/wiki/Chomsky_hierarchy for the
| different categories of languages and the required tech to
| parse them.
|
| I consider CS theory to be about the deepest and darkest
| magic I get to use on a daily basis :P.
|
| > The only difference is layers. Enough regular expression
| becomes a parser. A slim enough compiler is just a
| templating engine.
|
| You can't add enough layers of standard regex to parse,
| say, C++ or even a tree of matching parenthesis. You need a
| different tech like CFGs.
|
| Hope this helps improve the grokking! The "interesting
| meat" here actually points at some really deep, fundamental
| CS theory that's really worth knowing.
| senkora wrote:
| Regular expressions can only parse regular languages. More
| powerful parser techniques can parse context-free languages
| or even recursively enumerable languages. These are
| fundamentally distinct levels of complexity.
|
| It is fundamentally impossible to correctly parse e.g. HTML
| with regular expressions. See:
| https://stackoverflow.com/a/1732454
| rob74 wrote:
| Ah yes, _the_ classic StackOverflow answer. If that doesn
| 't teach you not to parse HTML with regex, nothing
| will...
| mattgreenrocks wrote:
| Parsing is way more interesting than regexes. Look up
| recursive descent parsers to get an idea of where to start.
| They are conceptually simple and great for getting an
| overview of how, exactly, you can write a recognizer for
| different sets of text.
| import_awesome wrote:
| Oh no. I wrote a mini-language with a PEG parser and an AST. I
| guess I'm a compiler engineer now.
| seabass-labrax wrote:
| There are so many analogous concepts in the field of computer
| science and beyond (what might be called 'duality' in a
| mathematical setting) that if you actively look for ways to apply
| the wisdom and theory of another domain you will probably
| succeed. What an inspiring blog post - pardon me, I'm off to
| brush up on my Yacc skills!
| tetha wrote:
| Funny. This is happening at work. Like, technically, we're just
| sysadmins with a lot of systems and different services running on
| those. Kinda boring, just like a crud service. But if you squint
| a bit, we're seeing ideas of formal verification, algorithmic
| correctness proofs and empiric experiments creeping and seeping
| into gaps.
|
| Just today, me and a coworker kinda laughed about the fact that
| this was the second or third time we kinda ended up with an
| induction proof about the overall correctness of our backup
| retention handling for off-boarded customers, correctness being
| dictated by the contractual regulations of our customers. When we
| stopped laughing, our team lead pointed out a logical problem in
| one of the steps, and we were eventually able to construct a
| counter-example which would violate the contracts. Dang it.
|
| Call me silly, but at this point I'm entirely ready to take a day
| or three off of other work to pick up a formal verification tool
| to validate our backup retention strategies against the different
| classes of contracts we have.
|
| In a similar vein, we're starting to work towards certifications
| like an ISO27001. And there is a staggering amount of similarity
| between policies and invariants, if you look at them on a
| systemic level. And from there you can enter the ideas of
| different algebraic constructs - just like you can have broader
| and narrower clasess of servers with different amounts of
| invariants holding or not holding for them.
|
| On top, we've started to pull data from large numbers of system
| to jam them through jupyter notebooks so we can use data
| visualization tools to get a stronger impression of what reality
| is like. It's pretty interesting if you can use data
| visualization tools like scatter plots and such to deduce the
| existence of a default value for a config setting. Or, to pull
| data and to realize: "hello. This is reality calling. Two out of
| three assumptions you made about systems are incorrect."
| hcarvalhoalves wrote:
| Not sure about "compilers" specifically, but every software may
| be considered a domain-specific interpreter, if you consider it
| ingests instructions and mutates memory as a result. Many
| powerful software tend to be built around this idea: Emacs,
| AutoCAD, Postscript, OpenGL...
| m12k wrote:
| I find this applies to media production too. When you use
| something like LaTeX to write a report, it seems kinda obvious
| that the .tex file is source code and the generated .pdf is the
| binary built from that source. You use markdown to write blog
| posts and then use jekyll, hugo or similar to "compile" it into a
| static site. But it applies broader too, to images and video for
| example. If I import an image file into my app, but I don't have
| access to or control over the source image file that it was
| scaled and exported from, then I'm essentially building a
| dependency on a compiled binary that I don't have the source code
| for. If Apple decides that all iOS apps should now have a @4x
| high rez version of all images, then I'm not able to create one.
|
| There's a lot of lessons from software engineering that can be
| expanded to other fields with that insight. Best practices for
| version control of source code applies to anything that can be
| called a kind of "source". And you need not just to maintain the
| source, but also the tools with which the source is edited, and
| the pipeline with which that source is transformed into a
| shippable product.
|
| There's value even in the basic idea that any given "thing" can
| have multiple distinct representations, with one optimized for
| editing, versioning and collaboration, while another
| representation might be optimized for distribution and/or
| consumption (and you might have intermediate representations
| optimized for other steps in the pipeline toward distribution).
| It might be overkill for many use cases, but it's helpful simply
| to know that it's a sane model to evolve toward as a project
| grows.
| fudged71 wrote:
| I've been thinking about this for business communications made
| from GPT. A company policy to not edit the outputs, only the
| inputs, and keep them version controlled.
| asah wrote:
| Is it crazy to think that compilers will be revolutionized by AI
| ?
|
| i.e. in the future, DSL/languages will be input and output
| specifications, some guidelines/constraints and optimization
| functions... and maybe not even that? e.g. provide a few examples
| and the neural net writes the specifications, fleshes out more
| examples, you approve this and a (decently optimizing?) compiler
| is generated in seconds... ?
| smegsicle wrote:
| i too look forward to the day when 'don't blame the compiler'
| is no longer sensible advice
| allemagne wrote:
| Makes sense to me. Until we get diminishing returns from LLMs
| it seems like we're on track to get something like that sooner
| rather than later. I mean at some point if we can make an AI
| that actually understands computer science, software
| engineering, and human intentions well enough then why have
| compilers? The AI itself will just "compile" from human
| language to machine code.
|
| Humans will learn to just double-check its work the same way
| that non-programmers currently double-check programmers and the
| same way I double-check compilers in my day job.
___________________________________________________________________
(page generated 2023-03-27 23:01 UTC)