[HN Gopher] Towards Modern Development of Cloud Applications (2023)
___________________________________________________________________
Towards Modern Development of Cloud Applications (2023)
Author : signa11
Score : 130 points
Date : 2024-01-14 07:17 UTC (15 hours ago)
(HTM) web link (dl.acm.org)
(TXT) w3m dump (dl.acm.org)
| iamgopal wrote:
| Isn't this prevailing wisdom among HN crowd ?
| saulrh wrote:
| I think that this might actually be something new - you have
| the same topology as a microservice architecture, but rather
| than actually programming and deploying one binary per
| microservice, you build a single binary with all of the
| functionality and it gets deployed as a fleet of microservices,
| running a different subset of all of that functionality
| depending on what it's deployed as. Which means you have only
| one build and one deploy step and atomic deploys, like a
| monolith, but you also get to scale each component
| independently and isolate data like with microservices. A
| really nice looking middle ground.
| jen20 wrote:
| There's absolutely nothing new about a binary that can be run
| in a number of different roles depending on configuration
| (either statically configured or assigned by some controller
| process).
|
| For example, FoundationDB does exactly this (via dynamic
| assignment), as do many other databases. All of the HashiCorp
| runtime tools also do it. I'm sure there are also much
| earlier examples.
| zilti wrote:
| Ah, the good old "worst of both worlds" approach, I see
| saulrh wrote:
| This looks like it parallels the shift from MapReduce's paradigm
| of one binary per execution graph node to FlumeJava (and Apache
| Beam's) monolithic binary for all worker nodes that reconfigures
| itself as necessary for each stage. My experience is that
| Flume/Beam is a lot nicer in almost all ways, so I'm not
| surprised that the same thing works for services too.
| quickthrower2 wrote:
| Sounds like DCOM?
| crimbles wrote:
| Yeah exactly that. DCOM+DTC+MSMQ is the old new and the new
| old.
| sgt101 wrote:
| Bold claims...
|
| Performance is evaluated against a single example implementation
| (section 6.1) and 9x improvement was achieved when co-locating
| into a single process. To be fair this seems a reasonable thing
| to do if you can get the application to be efficient enough to
| allow it - but there may be very good reasons not to do it in
| many applications.
| CharlieDigital wrote:
| Network I/O is almost always the slowest part of any
| application regardless of the protocol and data encoding
| mechanism.
|
| It seems to reason that if you can co-locate your calls into a
| single process, you'd gain _at least_ 9x.
| sgt101 wrote:
| Yeah - I think that their argument would be that their
| approach enables you to do that, and that there are
| significant gains on the table without that last step.
|
| My main critique is that it's one example, and it would be
| good to see the technique exercised across a number so that
| we can see the strengths and weaknesses.
| bob1029 wrote:
| For same process + core + hot cache & pipeline, your upper
| bound is something like 1,000,000x faster than a trip within
| the same datacenter.
|
| This intuition is what encouraged us to run with SQLite in
| production back when most developers didn't think it was a
| good idea.
| dti wrote:
| ~2.5x when not co-located isn't bad either; and the beauty of
| it is that the programming model lets the runtime system
| perform these relocations, based on the application profile.
| revskill wrote:
| I've used this paradigm for a long time. What's new here ?
|
| Basically your code has no concept of "network boundary", there's
| only package import each other, there's no "microservice".
|
| But when deploying, i can choose which package to be deployed as
| a service.
| kitd wrote:
| What's new is that you don't choose, the runtime does,
| generating the cross-network interface as needed.
| revskill wrote:
| No way (or no need) to be such generic. You should have a
| choice (at least it's my choice). Because your app is
| separated by functionalities, not technical aspects.
| danielvaughn wrote:
| Unless I'm misunderstanding, separation by functionality is
| only useful for engineering teams. Once it's in production,
| the only thing that matters are the technical aspects. By
| technical aspects, I'm interpreting to mean mem/cpu usage,
| throughput, security requirements, etc.
| erulabs wrote:
| Interesting! This has been my recommended pattern for a for years
| now: a monorepo with multiple entrypoints. Worker and api and
| emailer etc etc services are running independently, but it's one
| codebase running with different options. All the developer
| benefits of a monolith and all the devops benefits of sane and
| isolated deployments.
| rezonant wrote:
| Also often allows you to simplify the development experience
| dramatically-- instead of spinning up twenty processes and
| running a complex live rebuild process or manually rebuilding
| and restarting certain parts, you just need to start the whole
| thing, because in development it all runs in a single process
| (or, at least less processes than you do when it is run in
| production)
| m0llusk wrote:
| So shipping two applications that use the same authentication
| service is a mistake?
| rezonant wrote:
| I've been experimenting with this sort of thing for awhile, but
| without automation to determine the deployment topology (nice!).
| Lacking a better name, I've been calling it a monolithic micro
| service architecture. Obviously nothing here prevents you from
| creating a distributed monolith, but it can allow a smaller team
| to enjoy better horizontal scaling without as much cost, while
| still retaining ease of forward development.
| tlarkworthy wrote:
| I am a big proponent of the fat lambda paradigm, but it falls
| short in a few areas and then you need to split - security
| boundaries (e.g. isolating user code) - dependency overheads
| progbits wrote:
| https://serviceweaver.dev/
|
| Because it is a bit buried in the paper, this is the prototype
| implementation they talk about.
| donjoe wrote:
| Discussed here: https://news.ycombinator.com/item?id=34986267
| mkleczek wrote:
| Mobile code and "agent systems" were very fashionable 20 years
| ago. Java introduced built-in RMI with automatic stub
| downloading. In 1998 Sun published
| https://en.wikipedia.org/wiki/Jini that was an extension of this
| idea. Several higher level frameworks emerged (JavaSpaces among
| the most prolific).
|
| Reading the paper two thoughts come to mind:
|
| - "What's old is new again"
|
| - "those who do not learn from history are doomed to repeat it"
|
| There were several attempts in history to implement transparent
| RPC (new again). All failed due to
| https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...
| (learn from history)
|
| Looks like any abstraction trying to hide distribution is
| inherently too leaky.
| signa11 wrote:
| > - "those who do not learn from history are doomed to repeat
| it"
|
| or as mark-twain would (probably) say
|
| "History Does Not Repeat Itself, But It Rhymes"
| pjmlp wrote:
| On the same vein, I now take a kick out of people criticizing
| Java and .NET application servers, while at the same time
| praising delivering WASM containers with Kubernetes YAML
| spaghetti into production.
|
| Need to get that VC money into the hot WASM space.
| mkleczek wrote:
| Let's be honest - deployment story with J2EE application
| servers was total mess and I still have nightmares.
|
| But you're right - today's world of YAML programming is no
| better.
|
| Nix and Guix are steps in the right direction (but both have
| their fair share of issues)
| pjmlp wrote:
| I rather deal with Websphere 5 (on purpose not the later
| much improved 6 and 7) than Kubernetes.
| c0l0 wrote:
| I find it very amusing that most of the people who abhor
| dynamic linking of libraries and executables as an over-
| complex, error-prone mistake of computer history these days
| seem to have not the faintest trace of restraint when they're
| about to turn EVERYTHING into a RPC/RMI over complex, multi-
| layer (and I don't mean OSI), often completely proprietary and
| opaque network abstractions like there's no tomorrow.
| projectileboy wrote:
| Your last sentence is almost the only reply the article
| requires.
| dti wrote:
| Regarding the comparison with RMI, the authors did mention it:
|
| > Java RMI use a programming model similar to ours but suffered
| from a number of technical and organizational issues [58] and
| don't fully address C1-C5 either
|
| Apart from that, it looks like Java RMI allowed remote objects
| returning other remote objects, rather than only immutable
| values. With that you could abuse it by making a call to one
| java.rmi.Remote object, getting another java.rmi.Remote object
| in response, then passing it around, and then finding a totally
| different subsystem suddenly make RPCs (however, such abuse
| probably would be easy to spot in a code review, as it requires
| a modification to the remote object interface).
|
| ---
|
| The authors also acknowledge that it doesn't solve the
| distributed computing challenges:
|
| > our proposal does not solve fundamental challenges of
| distributed systems [53, 68, 76]. Application developers still
| need to be aware that components may fail or experience high
| latency
|
| I think at least in terms of latencies their platform can
| occasionally inject latencies into some percentage of the
| tasks, then verifying if any alerts fire, if there is a fear
| the components become dependent on a certain deployment shape
| (within a cluster).
| mkleczek wrote:
| > Apart from that, it looks like Java RMI allowed remote
| objects returning other remote objects, rather than only
| immutable values.
|
| Actually this is a very powerful concept as it allows one to
| achieve high level of reuse. Jini
| (https://jan.newmarch.name/java/jini/tutorial/Jini.html) made
| mobile objects the core idea of its architecture.
|
| How powerful the approach is one can see when looking for
| example at Jini concept of a Lease and LeaseRevenewalService:
|
| a server program can register an object (client side
| implementation of a service) in ServiceRegistrar (to make it
| discoverable and downloadable). Registration is lease based
| so the server has to renew it periodically. But it can be
| delegated to a LeaseRenewalService that can do it on its
| behalf so that the server can go to sleep (ie. not use any
| server machine resources).
|
| All of the above happens without any party a-priori knowledge
| about any code that needs to be present at use site - code is
| downloaded automatically on-demand - the only thing common to
| client and service is a Java interface.
| dti wrote:
| > trying to hide distribution
|
| The paper unfortunately hides that in reality you have to pass
| a context object in your RPC calls, hence there is no ambiguity
| whether you are calling a potentially remote object.
|
| It's in the example on the project home page:
| https://serviceweaver.dev/ // The "RPC" handler
| func (adder) Add(_ context.Context, x, y int) (int, error) {
| return x + y, nil } // The call-site var
| adder Adder = ... // See documentation sum, err :=
| adder.Add(ctx, 1, 2)
| signa11 wrote:
| the _proclet_ name brings back so many old memories of the
| wonderful technology at startent-networks ! what is described
| here is how (in principle) various things (inter)worked.
|
| what a lovely time !
| asim wrote:
| Google has always been a head of the curve: from mapreduce to big
| table and grpc. They're always building for scale and
| productivity at the company. Service weaver is an evolution of
| that. The idea that developers themselves don't have to decide
| how to separate and manage services. There's so much plumbing to
| grpc that's unnecessary for developers to deal with. But the
| service boundary and development style that lets you write
| modular code is really important. The thing is, most code has
| overlapping concerns and cross dependencies. Network based
| boundaries using APIs were a way to allow teams to operate in
| isolation while providing external APIs to access different
| services but it comes at a huge costs. It was the hammer for
| people and team scale, not compute scale. Now the tools are
| evolving, where we can actually operate on monolithic codebases
| at huge scale as Google and others have shown and that also means
| the deployment technology can also evolve to cater to that mode
| of development while seamlessly handling the technical scale
| details for separating and deploying the code for specific
| services across data centers and the cloud at large.
|
| What does it all mean. Well it's a technology built for Google
| scale. It may have merits in other place as a lot of tech has
| done, but at the same time, for 90% of teams this doesn't matter.
| You have a monolithic code base in a single repo and you can
| deploy and vertically or horizontally scale quite easily
| depending on your requirements. For companies that are 200+
| engineers split across 15-20 teams this might matter. They
| already be doing some sort of microservices or service splitting
| while still using a monorepo. Being able to remove a lot of
| platform level code that you manage versus it being an open
| source thing is advantageous because you can go back to focusing
| on the business case not the glue code.
| pjmlp wrote:
| There is nothing about being ahead of the curve with gRPC.
|
| That is only people getting the point parsing JSON and XML all
| over the place doesn't scale and there is a reason why SUN-RPC,
| DCE, CORBA, DCOM, Java RMI, and .NET Remoting existed in first
| place.
| synthc wrote:
| Interesting paper, i've thought about this topic a lot.
|
| In previous projects I defined systems in a single code base, and
| parts could be deployed separately by providing different
| configuration files.
|
| It was a very productive approach: one could run the whole system
| in a single process during development, which made writing
| integration tests a lot easier than spinning up dozens of docker
| images.
|
| It still required some manual work, and deployments were still to
| static for my liking. Ideally it should be possible to split off
| and scale subparts dynamically.
|
| In the Clojure world there are several projects now that explore
| splitting up services in a transparent way.
|
| For example Electric Clojure splits up your code into frontend
| and backend parts, making the frontend-backend split transparent.
| Another project is Rama, which does something similar but for
| distributed steam processing and partitioning.
|
| I'd love to explore something like this but for enterprisey
| service meshes: the programmer just defines services, and a
| compiler decides how to split these over different machines, and
| all the RPC/serialization/deserialization is done for you.
| _a_a_a_ wrote:
| Regarding your last paragraph, interesting idea about the
| compiler doing the work, but isn't RPC/serdes basically built
| into CORBA or OLE transparently? Never used the former, and the
| latter was so long ago I forgotten it, but I thought that was
| the idea.
| fuzzyengineer wrote:
| Interesting indeed. While still doing two services we manage
| local development using gems in ruby which allow us to seamlessly
| include the other service's code as a library and we write unit
| tests using that integration. The two binaries are built as two
| gems with versions and tested locally. When it comes to deploy we
| deploy both at the same time. We get many advantages mentioned
| here while the services operate independently.
| mahkoh wrote:
| The component model seems like it's just a re-invention of JBoss-
| like application servers.
|
| Maybe it is explained somewhere in the paper but how do atomic
| updates work if at least one of the components has to be a
| singleton that does not support rolling upgrades?
| hugodan wrote:
| Feels a lot like old school CORBA - quoting from the paper:
| "Components may be hosted by different OS processes (perhaps
| across many machines). Component method invocations turn into
| remote procedure calls where necessary, but remain local
| procedure calls if the caller and callee component are in the
| same process."
|
| What is old is new again
| RaftPeople wrote:
| I had the exact same thought when I read that.
| qu1cks0rt wrote:
| Implementing proposed approach is not easy without hurting
| maintenance, scalability and readability. At Google, there is a
| dedicated framework for writing distributed systems that
| implements proposals from the document.
| notso411 wrote:
| And microservices and nanorepositories solve this issue I
| assume
| dti wrote:
| Looks interesting.
|
| > The call to hello.Greet looks like a regular method call
|
| That's a departure from how components interact in boq -- an
| internal and widely used production platform that has _some_ of
| the features from the paper. There component interfaces _are_ RPC
| interfaces (e.g., Stubby / gRPC + protocol buffers), and
| interaction between them is possible exclusively through the
| component interfaces. Hence it's very explicit at the call site
| that an RPC is being made (which could happen to execute locally
| with all the standard RPC functionality -- context and deadline
| propagation, etc.).
|
| RPCs looking like regular method calls sound a bit scary (easy to
| miss in code reviews); I wonder if enforced naming conventions +
| IDE + code review tool support would be enough.
|
| _Edit:_ it seems to require to pass a context object, so the
| readers won 't confuse it with a local call (from
| https://serviceweaver.dev/): sum, err :=
| adder.Add(ctx, 1, 2)
|
| ---
|
| Also, the paper claims that most benefits come from a non-
| versioned serialization format:
|
| > Most of the performance benefits of our prototype come from its
| use of a custom serialization format designed for non-versioned
| data exchange [...]
|
| However, I don't understand why local RPC calls have to serialize
| protocol buffer messages -- can't they already pass them as-is to
| the local handler?
|
| (disclaimer: a googler, no internal knowledge on ServiceWeaver)
| sanderjd wrote:
| > _However, I don't understand why local RPC calls have to
| serialize protocol buffer messages -- can't they already pass
| them as-is to the local handler?_
|
| I didn't read the paper in enough detail to know the answer to
| this, but mightn't this enable different implementation
| languages for different components? In my experience, it's
| difficult to accomplish reliably that without using a language-
| agnostic serialization format (like proto).
|
| Even if that's the goal, it seems like a handler could
| determine whether it could elide the serialization depending on
| the implementation details of the components.
| dti wrote:
| I see I may have been unclear -- I was surprised they don't
| use protobufs (which one should be able to pass as-is without
| serialization to the locally-deployed component), but
| apparently using a custom optimized format for non-local
| calls is the primary motivation (not local calls with grpc
| requiring serialization -- that shouldn't be the case).
|
| However, now that I think again about the serialization
| format choice, it may result in a limitation on the size of
| monoliths (in terms of the number of people / teams
| contributing to it). When the number of contributors grow,
| the likelihood of bugs in a binary grows, and teams adopt
| more elaborate qualification processes, and also become much
| more sensitive to binary rollbacks as a remedy to discovering
| bugs in prod. Then they could institute policies like all
| changes should be protected by a feature flag (aka an
| experiment).
|
| If non-versioned serialization format is used, that means
| that the platform cannot possibly rollback a single
| component. However, using versioned serialization won't be
| enough on its own to support per-component rollbacks -- it at
| least requires independent component qualification (where
| each component is tested against "stable" versions of other
| components) + rollback testing to make rollbacks A2 -> B2 to
| A2 -> B1 safe.
|
| I wonder if it's an explicit design choice -- i.e., whether
| Service Weaver supports monoliths up to a certain
| organizational size (and then you should split into separate
| service weaver apps)?
| arter4 wrote:
| In your experience, how does this kind of approach behave with
| asynchronous dependencies?
|
| Let's say you start from a codebase with three portions (call
| them services, modules, whatever): A, B, C, D. A sends a
| (synchronous) remote procedure call to B, which sends a message
| over a message bus which is also used by C and D. C and D do
| not talk to each other except over the bus.
|
| It sounds like this approach would identify the remote call
| dependency between A and B (which could be split into different
| deployment units), but not the message bus usage. Or, at least,
| it can't identify who is subscribing to a topic where B pushes
| its events.
|
| As a result, you would get two deployment modules:
|
| - A
|
| - B and "everything else"
|
| Which doesn't sound right.
|
| Am I missing something?
| charkubi wrote:
| Great ideas. Observations:
|
| - Making remote calls seem like they are local resulted in poor
| design decisions, the benefit of SOAP/REST was that people
| considered what the interface a useful service should be.
|
| - Why not flip it and look to move groups of microservices onto
| the same machine, updating how the app communicate. - if
| component boundaries are fine grained, the combinations of
| local/remote services relative to each other increases, along
| with the testing burden; just because the system hides remote
| deploying, it still should be tested for.
|
| - Incorporating this with storage, eg dynamic shard rebalancing
| would be super cool
| rockwotj wrote:
| I recently wrote a blog post about how I think WasmCloud is very
| close to this with the new component model. They use IDL instead
| of language constructs to specify boundaries. Pros and cons of
| either
|
| https://blog.rockwotj.com/musings-on-kubecon-wasmday-2023
| awenix wrote:
| Looks very similar to Service Fabric
| https://learn.microsoft.com/en-us/azure/service-fabric/servi....
| I have heard it scale upto 2000 nodes and extensively used
| internally at microsoft.
| arter4 wrote:
| I have read the paper, although maybe not deeply enough.
|
| It is an interesting idea, but I'm not sure I'm fully convinced.
|
| Sure, you can parse remote calls and package imports to find
| dependencies. But this assumes applications are integrated using
| remote procedure calls. What if applications talk to each other
| using a message bus? Asynchronous patterns are meant precisely to
| decouple publishers and subscribers - in number (how many
| processes will read a message?), identity (who are those
| processes?) and time. It looks like this system would not be able
| to decouple portions of code talking to each other using a
| message queue.
|
| Any thoughts?
___________________________________________________________________
(page generated 2024-01-14 23:02 UTC)