[HN Gopher] InfiniCache: In-memory cache that is built atop ephe...
___________________________________________________________________
InfiniCache: In-memory cache that is built atop ephemeral
serverless functions
Author : mastabadtomm
Score : 62 points
Date : 2021-01-15 09:49 UTC (13 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| aaronbwebber wrote:
| Very important caveats from the paper review at
| https://mikhail.io/2020/03/infinicache-distributed-cache-on-...:
|
| > Mind that the workload was selected to be a good match for
| InfiniCache properties (large objects with infrequent access).
|
| > The hourly cost increases monotonically with the access rate,
| and eventually overshoots ElastiCache when the access rate
| exceeds 312 K requests per hour (86 requests per second).
|
| So this is not a replacement for the most common use cases for
| Elasticache. It is interesting as a cache in front of S3, but if
| you want a cache in front of S3...just use Cloudfront?
|
| Honestly if I was AWS I would be ecstatic if people used this,
| AWS probably breaks even or maybe loses a little bit on optimal
| usage of this, and makes an absolute killing if someone using
| this has a traffic spike and ends up hitting their lambda
| millions of times.
| throwaway888abc wrote:
| InfiniCache: Exploiting Ephemeral Serverless Functions to Build a
| Cost-Effective Memory Cache
|
| The paper name fits the title better.
|
| The keyword here is 'Cost-Effective'.
| shaicoleman wrote:
| Presentation (slides+video):
|
| https://www.usenix.org/conference/fast20/presentation/wang-a...
| dismalpedigree wrote:
| So it uses serverless functions, but still requires EC2. " deploy
| InfiniCache on a EC2 instance with powerful CPU and high
| bandwidth"
|
| I don't understand.
| rossmohax wrote:
| EC2 runs proxy, which does erasure coding and lambda periodic
| pings to keep them running and therefore preserving cache.
| zapt02 wrote:
| So basically abusing how serverless functions are supposed to
| work by "warming them up". I wouldn't count on this working
| long-term as cloud vendors will be more aggressive in
| discouraging this behaviour.
| coredog64 wrote:
| Long before AWS implemented provisioned capacity, these
| kinds of hacks were pretty common.
|
| I would expect that the number of customers that would roll
| this out and run it without any mistakes that drive up the
| monthly spend is small enough that Amazon doesn't care.
| rad_gruchalski wrote:
| Eventually it gives them an incentive to embed such
| functionalities out of the box if they notice significant
| usage pattern. Why fight it when you can embrace it and
| profit from it?
| wcdolphin wrote:
| Very creative, extremely complex, what a fun hack. TL;DR exploit
| the fact that AWS keeps your lambda process running (and in
| memory) beyond the time you are billed for in order to build an
| ephemeral cache. Creates one lambda per slice of cache (think
| redis slots) and provides a proxy to hide the (substantial)
| complexity from clients, keeping those lambdas warm, etc. I could
| see this being useful for cost hacking if the request volume was
| on the order of one request per key every few minutes.
| tener wrote:
| Interesting design, but it seems viable only by exploiting "free"
| cacheing given to AWS lambda functions. There are no guarantees
| here from Amazon, and I'm pretty sure whatever free capacity is
| there right now would evaporate once people started to exploit it
| more. AWS lambda should be more convenient than manual scaling,
| but never cheaper than hand-optimized service implementation.
| rossmohax wrote:
| For serverless to be viable, especially for languages like
| Java, instance reuse is really important, otherwise every call
| results in a cold start with unacceptable delays. So serverless
| provides can't really do much here.
|
| They could rise prices, but that will likely hinder adoption.
| wpietri wrote:
| I suspect they could do a fair bit. An obvious thing is that
| they look for this pattern of behavior and turn up the speed
| with which they kill processes. They're still meeting their
| terms, and the normal Java users never notice.
|
| A more subtle thing is just to tune the lambda eviction code
| to be aware of profitability. The instances that get evicted
| first are the least profitable, the ones that have the worst
| ratio of billable to unbillable GB-miliseconds. Good
| customers get good performance. And the other ones say,
| "Gosh, cold starts are expensive; I'd better pay extra for
| provisioned concurrency."
| jacobr1 wrote:
| I've been toying with the idea of using some form of memory
| suspension. The theoretic cost of a cold start them becomes
| just the time it takes to rematerialize the memory from
| disk. Has anyone seen anything similar out there that could
| be a starting point?
| ignoramous wrote:
| If distributed, lazy, just-in-time, global cache is what you
| want, Cloudflare throws it in free (with a few limitations) with
| Workers [0]. It is so much more straight forward than
| InfiniCache. I use Workers to cache content from few KBs to high
| MBs sourced from single-region S3 (could use Scaleway, Backblaze,
| or Wasabi for even more savings [1]) and DynamoDB (Workers KV can
| store 25MB [2] and is a viable alternative, but then cache may
| not be even required).
|
| [0] https://developers.cloudflare.com/workers/runtime-apis/cache
|
| [1] https://www.cloudflare.com/bandwidth-alliance/
|
| [2]
| https://developers.cloudflare.com/workers/platform/limits#kv...
| rossmohax wrote:
| I like the hacking aspect of their solution. It's always
| interesting to see products and services used creatively.
| MereInterest wrote:
| It's always interesting to see terminology evolve. "Serverless"
| started as a hosting terminology, to mean "somebody else's
| servers". Here, it's still being used to mean "somebody else's
| servers", but in a domain that could reasonably use "serverless"
| to mean "without a server" instead. Caching to a local hard drive
| is, almost by definition, serverless.
|
| It's a very neat application, but the terminology erks me.
| junon wrote:
| "Serverless" was a company name, IIRC. Their model caught on
| and was hijacked by the community. Now it's one of those
| nebulous buzzwords that means everything and nothing.
| erikerikson wrote:
| Yes, Serverless Inc. creators of the serverless framework
| (but it went the other direction). The framework is an open
| source tool that you can use to "deploy" "serverless" assets.
| Basically it covers most of those scripts you usually have to
| write and gives you a declarative format with some great
| variable capabilities that help you managed config and
| declare your components clearly. Running the tool deploys to
| your cloud of choice and results in functions, APIs,
| workflows, databases, or whatever you declare now existing
| and ready for use.
|
| Far more seriously in usage, the framework is separate from
| the class of services regarded as serverless. The star that
| kicked it off was AWS Lambda. Serverless colloquially means
| you are dealing with an abstract service contract rather than
| a server (e.g. the oldest is S3). This removes patching and
| other maintenance that usually does not directly support
| business value. More formally, serverless includes auto-
| scaling to zero, paying only for what you use, high
| availability, and other design patterns most outside the
| large tech houses cannot use at low to no cost.
|
| Full disclosure: I contributed to the serverless framework.
| I'm a biased fanboy.
| rossmohax wrote:
| Name reminds me of horrors of JBoss Infinispan cache
| jarym wrote:
| Infinipain
| dismalpedigree wrote:
| I actually thought it was a reboot of Infinispan cache at first
| numlock86 wrote:
| It's not? Bummer.
___________________________________________________________________
(page generated 2021-01-15 23:03 UTC)