[HN Gopher] Kawipiko - fast static HTTP server in Go
___________________________________________________________________
Kawipiko - fast static HTTP server in Go
Author : enduku
Score : 36 points
Date : 2022-08-28 17:50 UTC (5 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Panino wrote:
| > the static content is served from a CDB file with almost no
| latency
|
| That is cool! I use CDB in some projects and love it.
| BossingAround wrote:
| What's CDB? A quick search reveals it's a SAP's data archive
| format? (not sure how it relates to an HTTP server)
| Panino wrote:
| https://cr.yp.to/cdb.html
| ngrilly wrote:
| The fact it is using three different entirely HTTP libraries to
| be able to support HTTP/1, 2 and 3 is kind of telling on the
| complexity of modern HTTP.
| ciprian_craciun wrote:
| [Kawipiko's author here] The main HTTP "engine" behind Kawipiko
| is `fasthttp`, <https://github.com/valyala/fasthttp> that
| supports only HTTP/1.1 (with or without TLS). However for
| experimentation I've also integrated HTTP/2 and HTTP/3 support,
| although the performance isn't that great...
|
| In fact, there is one ticket open on `fasthttp` about HTTP/2
| support, however (as I've also commented on this issue), I
| don't think that's the way to go for the intended use-cases:
| <https://github.com/valyala/fasthttp/issues/144>
|
| My current opinion is this: HTTP/2, and especially HTTP/3, are
| mainly meant for CDN's and highly performant HTTP load-
| balancers (like HAProxy), meanwhile HTTP/1.1 is the best
| solution for the backend.
| cosmotic wrote:
| How does this compare to Caddy?
| francislavoie wrote:
| [Caddy maintainer here] It's probably faster, because it
| focuses on _only_ serving static content. Caddy is designed to
| be general purpose. Caddy uses net /http (Go's stdlib
| implementation), Kawipiko uses "fasthttp" which is a separate
| library that should be more performant for simple uses.
|
| But of course, do your own benchmarking, for your own usecase,
| and see which one works best for your needs. IMO benchmarking
| webservers is usually misleading because everyone has slightly
| different needs and since webservers are designed to be
| configurable.
|
| All that said, with the upcoming release, Caddy will support
| being built with an embedded filesystem with
| https://pkg.go.dev/embed so your static content can ship with
| your Caddy binary and get served from memory. We haven't done
| much testing to see how fast this is in general, but it should
| be pretty fast in theory.
| ciprian_craciun wrote:
| The main problem with Go when it comes to raw performance is
| heap memory allocation. `fasthttp` (and the rest of
| Kawipiko's code on the main serving path) does all it can to
| avoid any sort of memory allocation (or "heap escape").
|
| On the other hand, Go's `net/http` uses quite a bit of heap
| allocation, thus it will never beat `fasthttp` in terms of
| raw performance.
|
| That being said, the numbers are pointless... From what I
| remember Go's `net/http` is able to serve a few tens of
| thousands of requests per second, thus it won't be the main
| bottleneck.
| ciprian_craciun wrote:
| [Kawipiko's author here] Caddy, and Nginx, Apache, etc., are
| general purpose web servers that either serve content directly
| from the file-system or reverse proxy requests to other
| backends. Meanwhile Kawipiko has only one purpose: serve
| resources from an embedded key-value database, namely the
| venerable CDB <https://cr.yp.to/cdb.html>.
|
| While general purpose web servers use open-(read+send)*-close
| syscalls (and other tricks such as `sendfile`) for each
| request, Kawipiko uses a single memory-mapped database for all
| requests, thus all the I/O is mediated (and cached) by the OS.
|
| Thus, one could say that Kawipiko serves a well defined niche,
| heavily optimizing for that particular use-case, meanwhile the
| other servers although can do lots of other things, they can't
| be optimized for any of those use-cases.
| cosmotic wrote:
| I guess I'm curious how all the theoretical benefits pan out
| in the end and if the benefit is worth the separate project.
| the_arun wrote:
| Useful for proof of concepts & private content. I'm wondering,
| for public static content, do anyone use webservers these days
| when CDNs are available as a service?
| ciprian_craciun wrote:
| Well, as the CDN initials hint (Content Delivery Network),
| these services only "deliver" your content; the content must
| still live somewhere...
|
| That "somewhere" can either be another cloud service dedicated
| to static sites such as Netlify, GitHub Pages, CloudFlare
| Pages; a general purpose storage service such as AWS S3; a
| "serverless" service running on something like AWS Lambda or
| Fly.io; or even a self-hosted Linux server (running on hardware
| or as a VM in a IaaS provider such as Linode).
|
| Kawipiko tries to solve the last two scenarios, when one wants
| to serve the content from a self-hosted VM.
| RenThraysk wrote:
| So from a quick glimpse, everything is served as immutable with
| 60 minutes max-age? So swapping out the cdb, will take 60 minutes
| for client caches to be fully updated?
___________________________________________________________________
(page generated 2022-08-28 23:01 UTC)