[HN Gopher] Stalwart All-in-One Mail Server (IMAP, JMAP, SMTP)
___________________________________________________________________
Stalwart All-in-One Mail Server (IMAP, JMAP, SMTP)
Author : erlend_sh
Score : 198 points
Date : 2023-07-18 07:50 UTC (15 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| [deleted]
| [deleted]
| yubiox wrote:
| Does it come with a delivery agent like procmail or maildrop?
| Does it use maildir or mbox or its own mailbox format?
| zie wrote:
| If you read the readme, it clearly covers this, it delivers to
| Maildir or blob storage like S3, your choice.
| yubiox wrote:
| Clearly? I searched for maildir and mbox, my browser can't
| find either on that page. What does cloud storage have to do
| with the MDA?
| jonas-w wrote:
| It is not directly in the README but there is a link in it
| to a getting started guide [0] which covers this. The user
| "zie" probably got it confused with the README.
|
| [0] https://stalw.art/docs/get-started/
| drdaeman wrote:
| Storage options: https://stalw.art/docs/get-
| started#supported-blob-stores On the first glance it seems
| that it's designed in a easily extensible way, so new
| storage options shouldn't be hard to add
| (https://github.com/stalwartlabs/mail-
| server/blob/main/crates...)
|
| Given it's all-in-one thing, MDA/LDA is integrated, to best
| of my (very shallow, just ~30 minutes haphazardly checking
| docs and source code) understanding there is no separate
| delivery agent program sitting in-between the components,
| it's all a single process, but it has places where you can
| hook arbitrary external filters/transformers
| (https://stalw.art/docs/smtp/inbound/data#content-filters)
| flipbrad wrote:
| Super exciting. This may become my selfhosting project for the
| year ahead...
| freedude wrote:
| It is new, but it would be interesting to see a list of customers
| using the system.
|
| One concern is that Email is designed to be highly modular in the
| UNIX sense. Does this eliminate some of this modularity? For,
| instance can I still use Dovecot for IMAP, POP3 if I want?
| xmichael909 wrote:
| Thanks, this looks great, I hope to see a turnkey linux VM of it
| one day!
| huslage wrote:
| Why is this not set up to filter spam out of the box? It's not
| really an optional thing at this point, and writing Sieve scripts
| that work well is not necessarily easy.
| Avamander wrote:
| Because it's kinda difficult to make the choice how it should
| be done, especially as it depends on scale.
|
| Though it's not that difficult to get a well-working rspamd
| setup.
| chewmieser wrote:
| It at least seems pretty easy to integrate with rspamd or
| spamassasain etc:
|
| https://stalw.art/docs/smtp/inbound/data/#spam-filtering
| candiddevmike wrote:
| Anyone know how this compares to Maddy (written in Go)? Seems
| like Stalwart has more features just from a cursory glance
| cuu508 wrote:
| Maddy (also) has a single maintainer. Development activity
| seems to be low, but the couple PRs I submitted (documentation
| fixes and cleanup) did get accepted.
| eddieroger wrote:
| Of note, this project also has one maintainer. It is cool to
| see individuals interested enough in mail to be putting
| effort in to it these days.
|
| > Stalwart Labs Ltd. is a very small team consisting of just
| one developer, who has been implementing email software in C
| since the mid-90s.
|
| https://github.com/stalwartlabs#team
| ttul wrote:
| The developer is a beast. This is a huge project to build
| from scratch. So much territory to cover.
| Avamander wrote:
| EDIT: Missed a section of the readme. Ignore the following.
|
| Maddy mentions actual security features (DMARC, MTA-STS, DANE,
| DNSSEC, DKIM) in its overview and compatibility with rspamd
| etc. (which is rather vital if you want to properly handle
| spam).
| singhrac wrote:
| Are we reading the same Github repo? All of those acronyms
| are in the second bullet.
| Avamander wrote:
| Ahh, I scrolled down to "Security" and missed that it was
| listed above.
| sdesol wrote:
| Here's some community information regarding Maddy and Stalwart
|
| https://devboard.gitsense.com/stalwartlabs?repos=imap-server...
|
| https://devboard.gitsense.com/foxcpp?repos=go-jmap,maddy,mai...
|
| Not sure if Stalwart recently got funding, but the number of
| participants shot up in the last week. Stalwart's popularity
| (stars/watch events) also shot up like crazy about 5 weeks ago.
|
| Note, I'm not indexing the code history for both projects right
| now (they are queued but they probably won't be indexed for
| another hour or two) so the community insights is incomplete.
|
| Full Disclosure: This is my tool
| Daril wrote:
| Congratulations! Very interesting project ! I have been running
| my own servers with ISPConfig with PostFix and Dovecot for many
| years, but this modern all-in-one solution seems better
| integrated and complete. As others have commented, rspamd
| integration would be a great addition, although I think using
| Proxmox Mail Gateway instead would give more control with more
| ease. Proxmox doesn't need any special integration. It works like
| a firewall for mail.
| geocrasher wrote:
| ISPConfig, unless they've changed it, is a very dangerous
| control panel. ISPConfig3, at least, runs the control panel as
| a vhost on the same Apache instance as the users, which is Very
| Bad. I actually wrote an article a few years ago that roasted
| it pretty hard. Try Virtualmin for a more robust, secure
| control panel.
| Daril wrote:
| Other important additions would be user mail encryption with
| GPG and synchronisation of multiple servers to build a fault
| tolerant service.
| maxpert wrote:
| Amazing I was just looking for a good mail server to configure
| for my demo. Which reminds me since you folks have mentioned
| LiteStream, have you tried Marmot
| (https://github.com/maxpert/marmot); I recently configured Isso
| with Marmot to scale it out horizontally
| (https://maxpert.github.io/marmot/demo). I am super curious what
| kind of write workload on a sub thousand people organization will
| have and if Marmot can help scale it horizontally without
| Foundation DB. I always find the the convenience of SQLite
| amazing.
| patchtopic wrote:
| can this handle multiple email domains on the same server?
|
| i.e. mary@domaina.com and mary@domainb.com as different email
| accounts?
| ocdtrekkie wrote:
| Fantastic to see JMAP support right from the start!
| gumby wrote:
| Are there any clients of note?
| ocdtrekkie wrote:
| This is the chicken and egg problem. It is far easier to
| write a good JMAP client than a good IMAP client, but we need
| servers and providers to support JMAP before it makes sense
| for app developers to write clients with JMAP.
| chewmieser wrote:
| Apparently Stalwart provides a rust library as well to
| interact with JMAP if anyone was looking to write a
| client...
|
| https://github.com/stalwartlabs/jmap-client
| heliostatic wrote:
| In the terminal, I use aerc, which added jmap support
| recently. Currently only available on master, but a new
| release should be out soon: https://git.sr.ht/~rjarry/aerc/co
| mmit/be0bfc1ae28b49be654662...
| mzs wrote:
| some comments from the folks responsible yesterday here:
| https://news.ycombinator.com/item?id=36757296
| uvesten wrote:
| Yay!
|
| I'm just about to set up a couple of new domains, and was
| procrastinating because email. This looks perfect, something new
| to play with and get email set-up done at the same time :)
| jvdvegt wrote:
| Looks nice! What kind of (server based) calendar could be used
| with this? Can e.g. Ldap be used for a shared/corporate address
| book?
| avhception wrote:
| Maybe https://radicale.org/ could fit the bill, CalDav as a
| calendar server and CardDav for contacts.
| nubinetwork wrote:
| https://news.ycombinator.com/item?id=32894429
| https://news.ycombinator.com/item?id=34992978
| https://news.ycombinator.com/item?id=35040256
| https://news.ycombinator.com/item?id=36757296
| mewmew07 wrote:
| what are those links for?
| djbusby wrote:
| Previous submissions of same project
| bozhark wrote:
| Reposts get relinked.
|
| I've always seen it as a good way to let people new to a
| topic see the full discussion that's already occurred. You
| get more insight
| blooalien wrote:
| My question is; Why did it get down-voted to oblivion,
| despite it being a normal practice on other posts?
| freedomben wrote:
| My guess is usually when I see it, there's at least a
| single word description with it, such as "dupe" or "see
| also:"
| LoganDark wrote:
| it can be read as "this has already been posted" instead
| of "prior art" when the latter isn't explicitly specified
| xoa wrote:
| Definitely exciting to see another very promising modern mail
| server option that seems to really be developing nicely. The
| major pain of self-hosting email for a long was ensuring
| delivery, but with SMTP relay services also getting quite slick
| that doesn't seem as much of a blocker anymore. Also means being
| able to clamp down even more tightly on the mail server at the
| network level as well as its own security since it only needs to
| talk out to the relay service and nothing else. In an ideal world
| there'd be a real solid secure DNS and in turn full e2ee email-
| like standard itself, and authentication could at least be one by
| certs in DNS.
|
| But in the meantime email still fills an important role, and
| perhaps enough layers of options will get us close. After the
| Gandi.net sellout most recently this feels like propitious timing
| to me, I really dislike the typical email service pricing models.
| Paying just for the domains and relay, both of which are trivial
| to swap around at will, and then otherwise having that on my own
| infrastructure certainly feels attractive to try firing up again
| at least for a handful of domains.
| neilv wrote:
| > _pain of self-hosting email for a long was ensuring delivery,
| but with SMTP relay services also getting quite slick that
| doesn 't seem as much of a blocker anymore_
|
| Has the definition of self-hosting email evolved to include
| using a third-party SMTP relay service?
|
| (It's always been a slightly fuzzy definition, and maybe the
| last time it shifted was when it included running on cloud
| servers/VPS rather computer hardware that you own.)
| EvanAnderson wrote:
| I used ISP SMTP servers for outbound relay back in the late
| 90s when installing on-prem email servers. It struck me as a
| fairly common thing then.
| aidenn0 wrote:
| Can you recommend any SMTP relay services? I've been interested
| in self-hosting for a while, but building reputation to get
| reliable delivery seems like a full-time job.
| Daril wrote:
| I have been running my own services for years, all self-
| hosted, with different VPS providers: A2Hosting, Digial Ocean
| and now Contabo. I use ISPConfig and host about 25 domains
| and 200 email addresses. You have to configure SPF, DKIM,
| DMARC, reverse PTR address, but nothing impossible.
| Encountered some problems with some providers blocking the IP
| range of my VPS provider (Microsoft in particular), but if
| you send an email and explain you have a new server
| configured form scratch and carefully configured, they add an
| exception for your IP. Check your IP addresses are not list
| in any black list and you are done. I use Rspamd and I had to
| add some domains to the whitelist ... There are many servers
| out there (even of medium sized companies, very badly
| configured) that don't even respect the minimum requirements
| like a public registered and valid server name for the smtp
| server. I avoid any external SMTP service ... they can read
| all your emails ... There are many ready-to-use solutions,
| such as mailcow, which simplify the management of the mail
| server.
| Daril wrote:
| Forgot : I configured the compression with LZ4 algorithm
| (it saves a lot of space) and encryption of the mails. The
| encryption uses a master key. It is possible to encrypt
| every single mail file using the password of the account,
| but if the user forget or lose (and it happens sometime)
| his own password all the mails are gone ... Another option
| to explore is to autmatically encrypt every mail with the
| GPG public key of the owner, again if he lose the GPG
| private key or the password, the mails are lost forever,
| but unfortunately I don't see any interest in this by my
| clients.
| EVa5I7bHFq9mnYK wrote:
| SMTP2GO has been working pretty well for me for the last few
| years, First 1000 emails/month are free.
| detourdog wrote:
| I think one just needs DKIM, SPF and DMARC. I had to go
| through that configuration change sometime after 2008. I have
| no idea if domain reputation counts. I certain get no special
| consideration from google and my domain might be older. I
| guess one needs an ssl cert for clients on iOS.
|
| panix.com hosts my start of authority for DNS and they
| probably have a product that might fit your needs.
| technothrasher wrote:
| I've tried a few of the free or low cost ones and have found
| direct from my mail server provides better deliverability in
| all cases. I'm not sure how well the more expensive ones
| would do, although one would hope it would be better.
| FuriouslyAdrift wrote:
| SMTP2GO is really really good...
| xoa wrote:
| As far as senders, from what I've seen AWS SES is still
| probably the basic go-to for the HN type, pay as you go
| pricing looks to be quite good in this context and most of us
| are familiar with navigating AWS. Without any major
| experience, it seems to work in my light kicking of the tires
| so far. When I tried Postmark more heavily like a year and a
| half ago it seemed truly excellent for a more full fat flat
| per month service, and that's what I'd planned to move to
| already following the legacy GSuite sunsetting. Unfortunately
| bad timing for me, they finally decided it was time to move
| on and sold last year to a marketing company [0], and since
| then there have been significant price increases, elimination
| of non-subs, and a few concerning events. I think they were
| the last of a big grouping of '09/10 email startups to do the
| acqui-exit. Doesn't mean they won't still work and aren't
| mostly still fine, but something of note. Mailgun, Sendgrid
| and so on are all farther along the post-acquisition curve
| there. Last month there was a new one announced on HN called
| Resend [1] which is more development oriented but still of
| interest.
|
| On pricing/ROI: most of the paid tiers for monthly plans seem
| to start $15-20/month now though with free tiers to
| experiment with first. I think self-hosting tends to pay for
| itself best if you fall into certain now neglected niches and
| have existing infra, or else are willing to pay some premium
| ideologically. Most email services now tend to squish a bunch
| of the actual underlying stuff into a specific payment model:
| mailbox (email address) is 1:1 with a person, and also covers
| storage, while people don't really think about sending
| numbers. Whereas underlying storage is actually dirt cheap
| particularly in the context of email, mailboxes are
| effectively free, but sending emails costs. So for example I
| have a bunch of domains and lots of email accounts at them, I
| was always in the habit of making heavy use of separate
| mailboxes for basic utility usage like a server sending a
| status alert (and that also means the server email address
| can be restricted and not have credentials fro my personal or
| work email etc). Low volume, tons of mailboxes, occasional
| big messages with logs and such is an absolutely _awful_ fit
| for most mail services and getting worse. I also have
| reasonably solid self-hosting infrastructure already that I
| 've amortized for other things, so at this point essentially
| adding another VM is quite efficient. For someone who falls
| into the general bucket, just going somewhwre like Fastmail
| or even GSuite or the like would almost certainly make more
| sense. $15/month would buy 3 of Fastmail's standard "users"
| (ie, mailboxes/different addresses). But I have way more than
| that, lots of which only send a handful of emails. Doing that
| with Fastmail/ProtonMail/Gmail/etc type pricing would be
| hundreds of dollars including $5/month accounts that receive
| nothing and might not send more than a handful of emails per
| year.
|
| Anyway, that's my thinking and what I've been experimenting
| with so far. But ultimately part of the point/value of it all
| is that on the "difficulty of change" scale, moving to a new
| email address entirely is the worst though cheapest, owning
| your own domain and being able to point at a new email
| provider then is vastly easier but costs domain/year (this
| mid level is probably best for most people), and having
| merely to change relays on a server costs the most but is the
| most transparent. So trying to get out of the habit of
| thinking of these things as needing to be long term
| relationships. If a relay service isn't working for me with
| self-host or someone offers better I'll just move. I'll
| probably keep one or two addresses traditional too as
| fallbacks.
|
| ----
|
| 0: https://news.ycombinator.com/item?id=31247296
|
| 1: https://news.ycombinator.com/item?id=36309120
| bityard wrote:
| I've looked into this extensively and there are essentially
| four options:
|
| 1. Self-host on a reputable VPS provider. Deliverability is
| usually not a problem if your provider actually takes action
| against spammers on their network. They tend to work pretty
| hard to keep up the reputation of their IP space. I have
| self-hosted my own email for well over a decade on providers
| like these with no serious issues. It's not generally too
| hard to test the reputation of an IP/domain before putting it
| into production.
|
| 2. You can use a dedicated SMTP relay service, but these are
| usually quite expensive and their customers tend to be bulk
| email senders for blasting out marketing wank. You probably
| don't want your domain to be associated with these anyway.
|
| 3. Another option is to buy a full-fledged email account from
| any of the common providers and just use their SMTP servers.
| (Make sure to set your SPF records appropriately.) Not
| expensive but typically far from free. And it seems silly to
| self-host your mail if you're literally paying a company to
| provide the same service...
|
| 4. This is less "self-hosted" but the cheapest and most
| reliable way to get your email out is to sign up for a cloud
| account at any of the major providers and just use their SMTP
| relays. Most will allow you to send from outside their
| network, after proper authentication is set up. Unless your
| monthly email volume is north of 4 figures, it will likely be
| free or cost pennies per month.
|
| Don't forget that amongst email providers reputation is a
| thing, and if you go out and purchase a domain, it may be
| some time before you can actually use it. Lots of providers
| specifically penalize domains younger than X months old, they
| will "graylist" IPs that haven't talked to them before, and a
| few will flat-out blacklist entire gTLDs known to be heavily
| used by spammers.
| aidenn0 wrote:
| > and a few will flat-out blacklist entire gTLDs known to
| be heavily used by spammers.
|
| This explains poor delivery for emails from a .xyz domain I
| have, even hosted with an e-mail provider.
| EvanAnderson wrote:
| > 2. You can use a dedicated SMTP relay service, but these
| are usually quite expensive ...
|
| I don't typically give endorsements, but I've been using
| DuoCircle.com since back when they were part of "dyn.com"
| and I've been very pleased. Apparently they have a free
| tier, but their current pricing is very reasonable to my
| eye. (I'm on an old annual plan that doesn't appear to be
| offered anymore...)
|
| https://www.duocircle.com/email/outbound-smtp
|
| Deliverability through them has been very good over the
| years.
| tomatocracy wrote:
| I've been doing a combination of 1 (good reputation IP) and
| 4 (using AWS in my case). I switched VPS provider a couple
| of years ago - before that I had deliverability issues to
| anyone using MS 365 (despite jumping through Microsoft's
| various hoops etc) - since then everything has been fine.
|
| I also have my outbound SMTP server set to send via AWS if
| my email includes a particular custom header (which it also
| then strips out before forwarding on) - which means for
| domains I think might have deliverability issues I can deal
| with this without needing to make a huge effort.
| mindslight wrote:
| Also, email reputation and deliverability only applies to
| _sending_ email. While many of the advantages of self
| hosting only require _receiving_ email - eg better control
| over your root of trust for account auth, different address
| per account to avoid your email address being used as a
| join key for cross-company surveillance, etc. Self hosting
| doesn 't need to be an "all or nothing" affair. Set up your
| domain with whatever server setup you think you'd like and
| start switching account identities over to that. Then only
| after you've gotten comfortable running it and tested
| deliverability, start switching over your personal
| correspondence.
| gmzamz wrote:
| Utilizing fancy gTLDs can still prevent you from
| receiving mail. Not due to a decision to explicitly block
| it but because it doesn't match whatever regular
| expression they use to validate. Notably, .email fails
| consistently due to it being >3 characters. I tried to
| convert to using first@last.email and there is a
| significant minority of sites that didn't allow it.
| mindslight wrote:
| Good point! That's still kind of orthogonal to
| deliverability though. In fact in line with my point,
| you're better off finding this out before you start
| transferring personal correspondence to that domain.
|
| The only similar problem I've experienced is sometimes
| companies will get uppity if you put their company name
| in the email address you give them. But it's easy enough
| to just make up a difference nonce for those cases (or
| start your scheme based on opaque nonces for everyone).
| I'm still waiting for the other shoe to drop and
| surveillance companies to start discriminating against
| non-surveillance-company email addresses the way they do
| against VOIP phone numbers.
| vel0city wrote:
| I've been using gTLDs for an email for several years now
| (about the time gTLDs came out). It was really rough
| going for a while but these last couple of years it's
| been rare for me to have an issue.
| lyu07282 wrote:
| Used mailroute for years, works great
|
| https://mailroute.net/
| chepurko wrote:
| anydomain.net/anymxrelay/
| carstentr wrote:
| Isn't the major pain
|
| - 24x7 running - spam - security
|
| rather then proof-of-concept?
| ttul wrote:
| Those are definitely "step two" once you have managed to get
| the relevant services running. Fortunately, spam and basic
| phishing filtering isn't too hard at small scale. You can get
| free access to very high quality blocklists for non-commercial
| use and you can use rspamd's fuzzy hash API and various
| phishing URL data sources for free as well.
|
| The really hard thing is to self host outbound email delivery.
| You almost have to use a relay service to get mail delivered
| these days. Most IPs at cheap hosting services are in a bad
| neighborhood and will be treated poorly by association. On the
| other hand, most transactional email services have a generous
| free tier that would work for a lot of self-hosted setups.
|
| Relaying through MailChannels is free with no volume limits if
| you do it via Cloudflare Workers. Would be nice to see someone
| merge that with this project as an option.
| darkclouds wrote:
| Looks interesting, but something I always look for, is what it
| doesnt do and then try to find out why, partly because I'm not up
| to date with all the RFC's, so I couldnt tell straight away if
| those RFC's are the latest, proposed or deprecated.
| ape4 wrote:
| It could also include POP3 and LMTP (local mail transport
| protocol)
| e12e wrote:
| https://stalw.art/docs/smtp/outbound/routing#lmtp-delivery
|
| Or did you mean incoming?
| ape4 wrote:
| That's what I meant - didn't see it. Thx
| Nux wrote:
| Very nice! Would love some docs on anti-spam though.
| mewmew07 wrote:
| I think some of the entries in the Inbound section will inform
| how to deal with spam, I saw some references to spam assassin
| there.
|
| https://stalw.art/docs/category/inbound
| tankenmate wrote:
| In particular the DATA (or BDAT) stage configuration; it runs
| a command and send the headers / data to stdin and receives
| back the modified message from stdout.
|
| https://stalw.art/docs/smtp/inbound/data/#content-filters
___________________________________________________________________
(page generated 2023-07-18 23:01 UTC)