[HN Gopher] Deploy a Gmail-like email server in 30 (ish) minutes
___________________________________________________________________
Deploy a Gmail-like email server in 30 (ish) minutes
Author : acallaghan
Score : 134 points
Date : 2021-12-26 11:44 UTC (11 hours ago)
(HTM) web link (andycallaghan.com)
(TXT) w3m dump (andycallaghan.com)
| imwillofficial wrote:
| This absolutely nothing like gmail. The great parts of gmail
| can't be replicated at home anymore, unfortunately.
| freediver wrote:
| > The great parts of gmail can't be replicated at home anymore,
| unfortunately.
|
| Curious, what are those in your opinion?
| [deleted]
| imwillofficial wrote:
| Well interface is an easy one. All the available guis for
| email work like steaming garbage. Another few are mail
| deliverability SSO, and security. Back in my day,
| deliverability wasn't an issue, security wasn't an ever
| looming specter, and feature parity was mostly there.
|
| I ended up throwing in the towel with Hey mail, and have
| really found love for email again.
| oliwarner wrote:
| It opens with some fair points why somebody might not, but the
| main reason --perhaps on par with the constant security
| headaches-- isn't there. You will _never_ block spam as
| effectively as Google, Microsoft, FastMail, etc.
|
| They see [for lack of a better word] infinite times more spam and
| ham than you'll ever be able to train your little Spam Assassin
| database, and millions of users to sort through it.
|
| Email without spam control is not a pleasant experience.
| acallaghan wrote:
| This is a valid point - The setup I used integrates
| SpamAssassin for the basic setup which would get a fair number.
| There's a way to learn from listening to when you tag a message
| as spam that I didn't cover in my article. But yeah, you'd
| never do as well as spam protection as the big companies, but
| is that really such a deal breaker? Maybe if more people like
| us ran their own infra for email then we'd have better and
| stronger tools for spam protection?
| throwaway984393 wrote:
| You know, it's possible to build a house by yourself in about a
| day or two with no knowledge of carpentry. But I wouldn't want to
| live in it :)
| danlugo92 wrote:
| Zoho has a 1 dollar a year plan for using with your own domain.
| pigbearpig wrote:
| I'll be interested in the one month update post.
| jeroenhd wrote:
| You can't replicate Gmail, but with Mailcow I've gotta say the
| whole process is pretty seamless. You can throw it onto a 5 euro
| VPS at Contabo, run docker-compose up and be done. Just regularly
| run the update and backup scripts to make sure you're up to date
| but that's it, really.
|
| Exchange ActiveSync, multi domain + multi aliases with catchalls,
| (temporary) aliases, mail delivery rules, TLS requirements, you
| name it, all configurable in the web UI. There's even a built in
| DNS checking tool to verify that all the necessary records are
| set up right.
| mthld wrote:
| I concur, a very well crafted and maintained project.
| illuminated wrote:
| I'm using mailcow for year and a half now, moved to it from
| Kolab. It's really great and painless to manage it. The only
| thing missing for me is the LDAP auth, something I got used to
| over the years with Kolab. The OP's solutions has built-in LDAP
| auth, so I'll give it a try.
| acallaghan wrote:
| I'll take a look into this ta - the only thing really lacking
| in this setup is a UI for webmail or admin
| bsd44 wrote:
| I had to manage email infrastructure for years as part of my job
| and I really don't see how running your own email server can be a
| good idea for anyone. Setting it up superficially might be a
| quick and easy task but maintaining it stable takes hell of a lot
| of effort. I seriously cringe every time I see this type of guide
| and articles, it just makes me think that people who write them
| have zero experience running a mail server and have no idea what
| it takes to set up one that is secure and stable.
|
| For majority of people best middle ground is to buy a cheap
| domain and a cheap cPanel/web hosting and just use that to host
| emails. You'll be done in 5min, it will cost you a cup of coffee
| and you won't have the headache maintaining anything other than
| passwords.
| lgrapenthin wrote:
| I'm using mailu.io with docker and haven't had any trouble for
| two years now. Cost me 1-2d in total.
| johnklos wrote:
| This is nice, but even though I've administered email servers for
| a quarter of a century, I haven't got the foggiest clue what
| makes an email server "Gmail-like". What does "Gmail server"
| mean?
|
| I would think, if anything, that what Gmail has that typical
| email servers do not is somewhat decent webmail, but that can't
| be it because webmail isn't even mentioned.
|
| Or is this another one of those instances where people use
| "Linux" to refer to all things Unix? I genuinely would like to
| know.
| jahlove wrote:
| Seems like a way for the author to get clicks.
| acallaghan wrote:
| I was referring to Archive/All mail working, instead of just
| deleting all email, I should have been clearer I think - I
| don't use the web UI so don't need a full replacement for that,
| even though it might be helpful
| Tepix wrote:
| Now you have your own mail server. Great! But if you don't know
| how it works or if you don't have something that will help you
| maintain it, sooner or later it will break.
| acallaghan wrote:
| Isn't that the same with all things tech? Why try anything new
| at all without first getting a degree in it?
|
| I might get things wrong, so be it - I'll use it to learn and
| be better next time.
| 15characterslon wrote:
| > It was easier than I thought to create a mail server that works
| as well as Gmail's
|
| No it isn't and no you didn't.
|
| The article doesn't even cover basic stuff like email rules and
| spam filtering (incl. tuning and spam learning). It doesn't "look
| after itself" like the author wanted (article doesn't mention any
| update strategy). The author acknowledges that email servers are
| "open to attack" but this setup doesn't seem to include any
| security improvements over traditional setups. In fact,
| maintaining this looks harder due to the amount of custom scripts
| and lack of good documentation.
|
| And of course it doesn't cover any of the things that actually
| make Gmail special like labels, having a consistent set of apps
| for web and mobile, push notifications (esp. on iOS), really good
| spam filtering, really good search (incl. OCR for attachments),
| high availability, image proxying, smart suggestions, datacenter
| security, Google doing code and infrastructure audits all the
| time, using reproducible builds, ...
|
| It's great that the author is experimenting and learning, but if
| I had any private data hosted by the author, I would be worried
| now.
| bluedino wrote:
| Gmails spam filtering isn't exactly a high bar
| GordonS wrote:
| Seriously? I actually think gmail's spam filtering is
| _brilliant_ - I probably average less than a single spam
| email a year that it doesn 't catch.
|
| Contrast that with every corporate email spam filter I've
| ever been subject to, which vary from "shit" to "OK", and
| Gmail is completely in another league.
| [deleted]
| xhkkffbf wrote:
| My problem with Gmail is the false positives. (Or is it
| negatives?) They routinely send too much to the spam box
| and others tell me they have the same experience.
|
| The worst is when they take email from one Google hosted
| domain and send it to spam in another Google hosted domain,
| even though the email didn't leave their network at all.
|
| Still, I agree that the overall level is pretty good and
| hard to duplicate.
| jeffbee wrote:
| > even though the email didn't leave their network at
| all.
|
| FYI gmail treats all of its children equally. Mail from
| one Google user to another is subject to the exact same
| treatment as mail received via SMTP (and, indeed, Gmail
| sends traffic to itself over SMTP). If you study the
| headers of messages in Gmail, you can form a picture of
| how they allocate and use the virtual IPs.
| meibo wrote:
| Have not had to deal with spam on my personal Gmail address
| in the 10 years I've been using it, and I'm having the same
| experience running a big Workspace organization. Their
| spam/fishing detection is making my job a lot easier.
| yokoprime wrote:
| Gmails spam filtering is still the best I've seen from the
| major e-mail providers, so I disagree with your assessment.
| rahimnathwani wrote:
| Based on my experience running mail servers in the past (both
| personal and corporate), I'd say you're wrong.
| entropie wrote:
| I second this.
|
| Gmail spam filtering is top notch. I just stopped to care
| to obfuscate or hide my email adress (which I use since the
| beta invitation program of gmail) and I can count the spam
| I actually read in a year with one hand.
| rahimnathwani wrote:
| "I can count the spam I actually read in a year with one
| hand."
|
| This is partly because Gmail is good at classifying
| emails as spam/ham.
|
| But it's partly because it's more tolerant of false
| positives (ham sent to the spam folder) than you or I
| would be if we were tweaking our own spam filter.
|
| I occasionally check my spam folder, and there are
| usually some mailing list emails that I don't care about,
| but which I did actually subscribe to, and would have
| wanted to reach my inbox.
| denton-scratch wrote:
| > Gmail is good at classifying emails as spam/ham.
|
| I wish they'd apply that discrimination to their SMTP
| output.
| nulbyte wrote:
| I also have serious doubts about Google's spam fighting.
| While they catch a lot of spam in the spam folder, they are
| simultaneously overzealous, catching normal emails that I
| receive and read regularly, and underprepared, as if putting
| myusername@aol.com and sending the email to Gmail servers
| isn't totally obvious spam.
| throw0101a wrote:
| > _that actually make Gmail special like labels_
|
| I hate labels.
|
| At $WORK we use Gmail and I get a lot of automated stuff (cron,
| etc). I want these types of message to go into _folders_. I don
| 't want it in my "all" / archive area because they just clutter
| up searching for other things.
|
| Perhaps labels work for other people / general public, but for
| me 'traditional' folders is how things work best.
| petre wrote:
| I second that, my worst problem is spam filtering. The rest I
| have set up, except DKIM and DMARC which are not worth
| bothering with.
| vbezhenar wrote:
| In my experience spamassassin works wonderfully. There are
| some few false negatives (1-2 mails per week), but I did not
| have a single false positive which is very important for me.
| For example Google is much worse in that regard, which forces
| me to check spam every few days to ensure no legitimate mail
| ends up in spam, so it's like no spam filtering at all, I
| have to read it all anyway.
| LinuxBender wrote:
| For me personally, one of the most effective means of
| knocking out the first 95% of spam was using the S25R regex
| methodology [1] created by Asami Hideo which seems to keep
| the load on SpamAssassin and ClamAV really low. I've had to
| adjust the regex rules over the years a little bit but it's
| really low maintenance for my setup. There are also lists of
| IP addresses and networks you can block that are known to be
| malicious which also reduces the load and log volume. [2]
|
| [1] - http://www.gabacho-net.jp/en/anti-spam/anti-spam-
| system.html [No HTTPS, Sorry]
|
| [2] - https://github.com/firehol/blocklist-ipsets.git
| teitoklien wrote:
| Thanks a lot for this ! I'll try them out too
| acallaghan wrote:
| > if I had any private data hosted by the author, I would be
| worried now.
|
| Merry Christmas to you as well.
|
| Such negativity for just showing something I knocked up in half
| an hour. - something that I thought might be helpful, with
| experiences on how to make it more Gmail like.
|
| Attacking the writing is fine, but insinuating my custody of
| private data is at question is pretty shitty
| linza wrote:
| > Such negativity for just showing something I knocked up in
| half an hour. - something that I thought might be helpful,
| with experiences on how to make it more Gmail like.
|
| GP's feedback is direct but quite right imo. I trust the
| author had only best intentions in mind but "Knocking
| something out in half an hour" and sharing, but good privacy
| and security engineering requires probably much more time.
| Quite frankly, the wording of the article can be insulting
| even for folks that are working on that problem
| professionally for several years.
|
| Were it presented differently, it would get different
| feedback I'm sure. More like "hey HN, i made the first three
| steps what would be next?" -- i.e. efforts towards trying to
| understand the problem better.
| lazyweb wrote:
| In my experience [1] running a small private mail server is very
| much doable, and a good learning experience.
|
| [1] https://jschumacher.info/2021/05/running-a-private-mail-
| serv...
| boplicity wrote:
| You can deploy an email server almost instantly by getting a
| cheap vps with cPanel on it -- with everything you needed already
| configured, including spam filtering, security, etc.
| mysterydip wrote:
| Let's say I'm tired of self-hosting my email (for all the reasons
| previously mentioned by others). What's a good option of privacy-
| conscious provider I can move my domains to?
| pandemicsoul wrote:
| I think ProtonMail (https://protonmail.com/) is the only one
| that comes close to being "privacy conscious," mainly because
| it's their entire focus. Any other non-major, non-free provider
| might be an alternative but they're probably not as focused on
| being proactive about the privacy element?
| geoah wrote:
| > The blog article is the setup to make Docker Mailserver act
| like a Gmail server.
|
| I'm not sure what a Gmail server is. I was expecting this to
| include a web ui, admin ui, and the things that actually make
| Gmail hard to move away from. The docker-mailserver container
| doesn't seem to include something like that or am I just not
| seeing it?
|
| The killer feature for Gmail has always been the spam protection
| and the fact that the emails I sent actually get delivered.
| remram wrote:
| Does it have labels and full-text search, at least?
|
| edit: does the described setup include a UI at all?
| [deleted]
| focom wrote:
| I think those projects:
|
| - https://mailinabox.email/
|
| - https://github.com/modoboa/modoboa
|
| are better replacement. They are battery included with a webUI
| berkes wrote:
| There is also mailcow.
|
| I've been using mailinabox for years now, and it is really good
| in the sense that it gets out of my way.
|
| I've included it in my ansible setup, so the basis, distro, os
| updates, firewalls, backups are cositent with my other servers.
|
| That took some effort: mailinabox is opnionated (and that is
| good. It is the main reason it works well and is secure), which
| can be a bit confronting if your opninions are very different.
| siraben wrote:
| I recently got into running my own mail server on my NixOS
| instance using[0]. The server has a total of 1.5 GB RAM and 10 GB
| of disk space, but it was sufficient to get 10/10 on mail
| tester[1]. Here's my 12 line mailserver config[2]. It was quite
| liberating once everything was set up, because then you know you
| are in full control of your communications.
|
| It was more annoying to set up DNS than the mailserver itself, is
| there a good way to automate that as well?
|
| [0] https://gitlab.com/simple-nixos-mailserver/nixos-mailserver
|
| [1] https://www.mail-tester.com/
|
| [2]
| https://github.com/siraben/dotfiles/blob/master/server/mails...
| greggman3 wrote:
| According to many of the other posts here, running your own
| email server does not put you in"full control of your
| communications".
| denton-scratch wrote:
| Once your outbound mail leaves your server, it's destiny is
| out of your control. And you don't have full transparency
| into the processes your inbound mail has been subjected to.
| That's in the nature of email. It's orthogonal to running
| your own server.
| siraben wrote:
| The reason that motivated me to run a mailserver is best
| described as an artisanal choice, not a practical one[0].
| After all, I'm running this on a constrained system and
| certainly don't have ambitious to scale this to corporate
| scales.
|
| Like some other commentors here the point is mostly to learn
| and have something semi-useful at the same time (I've had
| some pleasant exchanges over my own email already.)
|
| [0] https://utcc.utoronto.ca/~cks/space/blog/sysadmin/EmailSe
| rve...
| carlhjerpe wrote:
| Terraform has modules for many popular DNS providers. But
| that's another tool with its own state to maintain. I've used
| terraform for both Route53 and Cloudflare.
|
| siraben.dev doesn't seem to be registered anywhere so I don't
| know if there's one for your provider.
| siraben wrote:
| It's registered via Google DNS. Looks like terraform can
| manage the records for it. Thanks!
| ohiovr wrote:
| Mail servers are all pretty much blacklisted by IP unless you
| work some kind of deal with someone somewhere.
| nulbyte wrote:
| Not really. By definition, a blacklist is a list of things to
| block, and things not found on the list get a pass. Most use IP
| address or domain. Unless someone has reported something, they
| don't get added. The only real exception I can think of is
| "residential IP addresses, " as some blacklists will try to
| keep up with multiple ISPs' residential assignments to block
| them. Even then, they still have to he added to the list to
| block. Most have forms easily accessible to request review or
| unblock. Even Spamhaus has a very easy process, and I find them
| to be rather more aggressive then the others.
| remram wrote:
| What's a good way to monitor a self-hosted mail server? I can
| easily set up uptimerobot.com or similar and get alerted if my
| website fails, whether it's a DNS, IP, firewall, nginx, TLS^,
| application, or database issue. Is there a way to check my mail
| server and get alerted if it is not accepting emails for some
| reason?
|
| ^: uptimerobot.com specifically doesn't warn you if your site
| works but is using an expired certificate, be careful there
| jeffbee wrote:
| I use and recommend the prometheus blackbox_exporter. You can
| configure a TCP connectivity check with TLS validation and an
| SMTP "expect" transcript. If you ran this in the cloud from
| multiple probing regions/clouds you'd have a monitoring scheme
| on par with what Google uses to monitor Gmail.
| avian wrote:
| It's not the initial setup. It's the maintenance over the years
| that really makes you question the universe, life and the
| decision to host your own mail. When you can't send that
| important mail because $big_provider is blocking you. When
| someone decides to run a persistent brute force attack from a
| botnet, eating up 100% of your CPU and you have no meaningful
| ways to block it. When you need to explain to people why they
| can't send you that 100 MB video attachment which they sent to
| other people just fine but only your address is bouncing and why
| don't you fix your email already. When you need to research,
| understand and implement standard X pushed by $big_provider
| because otherwise things will stop working and you have a ton of
| better things to do. When you get a random alert that email
| volume is too large and you panic because someone hacked your
| server and is probably sending spam but realize it was just
| triggered by a huge kernel patch series sent by someone on a
| mailing list. When a zero-day CVE for your mail software just hit
| the top of the HN and the fix is not in your distro yet and you
| scramble to find a workaround. When a bunch of weird log lines
| appear in your mail logs and you don't understand where they are
| coming from and they seem benign but can't lose the feeling that
| someone is trying something malicious. When you constantly fear
| that you'll lose that good IP and domain reputation and one day
| wake up with half of the internet blocking you.
| holri wrote:
| Well, then I must do something wrong. I run my personal email
| server based on Debian stable on my own SBC hardware in a colo
| for a couple of years now . Manual maintenance is just upgrade
| to the next stable Debian version every 3 years or so. I had
| zero of the problems you describe.
| sildur wrote:
| Yeah, me too. My life was basically set up and forget. Except
| for that time I upgraded my instance and the upgrade came
| with an unexpected and unwanted IP change. Unfortunately it
| seemed like the new IP has been used for spam and I had to
| spend some time clearing my IP status. But eventually
| everyone understood and my IP is clear and shiny now.
| acallaghan wrote:
| Just stick with Gmail then @avian, it's not for everyone. I
| think that you might be constantly think of worst cases, where
| as most of the time it'll just be fine? And even I it isn't,
| then I'm learning more about the web and email. When did people
| stop doing something because it was challenging?
| zigzag312 wrote:
| Unfortunate side effect of avoiding self-hosted email is that
| email is becoming centralized service.
| micromacrofoot wrote:
| if you've ever tried running your own email service you'd
| realize it might as well be already
| hdjjhhvvhga wrote:
| OK I run several mail servers and I do have problems but of
| another kind that you describe. But this one is a bit
| ridiculous:
|
| > When you need to explain to people why they can't send you
| that 100 MB video attachment which they sent to other people
| just fine but only your address is bouncing and why don't you
| fix your email already.
|
| The maximum attachment size for Gmail is still conservative 25
| MB and they basically dictate what is currently to be expected
| in terms of attachments going through.
| cmeacham98 wrote:
| https://support.google.com/a/answer/9050120
|
| According to this page the incoming limit "depends on several
| factors" and can be as high as 150MB.
| hdjjhhvvhga wrote:
| OK, so I tried to test it in practice, starting from 120 MB
| down to 25 MB. The largest that went through had 35 MB; the
| larger ones bounced with:
|
| > Remote Server returned '552-5.2.3 Your message exceeded
| Google's message size limits. Please visit 552-5.2.3
| https://support.google.com/mail/?p=MaxSizeError to view our
| size 552 5.2.3 guidelines.
| AshamedCaptain wrote:
| You can easily search around that the Gmail limit for
| personal users is 50MiB, recently enlarged from 25 MiB. As
| the other poster says, even between Gmail accounts I still
| have trouble sending files larger than 30MiB.
| txdv wrote:
| i get sweaty hands from reading this
| emptybottle wrote:
| These concerns are common across virtually any internet facing
| service.
|
| Yes, if you choose to run a service it will need to be
| maintained, and occasional issues will come up.
| harikb wrote:
| No, email is an especially brutal service. I had taken this
| path for 10+ years.
|
| If my custom media server or private photo site setup fails,
| it is not a big deal. But if I can't login to a shopping site
| or my family can't checkin to a flight because the two-factor
| auth email disappeared in to thin air, I am the "horrible IT
| person" who spoiled Christmas - end of story.
| denton-scratch wrote:
| In my experience, the brutality grows rapidly with the
| number of users. If it's just for you, and you have at
| least one alternative account, then it's not brutal at all.
|
| A mailserver administrator is a sysadmin. Being a sysadmin
| is on the face of it unrewarding - nobody pats you on the
| back when everything is working normally. They only call
| you when there's a problem (and it's usually urgent, and
| sometimes critical, and they'd like to know who to blame).
| So if you run a mailserver with users, it starts to become
| a people job, and things begin to matter.
|
| People certainly rely on email to a greater extent than
| other services - even mobile. I can order goods online
| without surrendering my mobile number, but I always have to
| provide an email address before I can complete the order.
|
| But I've still enjoyed administering mailservers. Perhaps I
| liked the slight paranoia induced by trying to reconcile
| the boss's demands for functionality with his demands for
| security.
| everforward wrote:
| What makes it particularly brutal, to me, is that the
| failures are typically silent. If GMail starts spamboxing
| my emails, I don't get feedback about that. If I'm not
| getting emails, I don't get feedback about that.
|
| You can script checking those, but you'll have to re-
| implement it for each provider. And there are some that you
| can't; if I'm applying for a job, they're not going to give
| me an email account to test whether my stuff is
| deliverable.
|
| HTTP services are absurdly easy to monitor in comparison.
| micromacrofoot wrote:
| I've tried testing gmail and it's very difficult to even
| get consistent results
| yokoprime wrote:
| This 100%, e-mail is too critical and is not core to what
| my skills are, so I outsource it, even if I think it would
| be fun setting up initially.
| johnklos wrote:
| Seriously?
|
| If other email providers are blocking you, smarthost through an
| email provider.
|
| If you're getting brute forced, learn how to set up and run
| blocklistd or fail2ban.
|
| Not getting 100 meg attachments is an issue that other email
| providers have, not people who run their own servers. If your
| server doesn't have any free disk space, that's on you. If it
| does, then set confMAX_MESSAGE_SIZE to whatever you want.
|
| If by "standard X" you're talking about SPF or DKIM, there are
| lots of tutorials.
|
| If your email software is vulnerable because of issues with
| your distro, you're doing things wrong.
|
| The point is if you can't, don't. If you don't want to think
| about issues like these, then you shouldn't be running servers,
| anyway, so you're definitely not the target audience.
|
| If you can, then these things aren't issues.
| yokoprime wrote:
| I get it, if you're genuinely interested in hosting email,
| it's fairly easy... but those who aren't will lose interest
| after a couple of major setbacks and that's that. E-mail is,
| whether you like it or not, pretty critical, so it's not the
| best place for hobbyists to start.
| cookiengineer wrote:
| > fail2ban
|
| As a sidenote: The two RCEs this year were enough for me to
| judge the quality of this software.
|
| If a whois entry of the attacker's IP/domain can RCE your
| intrusion blocking software, I mean...really?
| shmoogy wrote:
| Isn't the big/only issue regarding getting into inbox? I
| would imagine if warming / domain warming would be
| ridiculously difficult in modern times.
| rasengan wrote:
| That's true, but to do it securely isn't as simple as one
| would like to believe. Setting up "DKIM/SPF/etc." simply
| provides auth/verify security on send, but I'm more
| specifically referring to the receiver side security.
|
| If you're really properly securing your mail server, it would
| likely be isolated behind a firewall and only have a LAN ip
| of some kind and utilize UUCP for transport to another LAN
| machine that does not have WAN access, and then, only allow
| POP3/IMAP access to machines in the LAN or connected to the
| LAN via VPN tunnel. Finally, you would want to setup a backup
| system of some kind for this machine to periodically backup
| via rsync when the inotify/fswatch file modification
| triggers.
|
| Next, you'd have a separate SMTP machine. For things like
| critical deliverability, you can't rely on SMTP to 'retry'
| albeit it's how they are supposed to act, so it would make
| sense to have multiple SMTP machines across multiple
| different backbones in different physical locations with
| backup power and the like with different MX priorities set.
|
| The initial configuration and running a mail server are
| incredibly easy.
|
| It's running it securely that increases the difficulty on
| order of magnitude (because you essentially have to setup a
| proper security protocol across multiple machines (a network)
| - defense in depth).
|
| That said, it's easily doable if you're already running
| complex infrastructure. Hopefully, you're getting paid for
| your time and costs for doing so.
|
| If not, then I hope you need to rely upon the protection of
| needing a home-invasion warrant vs a simple-subpoena since a
| machine at your home can't really just be 'subpoenad' while a
| machine at some datacenter business can. This of course
| assumes you're even running the machine at home because if
| you're doing all this on some VM the value of doing so
| diminishes ever so quickly.
|
| EDIT: Just to be clear, you can't simply rely on fail2ban and
| some other on-machine script / snort / daemon / kernel
| feature to protect you. There are bugs in software/systems
| and 0days are very real (as well as the market places for
| them).
| TedDoesntTalk wrote:
| > If other email providers are blocking you, smarthost
| through an email provider.
|
| Can someone please explain what this means?
| e12e wrote:
| It means no longer hosting your own server for sending
| email, but rather use eg smtp.gmail.com to send outgoing
| mail. It would mean allowing Gmail.com to send email from
| your domain via SPF, thus allowing all with a Gmail account
| to spoof your from domain/header.
|
| Ed: as sibling correctly notes, "smart hosting"
| specifically referes to setting your smtp server to relay
| via another (eg: Gmail) - eg exim or postfix allow setting
| a "smart host" so that rather than looking up the receiving
| smtp server, all mail is shunted to the smarthost to figure
| out.
|
| Added: you might get good delivery to Gmail using Gmail
| smarthost (requiring you to have a Gmail account) - but you
| might need to use outlook.com as a smarthost to get good
| delivery to o365 accounts... Etc.
| easrng wrote:
| Google requires that you verify ownership of a domain
| before you send emails as it with Gmail.
| e12e wrote:
| Ah, indeed it does.
| DarylZero wrote:
| They also require a paid account for this now. Though, I
| have one grandfathered in from like 15 years ago.
| xanaxagoras wrote:
| Your SMTP server uses an authenticated relay through (let's
| say) sendgrid that you pay for. Sendgrid delivers your mail
| using their reputation.
| john2010 wrote:
| > If you're getting brute forced, learn how to set up and run
| blocklistd or fail2ban.
|
| What about updating OS/packages/CVE when on holiday? Note
| that many CVEs are usually sent only to top-tier providers.
| tuldia wrote:
| charcircuit wrote:
| All of these don't seem that out the ordinary
| tuldia wrote:
| If that was the case, we wouldn't even plug anything on the
| internet, ffs.
| endymi0n wrote:
| It sounds less like FUD than a change in perspective.
|
| When I was in my twenties, I would have empathized with your
| point. I used to host my own web servers, but back then, my
| main priorities were curiosity, privacy and independence.
|
| Not just that, I opened accounts for friends and family.
|
| A decade later, I made all my hosting someone elses' problem,
| because I had different priorities.
|
| There's nothing like the sound of a friend shouting in your
| ear because he trusted you with his mail address and he's
| running into weird errors. Or trying to get an important
| email delivered after a 10h crunch shift when you just want
| to bring your kids to bed instead.
|
| I'm thankful for all those learnings, but nowadays, I'm old
| enough to just want mail to frickin work, that's why Google
| does it for me on a custom domain.
| organsnyder wrote:
| As I got further into my thirties, I became much more aware
| of the concept of opportunity cost: by deciding to do one
| thing, I'm by definition deciding to not do others. Running
| my own mail server is one task that has not made the cut
| for being more worthwhile than other priorities in my life.
| agrippanux wrote:
| As mentioned last week here in a previous "run your own email"
| story, your setup can also get rejected for not sending enough
| email, meaning your volume can't establish good reputation.
| jlkuester7 wrote:
| Yeah, my solution for low-volume self-hosted email is to
| relay my outgoing SMTP traffic through Amazon SES. I get good
| delivery to all the big players, bit still control all the
| parts of the email stack that I care about. (Plus at low
| volume, SES is basically free...)
| ireflect wrote:
| I have had very few troubles sending outbound email
| directly, however there was one email provider that always
| rejected me because they were blocking all of
| DigitalOcean's IP space. This provider was quite niche, but
| it still bothered me.
|
| My solution was to set up SMTP relaying based on the
| recipient domain. So nearly all my email can still be sent
| direct, but I have a list of domains that get routed
| through mailgun.com (or you could use SES or whatever).
|
| More info here: https://github.com/docker-
| mailserver/docker-mailserver/issue...
| speedgoose wrote:
| Your solution makes a lot of sense if sending emails
| through AWS is acceptable, but I'm not sure we can say it's
| self-hosted.
| xanaxagoras wrote:
| When you send an e-mail, as soon as it leaves your mail
| server it's out of your control and you have no say in
| where it goes before it ends up at the recipient. There's
| no such thing as self hosted _sent_ e-mail.
| [deleted]
| speedgoose wrote:
| Sending an email through AWS has strong implications
| regarding privacy though.
| teitoklien wrote:
| Chances are you're already compromised eitherways , most
| mailservers interact with you over STARTTLS, which means
| any middleman/isp can strip your tls encryption on your
| emails, midway while you're reading them over imap. (ISPs
| worldwide have been caught doing this before)
|
| Most providers do not mandate a strong SSL only imap
| system.
|
| Along with other similar caveats, do not depend on your
| mail system to keep your emails private.
|
| If you want privacy, encrypt your emails yourself.
|
| Also most people who selfhost their email usually do not
| encrypt data at rest. As its not the default norm. As
| such anyone having access to your disk can compromise you
| at anytime too (especially likely if you're using a vps).
|
| So, unless youre encrypting your emails yourself E2E.
| Assume you have no privacy. If you're doing E2E, aws
| cannot decrypt your mails anyways.
| BusyLurker3K wrote:
| The odds of the emails going through AWS, GCE, or Azure
| infrastructure in one form or another is probably pretty
| damn high even if you host your own SMTP service.
| speedgoose wrote:
| Yes but that's the recipient choice. I'm not responsible
| if they use American cloud providers for their emails,
| but I am for the ones I send.
| BusyLurker3K wrote:
| Sure, then you can use whatever SMTP option that is best
| for you. Some people are fine with using American cloud
| providers and some are not. If they are technical enough
| to host their own email server, they are more than likely
| to understand what choices they are making.
| speedgoose wrote:
| My main point was that sending email using AWS SES is not
| self hosting. Like hosting a website on S3 + CloudFront
| is not.
| bla15e wrote:
| Do you have any more details on implementing this? Sounds
| like something a lot of people would be interested in
| jlkuester7 wrote:
| As others have mentioned, setting up SES is
| straightforward (as much as anything AWS) and for the
| rest of my setup I just use mailu.io containers. The
| config for setting up the relay is here: https://mailu.io
| /1.8/configuration.html?highlight=relay#mail...
| teitoklien wrote:
| I have set up the same thing for myself and have been
| using it for several years now, so i'll join in.
|
| Aws ses has this offer where for a few thousand emails
| per month, email sending is free.
|
| The steps are this:
|
| 1- Signup for aws ses, once you do that they'll put you
| in a sandbox environment
|
| 2- After that they'll ask you a few questions on why you
| need it, just tell them its because you're a growing
| startup who expects to send thousands of emails per
| month, (make sure to say this, they don't crosscheck
| later, if you dont say something along the lines of this,
| they usually reject your application to avoid having to
| serve small customers who might not scale their business
| later. )
|
| 3- After you're approved, they provide you with a mail
| relay api key, just take that api key and attach it to
| your postfix or other smtpd installation
|
| I use docker-mailserver[0] which packages everything I
| need for my mailserver into a small container and was
| good to go, it consumes minimal resources too.
|
| For me, i just had to add the ses relay api key to the
| config file of my docker-mailserver install and it was
| all setup.
|
| However you can do the same with any provider that gives
| you an option to act as your email relay, I remember both
| aws ses and sendgrid provide this service, but I'm sure
| there are more niche businesses providing this too.
|
| [0](https://github.com/docker-mailserver/docker-
| mailserver)
| brotherofsteel wrote:
| > _2- After that they'll ask you a few questions on why
| you need it, just tell them its because you're a growing
| startup who expects to send thousands of emails per
| month, (make sure to say this, they don't crosscheck
| later, if you dont say something along the lines of this,
| they usually reject your application to avoid having to
| serve small customers who might not scale their business
| later. )_
|
| I have the same setup as you, relaying outbound mails
| through SES. I told exactly how I was going to use it and
| was accepted promptly. Maybe I just got lucky.
| berkes wrote:
| ...or for choosing the wrong VPS provider. Op talks about
| Digital Ocean. But, just like Linode or EC2, their IP blocks
| are inevitably on some undisclosed blocklist that livemail,
| yahoo, gmail randomly use.
| filmgirlcw wrote:
| I was going to say, IP blocks for DO make it one of the
| last places I would use for serving mail I actually wanted
| to be deliverable.
| mattl wrote:
| Digital Ocean sends so much bad traffic in the form of bots
| trying to run cars numbers we've had their entire AS
| blocked for several years.
| InvaderFizz wrote:
| > bots trying to run cars numbers
|
| I'm having trouble parsing this. Are you talking about
| VINs?
| tylersmith wrote:
| My best guess is it was a typo for (credit/debit) card
| numbers.
| mattl wrote:
| Yep sorry. Autocorrect.
| anandsuresh wrote:
| This is a bandaid that avoids solving the harder problem of
| trust/spam. It is such design patterns that make a
| fundamentally open/federated protocol more centralized,
| exacerbating the problem.
|
| Personally, I think the use of proof-of-work like methods can
| mitigate the problem by a large extent, making it
| computationally expensive to spam users. This was one of the
| original goals of what has now become the "blockchain"
| revolution. Is anyone aware of any projects that are still
| implementing similar (open) systems?
| jlkuester7 wrote:
| I am happy to be proven wrong here (not an expert) but IMHO
| there is not much hope or solving the open decentralised
| communication problem with email at all. It seems that
| something like Matrix.org presents much more promise in
| this area. I also host my own Matrix server, but sadly not
| everyone I need to communicate with uses Matrix....
| dukeofdoom wrote:
| I the webmail client experience better?
|
| I was thinking of stopping to use gmail and hotmail, and run my
| own webmail client on a droplet. I don't like the idea of all
| my documents being tracked. Is there anything that competes
| with them that I could deploy.
| dqv wrote:
| Not really. Gmail is still one of the best webmail
| experiences IMO, so you have to prefer clients like
| Thunderbird if you want to go the self-hosted route. Some
| people really like RoundCube, but I only use it when it's
| absolutely necessary. I did get some novelty out of using
| Squirrelmail as a teenager.
| denton-scratch wrote:
| Roundcube has a built-in editor for Sieve scripts.
|
| Sieve is pretty cool; it runs user-defined scripts server-
| side, on delivery to addressee's mailbox. The scripts are
| in the Sieve language, which is just for mail filtering.
| But it's a bit abstruse - it may be rudimentary, but most
| users don't want to tangle with a language at all.
|
| Anyhow, the Roundcube UI includes natively a Sieve script
| editor made up of drop-downs, which makes it much clearer
| what filter-steps you're asking for. I'm rather minimalist;
| I prefeer Squirrelmail, if I have to use webmail. But this
| feature of Roundcube is really good.
| tuldia wrote:
| Try https://www.rainloop.net/ it remembers the previous Gmail
| webmail interface. Pretty nice IMO.
| mgarfias wrote:
| 100% this. I did it for years. I think I'd rather stab myself
| than do it again.
| systemsincode wrote:
| Good article - seems comparing with Gmail upset some peeps but
| well done for having a go and trying to not just using off the
| shelf saas for every little things. Progress should have made it
| easier to host stuff ourselves not harder right?
| acallaghan wrote:
| Thanks! It works well for me, so people just are so negative -
| I didn't really expect to upset their day so much... by 'Gmail
| like' I just meant Archive/All Mail working, not just deleting
| email
| augustuspolius wrote:
| This feature is not exclusive to gmail, I have it set up with
| all my email providers.
| maltris wrote:
| For docker and mail, also check out "mailcow dockerized". Lovely
| stack of software, been using it for 5 years with no problems.
| tuldia wrote:
| The nicest email stack is: postfix, dovecot, rspamd and rainloop.
|
| EDIT: go check it out :-) https://www.rainloop.net/
|
| EDIT 2: I don't understand why other comments are so agressive
| against the author for sharing how he runs his own mail server,
| I'm not sure if it comes from one's frustration, failures,
| unreasonable expectations about email, but I noticed that
| everything related to servers or email receives this hate (here
| on HN, eh?). Come on, let's start a new year where we appreciate
| someone sharing their experience in running a mail server :-)
|
| Happy Holidays!
| acallaghan wrote:
| Thanks and cheers for being nice! Happy holidays to you too -
| I'm not really that surprised how negative the comments are
| here, including attacks on me personally it feels like.
|
| If they don't like it, stay with Gmail, I don't care. I would
| just rather live in a world where the internet isn't controlled
| by 2 or 3 big companies. Hacking a server for email and making
| it work like gmail was the aim, and I did it in less than an
| hour. Some people on here are pissed that I didn't consider
| every eventuality, and filtering, and spam and this and that.
| Fine, but attacks on me as a person reflect more on who you are
| as a person.
|
| If you don't like how I wrote or setup the server, do one and
| make one yourself - or just stay with Gmail
| ShamelessC wrote:
| dang must have the day off or something. The number of
| rulebreaking "shallow dismissals" in the comments is
| staggering. Hope you don't let it get to you! This community
| often thinks it's the center of the internet in my
| experience, and by proxy any mistake that happens is some
| sort of crime against the internet that you should pay for.
| addicted wrote:
| If youre not on the cloud (and even there on AWS and maybe
| Google Cloud...Azure is ok in a pinch... and iCloud everything
| if you also overlap with the Mac crowd) are you even an
| engineer, seems to be one popular strain of HN thought, which
| comes out particularly aggressively against e-mail servers
| because they are arguably the worst type of server to run on
| your own.
| tuldia wrote:
| Yes, and doesn't makes sense, I don't know, I run my own
| email server for more than 10 years and my experience has
| been "setup and forget".
|
| I don't understand why so much frustration coming against
| owning your own stuff.
| kova12 wrote:
| I think it's because many of us here also did run our own
| email servers at some point in time, until realization came
| how hard it is to implement and support all features one
| gets instantly and effortlessly with gmail and such
| rahimnathwani wrote:
| "I don't understand why other comments are so agressive against
| the author for sharing how he runs his own mail server"
|
| The author has been running his own mail server for less than
| half a week.
|
| There's no suggestion in the post that his setup is robust or
| 'Gmail-like', as claimed in the title.
| acallaghan wrote:
| The email inbox with Archive/ All mail does work like gmail -
| I don't use the UI. The filters are something I'm looking
| into
| rahimnathwani wrote:
| Here is a true statement: "I speak English and have black
| hair, like Keanu Reeves. I'm looking into growing a beard."
|
| If, based on the above, I were to tell people I looked like
| Keanu Reeves, would you consider that a reasonable claim?
|
| BTW I'm not deriding your efforts. I'm just saying there's
| a big gap between 'setting up my first email server with
| webmail and IMAP access' and 'setting up something with the
| features and reliability of Gmail'.
| twobitshifter wrote:
| If you just want your own address, iCloud+ now supports custom
| domains. You might already be subscribed to it and not know it.
| This also includes private relay and email hiding. It might be
| the easiest way to move your email out of gmail.
|
| https://support.apple.com/guide/icloud/add-a-custom-domain-m...
| albertgoeswoof wrote:
| What would be the benefit of moving from one mega billion
| dollar company's server to another?
| oskarc wrote:
| Important note - custom domains for iCloud is a very limited
| feature: - you can't send mails from more than 3 addresses per
| domain - it doesn't support catch-all
|
| Since I am in a time of moving to other city to study on
| university, I decided to abandon my mail server and migrate to
| iCloud... so now I am moving every of my service@domain.tld to
| prefix+service@domain.tld (tagging system that doesn't parse
| properly on some sites). It's no fun, but at least I'll take
| off my head caring whether my server is on fire, as it's now
| Apple's issue.
| ireflect wrote:
| So many negative comments, sheesh.
|
| Many of us run our own small email servers quite successfully,
| even in 2021. Every time there's a post about it on HN, all these
| commenters come forward to say it's a fools errand, that it's
| nearly impossible, nobody should try it, anybody who says it's a
| good idea is a lying idiot, etc.
|
| Sure, it's not for everyone and there are pitfalls that require
| effort and sometimes creative solutions to overcome. We should
| celebrate these projects like we do with other similarly
| challenging projects that get posted.
| eoo wrote:
| AFAIK Digital Ocean blocks outbound connections to port 25. Has
| the author actually tried this setup?
|
| Source: https://docs.digitalocean.com/support/why-is-smtp-
| blocked/
| acallaghan wrote:
| It works perfectly yes. Maybe it's not fully blocked, or DO
| restrict the amount of traffic over :25
| ireflect wrote:
| I run a production email server on Digital Ocean and have not
| experienced this issue.
|
| An issue I have experienced is that one email provider (who
| provides a white label service so that small regional ISPs can
| include a free email account to their customers) has blocked
| anything coming from DO's IP block. Ultimately my solution is
| to route those emails (and only those emails) through
| mailgun.com. The other 99.9% of my outbound email gets
| delivered directly to the final email server with no issues.
___________________________________________________________________
(page generated 2021-12-26 23:01 UTC)