[HN Gopher] Tosh: Changing your SSH server's listen address ever...
___________________________________________________________________
Tosh: Changing your SSH server's listen address every 30 seconds
based on TOTP
Author : Avamander
Score : 113 points
Date : 2021-05-22 19:26 UTC (3 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jreese wrote:
| And all you have to do is trust that your client and server will
| always have synchronized clocks...
| teraflop wrote:
| From looking at the code, seems like it would be a pretty
| simple change to make the server accept several addresses,
| derived from the current time plus or minus a small number of
| 30-second intervals. That allows you to tolerate a small amount
| of clock skew, and if your clock is off by more than a minute
| or two, you should probably fix it anyway.
| jeppesen-io wrote:
| I'm not sure how you could really run a server environment of
| any size without that
| alwaysjoe wrote:
| ... like you need to when using TOTP for anything
| nfoz wrote:
| Maybe I'm irrational, but it's one of the things that makes
| me real hesitant about where I deploy TOTP. Sometimes my
| cellphones randomly have a wildly wrong time -- a misbehaving
| (or malicious) cell-tower perhaps? And sometimes my computer
| gets the wrong time too -- e.g. booting between Windows and
| Linux screwing up the system timezone setting, or ntp failing
| to start properly, or when I busted up my CMOS. And I have to
| wonder, how secure is ntp from someone just spamming a system
| with the wrong times which can block me out?
|
| I'd almost rather a combined thing where it's HOTP but it
| also rotates once per day like at midnight? Does anything do
| that, or does it even make sense? Is there a reasonable
| alternative -- challenge-response maybe?
| jreese wrote:
| It's one thing to lock yourself out of your application or
| admin interface when NTP breaks. It's another thing entirely
| to lock yourself out of _recovering the server entirely_ when
| clock skew inevitably hits you.
|
| If you really want 2FA for SSH, use something like Yubikeys
| that increment a counter and generate tokens based on that
| counter. And use it during the actual authentication session,
| not for figuring out which magic port the server will be
| listening on. You never have to worry about synchronized
| clocks, just a database tracking the highest counter value
| ever seen, so that previous values can't be reused.
| anyfoo wrote:
| Your server has NTP. Your client likely, too, if it has enough
| network to connect to the server. If it does not use NTP, it's
| easy to set the time within the minute or so required manually.
| jreese wrote:
| Hardware clocks are famously unreliable and inaccurate. NTP
| has failure modes that can result in servers being wildly out
| of sync with reality. Letting either one of those hose your
| ability to log in and diagnose/recover the system is a
| mistake.
| jchook wrote:
| Would this potentially leak consecutive TOTPs to an attacker who
| connects to all 1,000,000 or so possible addresses at 30 second
| intervals (and records the one that responds)?
| AlexCoventry wrote:
| If that were useful, the underlying HMAC would be broken.
| wombatpm wrote:
| Make sure your time servers are configured correctly. My
| department had several Silicon Graphics workstations. Running
| IRIX, they machines determine amongst themselves which has the
| most accurate time and they vote that one as the Timemaster. Any
| new machines added would take their time from the Timemaster.
|
| The oldest machine was Timemaster and 8.5 minutes off. Took me
| week to figure out why my brand workstation had bad time. Fun
| times
| saltedonion wrote:
| Man. Time based bugs are some of the worst. I spent weeks
| trying to figure out what was wrong with some of my scripts
| running under WSL. Apparently wsl Linux kernel had a bug that
| could cause time to drift by minutes.
| imglorp wrote:
| Oohh, are we doing time bugs? Sun T4 servers around early
| 2010s, and I forget which Solaris release this was, had a
| beaut of a random clock jump.
|
| Every now and then, certain apps on the a system would crash
| around the same time. We'd scrape through the logs and
| usually see our app cratered, some databases, ntpd, sshd
| naturally. Logging of timestamps was iffy, obviously.
|
| The ntpd was the obvious suspect because, despite keeping a
| nice low offset to its peers like 1ms or two, for months at a
| time, out of the blue it would confess something like "time
| offset is too large, I can't fix that so I'll exit!". After
| chasing Sun ntpd bug reports[1] for a while, we ruled it out
| when we saw a pattern in the undamaged logs that looked like
| 09:59:58.000 ... 09:59:59.123 ...
| 09:09:01.345 ... 09:09:02.123 ...
|
| Yep the system clock really had jumped back almost an hour.
| That explained everything about the userspace going nuts
| including ntpd exiting as a symptom and not a culprit.
|
| After some Sun support and some sunbugs searching [1 again]
| we found the T4 in that Solaris rev had a hardware RTC with
| separate registers for H, M, S etc and a write mutex
| protecting them, but no read mutex. It was possible to read
| the RTC while it was being updated, which happened when it
| was syncing the OS clock to the RTC, or something like that.
| Fixed in a later release.
|
| 1: RIP sunbugs database. It was such a mature relationship
| where Sun would let everyone see what they were working on
| and customers could participate or at least know about known
| issues. I would love to find an archive. Of course Oracle
| shut that off immediately so you had to open a ticket and
| ask.
| dmos62 wrote:
| My most memorable bug (because of how long debugging took)
| was inconsistent use of local and UTC timezones. Learned to
| localize timedates as late as possible (or preferably never)
| and delocalize as early as possible.
| treeman79 wrote:
| This is why mission critical systems like Boeing airliners
| require a reboot. At some point it's just a lot more
| practical and safer.
| donkarma wrote:
| My Windows 10 clock goes 2-5x faster when I hibernate and
| come back
| emerged wrote:
| This makes me wonder if you could take this further and make the
| connection protocol dance across a variety of ports as part of
| the initial handshake.
| high_priest wrote:
| Isn't this idea port knocking in essence?
| emerged wrote:
| Oh cool, I hadn't even heard of that.
| walrus01 wrote:
| There should be no need to do this if you have a properly
| configured public/private key auth setup and disable password
| based login. And of course keep up to date on openssh patches and
| security advisories. I worry that something like this will
| provide a false sense of security for people who might ignore
| other more common-sense, fundamental precautions first.
|
| Before doing something like this I would worry a lot more about
| client endpoint security (exactly to what level do you fully
| trust all the people and workstations/laptops that are authorized
| to ssh to this thing?), as an overall more likely threat.
|
| There are also lots of less esoteric ways to not have a system
| listen on any publicly accessible IP address whatsoever. If it's
| really something critical you should be looking at a combination
| of making it purely an intranet service only, listening on an IP
| in an internal network block that isn't accessible from global
| routing tables at all. Or is completely firewalled off from the
| world, and only accessible once you've authenticated yourself to
| your VPN. Or only reachable once you first authenticate
| (public/private keys, two factor crypto key auth, etc) to a
| bastion host, and then reach the system from the bastion.
| johnklos wrote:
| Avoiding something like this because the possibility exists
| that someone will do something else wrong is just silly.
|
| Also, the amount of CPU taken by brute force ssh is not zero.
| There are plenty of good reasons for this. Even if this by
| itself isn't the best implementation, it's an example of how to
| make things much, much harder on attackers.
| walrus01 wrote:
| The amount of CPU taken by brute force SSH on any modern
| system is negligible - unless we're talking about traffic
| levels that would qualify as a DDoS. Maybe 0.02 points on a
| standard unix load scale. In any case you should have
| something like fail2ban or its equivalent that blackholes
| traffic from repeated failed attempts to authenticate, not
| just to your public facing ssh daemon, but lots of other
| things. The default debian fail2ban daemon configurations,
| easily toggled on or off to watch various log files, are
| quite sensible.
| lamontcg wrote:
| Yeah I suspect that up to date openssh with a config that
| passes ssh-audit's checks, with a fail2ban config, along with
| an ed25519 key unlocked by a yubikey will be entirely adequate
| security for SSH. Then time would be better spent securing VPNs
| and reducing internal trust.
|
| Bit like having a front door that would withstand a C4 blast,
| but now all your windows are shattered.
| ericyan wrote:
| This reminds of port knocking. Could be useful if you are really
| annoyed by those script kiddos, but in essence this is security
| by obscurity.
| megous wrote:
| Looks like fun. :) I do have nftables set up so that connection
| to normal ssh port blackholes the access to the real ssh port for
| that source IP address for some time.
|
| Not perfect, but good enough to deter people who'd first naively
| try to ssh to my server and then try to nmap it to find out if I
| have ssh running on some other port. :)
|
| 900 IP addresses blackholed just over a 1 day period. "security"
| scanning is incessant.
| xbar wrote:
| Hm. Am I ready to build toshnet, a cloud of systems running tosh
| and a toshepherd, the masternode that stores the totp to find and
| manage them all? Mayhaps.
| waihtis wrote:
| Really cool idea. If someone is interested further in the general
| concept it is called "moving target defense."
| bcoates wrote:
| It's also known as "security audit evasion" and is the kind of
| thing you might do to hide a hostile system on a network.
|
| You should never intentionally do it to your own systems. If
| you see something like this deployed assume it's either malice
| or laziness.
| [deleted]
| ryanlol wrote:
| Compared to just deploying totp normally as a PAM module this
| is a horrible idea. Much harder to ratelimit, much cheaper for
| the attacker to bruteforce codes.
|
| It's neat for sure, but not a good defense.
| waihtis wrote:
| Doesnt the TOTP in this case concern the listening port for
| sshd, so it doesnt actually touch the authentication in any
| way? Just switches the port in a TOTP'esque manner
| ranieuwe wrote:
| This solution switches the listening address (IPv6
| address), not the port.
| waihtis wrote:
| Indeed you are correct.. Fruits of multitasking..
| gruez wrote:
| The point is that if the attacker somehold got hold of the
| primary login credentials (username + key/password), then
| they can easily bypass this scheme with a port/address
| scan. This can be done very quick[1] and is hard to rate
| limit. Furthermore, an attacker that can eavesdrop on the
| user's connections can infer the OTP since it's being
| transmitted in the open., but if it was done through a PAM
| module they wouldn't be able to because it's encrypted.
|
| [1] https://nmap.org/book/synscan.html
| throwaway2048 wrote:
| there is no "easily bypassing this scheme with a
| port/address scan" when it comes to ipv6 /64 ranges.
|
| If you could scan 1 million IP addresses a second on a
| /64 (which is absurd), it would take 600K years to scan a
| full /64.
| waihtis wrote:
| Both scenarios assume a pretty heavy compromise already
| in place before the ssh control starts crumbling
| thecodemonkey wrote:
| That's a super cool concept, but doesn't this fall a bit under
| "security by obscurity"?
| n42 wrote:
| The "security by obscurity" one-liner is one of my favorite
| examples of the sort of black-and-white thinking that is
| harmful to software engineering.
|
| The truth is that playing defense is as much an exercise of
| technical design as it is economics.
|
| Yes - if someone finds the SSH port, they have a window of
| opportunity, and you will be owned if you are not properly
| securing your server through the normal channels.
|
| However, now they only have a small window of opportunity (say,
| 30 seconds). This does a few things:
|
| - it takes time (money) to attack a target. without access to
| the OTP secret, randomly assigning ports dramatically increases
| the cost (time) of attacking you. throw in a tar pit and it's
| even worse. if you're not a high value target, the attacker
| moves on.
|
| - now, any failed authentication attempt to your SSH server is
| a highly credible threat. repeat attempts are even more suspect
| - you are being targeted, and they probably have the OTP
| secret. effectively, you are able to resource your team more
| efficiently, because you can filter out noise.
|
| security is not black and white. if you are a valuable enough
| target, someone will find a way in. defense in depth helps you
| manage your defense with limited resources.
| walrus01 wrote:
| > The "security by obscurity" one-liner is one of my favorite
| examples of the sort of black-and-white thinking that is
| harmful to software engineering.
|
| My primary problem with people going for "security through
| obscurity" is that very often there are things which are
| intentionally obfuscated or obscured, in such a way that the
| implementer _thinks_ that their method of hiding things will
| provide a significant measure of security. But then all of
| the other more common sense security precautions that should
| be implemented _before_ the obscurity are ignored.
|
| If I had a dollar for every industrial/embedded/M2M/IOT type
| thing that tries to be secure through obscurity but has other
| gaping holes in it, once you're familiar with the technical
| workings of the product...
| jopsen wrote:
| > defense in depth helps you manage your defense with limited
| resources.
|
| Indeed, but might there not better/cheaper ways to secure
| SSH?
|
| Something that doesn't involve custom configuration that
| needs to be maintained.. like VPN to a jump-host... Or..?
|
| Configuring and maintaining custom hacks is not cheap.
| n42 wrote:
| agreed - I would not deploy this as is, but the idea is
| interesting and I could imagine the concept being given
| some UX love in a different application
| SrslyJosh wrote:
| I like this analysis, but I'm having a hard time seeing the
| advantage over port-knocking, which could also be randomized
| using OTP and would never reveal the SSH server to a port
| scan.
|
| Zero window seems better than a 30 second window.
|
| Excellent points otherwise.
| staticassertion wrote:
| Security by obscurity is when you hide implementation details
| to improve security. Secrets are not obscurity, randomness is
| not obscurity.
| angry_octet wrote:
| Except that if they know you are using this technique (e.g.
| from snooping traffic) then it is straightforward to bypass,
| either by tailgating onto a recent connection attempt (if
| they can snoop) or just brute forcing it (they can test the
| whole key space in seconds).
|
| There must be better ways to leverage long term shared
| secrets, recent authentication success, etc. I'd like to see
| something like Signal's ratchet mechanism.
| yonixw wrote:
| I think the time aspect makes it OK, otherwise TOTP itself
| should be abandoned due to the same principle. (SSH still has
| password)
| benjohnson wrote:
| It's another layer of security. There have been exploits of
| OpenSSH in past so this may be prudent.
| zbuf wrote:
| Trouble with extra layers, there's a point where it results
| in complexity. Which, in my experience, is more likely to be
| the root cause of a security problem.
|
| I'm not saying this little demo is a disaster or anything.
| But for example, perhaps it requires an awareness of this
| scheme in an external firewall's rules, and maybe another
| machine pops up in the rather large IPv6 range that's now
| available.
|
| At its extreme, these sorts of approaches can bring a lack of
| clarity which layer is providing the _actual_ security.
| dec0dedab0de wrote:
| Yes, but that's only bad if it's your only security.
| anjbe wrote:
| Layering has additional costs, like requiring additional
| client configuration and software and (in this case) only
| working over IPv6.
|
| The number one step any public-facing SSH server should take
| is to switch from password auth to keys only. Anyone who's
| still concerned can put it behind a WireGuard VPN. Layers
| typically added beyond that (like changing port, etc.) don't
| even register on the security scale, so to speak.
|
| The tweet that inspired the post mentioned port knocking
| which has always been rather ridiculous given those
| alternatives.
| Bancakes wrote:
| The simple fact that ssh is over IPv6 already leaves out
| 99% of potential hackers aka bots.
| alias_neo wrote:
| I have some "cloud" VMs which have been around several
| years, they're attacked constantly, I haven't seen a
| single incoming IPv6 source to date.
|
| I wonder if there's a "missed opportunity" for hackers
| there given presumably how many forget to configure their
| IPv6 firewall.
| oh_sigh wrote:
| Actually it is far better than having no security at all
| ad404b8a372f2b9 wrote:
| I don't know why people knock on "security by obscurity" in
| general, it's a great defense for many threat models that an
| average individual would fall under. A lot of credential
| stealing viruses just look for specific folders or file names
| for examples, some vulnerability scanners look for specific
| ports, etc ...
| taneq wrote:
| It's because they had some security experts say it (about
| things like moving your open mail relay to port 7384 to
| 'secures' it) and it's the only thing they know about
| security.
| not_kurt_godel wrote:
| I view it as a cost-benefit analysis between implementing
| security by obscurity versus actual security. If you have
| real security, then you don't need security by obscurity. If
| you have security by obscurity, you still need real security.
| So it's obvious which is a better ROI. That's not to say
| security by obscurity layered on top can't be useful, such as
| filtering out noise, but I think the point most people are
| trying to make is "this thing is not a solution to your
| security problem, it is a potentially dangerous distraction".
| MrStonedOne wrote:
| security by obscurity and defense in depth are synonyms.
| angry_octet wrote:
| No they are not. It doesn't help to have obscurity in depth.
| Godel_unicode wrote:
| In the same way that passwords, private keys, and safe
| combinations are security by obscurity, sure.
| a1369209993 wrote:
| No, the TOTP is[0] a shared-secret bitstream (like a stream
| cypher), which provides new randomness per use (obscurity means
| it relies on the attacker not knowing how it works in the first
| place). This is very _weak_ security, similar to a combination
| lock or PIN, and should not be used in place of proper SSH
| crypto, but it is actual security. (The point, IIUC, is to
| quickly exclude drive-by attackers, so that serious /targeted
| attacks are higher above the noise floor.)
|
| 0: assuming I'm not being overly charitable
| johnklos wrote:
| Many of us have done something similar, so it's good to see
| something that's better than a few shell scrips. OTOH, it's
| written in Rust, which is a bit heavy for something so simple.
| timf wrote:
| It all depends on your constraints and who needs SSH access, but
| if you're like me, you (1) have a number of personal VM instances
| on major cloud providers and (2) don't want to deal with anything
| but 'vanilla' Linux and sshd. No customizing kernels, no
| portknocking, no TCP wrappers, no fail2ban, no VPNs (not even
| tailscale which is very nice).
|
| The approach that I've been using for about a decade is a script
| that gets your current (internet facing) IP and then uses a cloud
| API credential to add that IP to the cloud provider's firewall as
| a valid source IP to port 22. The API security and cloud firewall
| implementation is left to the major cloud providers (and they are
| very good at these things).
|
| I run it manually whenever I'm in a new location or in the rare
| cases my home IPs change. You can add the IP to the list or
| replace the list each time. Or clean it up after a trip with a
| separate CLI flag. I figured one day I would automate it to track
| IPs, run constantly, and trigger the change if it detected a new
| IP - but that day never came because it never annoyed me (ymmv).
|
| This approach could be extended to teams, but there are more
| details to think through about API permissions, etc.
| walrus01 wrote:
| Can I ask why you don't trust running fail2ban on 'cloud' based
| VMs? The defaults for how long it will ban failed ssh attempts
| are quite sensible, and short enough to prevent you from
| accidentally locking yourself out for a very long period of
| time (automatic unban time to remove iptables rules).
|
| If nothing else, it serves the useful purpose of stopping the
| log files from being cluttered up with various botnets' fully
| automated ssh username/password attempts that are out there,
| trying to gain access via well known factory default
| credentials.
| timf wrote:
| It's not really about trust, it's about the hassle of adding
| and maintaining things (including keeping up with their
| changelogs and security issues).
|
| As for log clutter, my approach comprehensively stops the
| logs files from being cluttered with failed attempts; that's
| one of the main reasons I do it.
| walrus01 wrote:
| That is a good point but I should also add that fail2ban as
| a daemon and its default configuration to watch log files
| for other popular daemons (example: a postfix smtp traffic
| log) has changed very, very little in the past 5-6 years.
| It's not very much of a concern in terms of keeping track
| of configuration changes in major system upgrades.
| cliffy wrote:
| Anyone done TOTP port-knocking to open an SSH listen port with
| just the originator white-listed for a short window?
| Zenst wrote:
| I had a port-knocking ssh OpenBSD setup late 90's, did move the
| port about though, just white listed opened. Used SKEY for port
| sequence, was easy to run even on a Nokia in Java( _) in that
| era.
|
| [EDIT ADD]>_Actually was some Symbian and then Java
| implementation of the skey. Had Psion Series 5 used initially.
| ffk wrote:
| Why not just implement Single Packet Authorization (SPA)? The
| port only opens to that specific source on a recent
| cryptographically signed request with a timestamp.
| https://www.linuxjournal.com/article/9621
| thomashabets2 wrote:
| Stop it with this cloak and dagger BS. Just set a shared secret
| on it: https://github.com/google/tcpauth
|
| (Yes, MD5 is safe for this use)
|
| With tcp MD5 your connection is even secured against an active
| attacker who can sniff. They can't inject, or even RST the
| connection. Even if they can sniff and spoof everything.
| tptacek wrote:
| If this is a joke, it is very well executed.
| ffk wrote:
| It's not a joke, BGP servers used to authenticate each other
| with this in the late 90s. This was also before NAT was
| really a thing.
| walrus01 wrote:
| a very small percentage of ISPs at some major IX points
| still want MD5 auth on BGP sessions across the fabric.
| Usually a moot point these days since the IX operator
| should have solid, reliable documentation of exactly what
| switch port and fiber patch panel assignment goes to which
| cage/suite/cabinet and ISP.
|
| Or in the case of a PNI between two ISPs over their own
| cross connect, you absolutely want to have a mutual level
| of trust and cooperation between the BGP peers on both
| sides of the session.
|
| And then other more modern methods of verifying that the IP
| blocks you're seeing from some other AS are legit, like
| verifying their RPKI signatures, IRR entries, etc.
| top_post wrote:
| Heh guess I can cross this off my ideas list. 100% this is the
| way to go.
| ekarulf wrote:
| Is this just spiped with MD5 instead of HMAC-SHA256?
|
| https://www.tarsnap.com/spiped.html
| content_sesh wrote:
| Reminds me of frequency hopping.
|
| https://en.wikipedia.org/wiki/Frequency-hopping_spread_spect...
|
| Interesting side-note, one of the people who re-invented this
| technique was actress Hedy Lamarr who was quite an accomplished
| inventor
| tekacs wrote:
| Unlike port knocking (which folks seem to be mentioning a lot),
| this seems like it only has 1,000,000 possible locations that can
| rapidly be scanned.
|
| Knocking has multiplicative growth and so many more
| possibilities.
|
| Perhaps you could include honeypots in the IPv6 range where
| you're not bound that block the user, but this seems less
| reliable overall.
|
| I'm sure this is just great fun, but perhaps not something to
| think of as secure so much as a fun idea -- to make it secure you
| might want to use a system more like knocking.
| notorandit wrote:
| Super koel!
| jftuga wrote:
| I have taken a different approach to remote access in limited
| scenarios:
|
| gofwd: A cross-platform TCP port forwarder with Duo 2FA and Geo-
| IP integration. Its use case is to help protect services when
| using a VPN is not possible. Before a connection is forwarded,
| the remote IP address is geographically checked against city,
| region (state), and/or country. Distance (in miles) can also be
| used. If this condition is satisfied, a Duo 2FA request can then
| be sent to a mobile device. The connection is only forwarded
| after Duo has verified the user.
|
| https://github.com/jftuga/gofwd
| electric_mayhem wrote:
| I invented this in November 2019. Regrettably the write up and my
| PoC implementation are owned by my employer. :-/
|
| Good on you for coming up with it independently. I'm a little
| jealous, but happy the idea is out in the world.
| andi999 wrote:
| Somehow my windows computer doesn't want to fetch the correct
| time. Would probably be a nightmare.
| atatatat wrote:
| Win10?
| pmlnr wrote:
| systemd-timesyncd is also a lovely possibility of forking it
| up.
| andi999 wrote:
| Yup
___________________________________________________________________
(page generated 2021-05-22 23:00 UTC)