[HN Gopher] If you're not using SSH certificates you're doing SS...
       ___________________________________________________________________
        
       If you're not using SSH certificates you're doing SSH wrong (2019)
        
       Author : noyesno
       Score  : 367 points
       Date   : 2022-03-24 11:46 UTC (11 hours ago)
        
 (HTM) web link (smallstep.com)
 (TXT) w3m dump (smallstep.com)
        
       | politician wrote:
       | The SSH servers that I'm familiar with are spun up with a host
       | cert, so all of the FUD in this article about connecting to an
       | unknown host is a non-issue. Check that the host cert matches the
       | one you expect once, and the tooling makes sure to notify you if
       | it changes.
       | 
       | As far as provisioning, maintaining a secure CA signing practice
       | is a nightmare. It's K8S level of self-inflicted pain for a
       | startup. If you're running at a larger scale and can dedicate a
       | team to it, fine. If you're a dozen people trying to launch,
       | getting the devops guy to run `ssh-copy-id` is not the challenge
       | that this article makes it out to be. Nor is the slightly more
       | automated Terraform script that installs and uninstalls
       | authorized keys from servers.
        
       | soraminazuki wrote:
       | I feel that trying to make SSH keys short-lived is becoming more
       | painful each year because there's an increase of tools that use
       | SSH keys for purposes other than SSH logins. For example, age [1]
       | encrypts files with SSH keys, agenix [2] does secrets management
       | with it, Git can now sign commits with it [3], and even ssh-
       | keygen can now sign arbitrary data [4]. All of these become
       | useless the moment you start using short-lived keys.
       | 
       | [1]: https://github.com/FiloSottile/age
       | 
       | [2]: https://github.com/ryantm/agenix
       | 
       | [3]: https://calebhearth.com/sign-git-with-ssh
       | 
       | [4]: https://www.man7.org/linux/man-pages/man1/ssh-keygen.1.html
        
         | ayushnix wrote:
         | Umm, please correct me if I'm wrong but I think you're
         | confusing SSH keys with SSH certificates. A SSH client key can
         | be reused to create short lived SSH client certificates. You
         | can keep using that SSH client key to encrypt data, sign data,
         | login to GitHub etc. There's no such thing as "short-lived
         | keys", there's short lived SSH certificates.
        
           | soraminazuki wrote:
           | Yes, but the entire point of the described setup is to get
           | rid of traditional long lasting keys in favor of ephemeral
           | certificates (which I believe is another way of saying signed
           | keys) obtained through SSO. Signing certificates with your
           | existing keys kind of make the whole point moot.
        
             | ayushnix wrote:
             | I doubt we're getting rid of traditional long lasting keys
             | anytime soon. They do have their uses and it'd be a waste
             | not to use them.
             | 
             | > Signing certificates with your existing keys kind of make
             | the whole point moot.
             | 
             | You mean getting signed certificates from a SSH CA with an
             | existing client public key? Why does it make the whole
             | point moot? The SSO is responsible for verifying your
             | identity. Rather, it seems pointless to generate a new SSH
             | keypair each and everytime you want to get a certificate
             | and login to a machine. You can certainly do it if you want
             | but I don't see the point. You can keep using your existing
             | SSH public keys to get short lived certificates and login
             | to a machine, do you job, logout, and repeat the process
             | with the same keys.
        
           | [deleted]
        
           | pphysch wrote:
           | Yes, a cert is just a public key that's been "stamped" by a
           | certificate-authority (CA), allowing it to be validated by
           | servers holding the CA public key (as well as enforcing other
           | policies like lifespan, principles). It is a totally separate
           | file and does not modify the original public or private key,
           | which indeed have no notion of lifespan.
           | 
           | If you are constantly regenerating uncompromised SSH _keys_ ,
           | you are probably doing something wrong.
           | 
           | The GP is misleading in this way.
        
             | ayushnix wrote:
             | > If you are constantly regenerating uncompromised SSH
             | keys, you are probably doing something wrong.
             | 
             | Yup, there's no real reason to generate a new SSH key pair
             | each and every time you want to get a short lived
             | certificate. The SSO or CA management system (like Vault)
             | is responsible for verifying your identity.
        
               | dfox wrote:
               | This is also true for X.509, there is exactly zero reason
               | to generate new key (if it was not compromised) or even
               | new CSR for certificate renewal. Yet people tend to do
               | this which only increases the opportunities to fuck
               | something up in the process. (Well, it does not make much
               | sense, but over last year I have seen at least three
               | instances of somebody overwriting the only copy of newly
               | generated private key with the old one...)
        
         | imwillofficial wrote:
         | Different strokes for different folks.
         | 
         | Different keys for uhh, different complex application use
         | cases.
        
       | gotaquestion wrote:
       | Cool, how do I set up a CA?
        
         | tfigment wrote:
         | Setup Hashicorp Vault. Almost easy but actyally hard to do
         | right. Policies are easy to make too open and possibly
         | insecure.
        
         | 0xdeadb00f wrote:
         | Precisely this. From what I've read it isn't that easy to setup
         | a CA.
         | 
         | Look into step-ca though, I've heard it's.. Okay? I don't know.
         | It seems too complicated still - I'd rather stick with pubkey
         | auth
        
         | aidenn0 wrote:
         | All you need is an ssh key, which you can generate like this:
         | ssh-keygen -f ca.key
         | 
         | Then you can generate certificates like this:                 #
         | user key       ssh-keygen -s ca.key -I key_id
         | /path/to/user_key.pub       # host key       ssh-keygen -s
         | ca.key -I key_id -h /path/to/host_key.pub
         | 
         | Secure ca.key according to whatever level of paranoia you
         | desire. e.g. Passphrase, hardware security module (PKCS#12 is
         | supported for generating certs), airgap the machine. anyone who
         | gets access to ca.key has access to everything that trusts
         | ca.key
        
       | charles_f wrote:
       | > you're doing SSH wrong
       | 
       | I understand the theoretical superiority to keys, but do we have
       | some data per practically how many times key security actually
       | failed someone?
        
         | haarts wrote:
         | I would very much like to read about that too.
         | 
         | Setting some sane security parameters for your SSH setup looks
         | like a less jarring/drastic approach into securing SSH
         | further[1]:
         | 
         | - Use keys.
         | 
         | - Allowing only strong cyphers.
         | 
         | - Remove weak primes.
         | 
         | [1]: https://disknotifier.com/blog/simple-ssh-security/
        
       | FriedrichN wrote:
       | Stuff like this is really cool. But the problem is that certain
       | clients (as in clients that pay your bills) can't even get public
       | keys to work and we end up allowing password logins. So I don't
       | know how I could make this work in the real world.
       | 
       | That's where security ends, when 'it just has to work' because
       | 'they're the ones paying/in charge'.
        
       | _wldu wrote:
       | SSH certificates are useful in large environments when scaling,
       | automatic onboarding and offboarding are important, but IMO,
       | small teams can (and should) continue using authorized key files
       | as they have for years. They don't really need these features.
        
       | gernb wrote:
       | dumb question but .... isn't it a problem that private ssh keys
       | are stored in ~/.ssh and that any random app, npm dependency,
       | build script, etc could copy them across the network?
        
         | marwis wrote:
         | You can password protect them and optionally load them to agent
         | at logon.
        
       | dmuth wrote:
       | Agreed--SSH certificate authorities (and principals) are powerful
       | things that can be used to manage SSH access at scale. My
       | workplace is a large enterprise that uses our own CA for getting
       | access to systems--the keys it issues are good for 8 hours, then
       | we have to grab a new key (using an internal utility).
       | 
       | For anyone who is interested, I put together a little playground
       | which can be spun up in Docker that allows you to play around
       | with and learn how SSH CAs and Principals work:
       | 
       | https://github.com/dmuth/ssh-principal-and-ca-playground
        
       | quillo wrote:
       | I have built a small signing service that works by a user SSHing
       | in, performing LDAP (password) authentication and 2FA with duo,
       | then injecting a time-limited certificate signed by Hashicorp
       | Vault back to their user agent (although it could be modified to
       | remove the Vault requirement). The UX is very simple (SSH to this
       | address once per day), but the backend is complicated, so if
       | there is any demand for me to put this on Github I am happy to do
       | so.
        
       | client4 wrote:
       | Keybase had an elegant solution I use
       | https://keybase.io/blog/keybase-ssh-ca
        
         | bloopernova wrote:
         | I used to like Keybase so much, it felt like a Next Big Thing.
         | I just wish they had stuck to identity management and
         | validation, providing SSO etc. Instead they tried to be a chat
         | client, git host, and crypto wallet too. They spread themselves
         | too thinly trying to compete with dozens of rivals for each
         | function they added.
         | 
         | I wish they'd have been a standard that Microsoft, Apple,
         | Google etc provided implementations of.
        
       | tonymet wrote:
       | Great article. All of your internal authentication should be
       | using certificates. Web auth, Wifi, VPN, SSH
       | 
       | In the late 90s we came close to having this for the public
       | internet as well but it never caught on. We paid the price with
       | endless breaches and unmanageable credentials.
        
       | brightball wrote:
       | Isn't this the process that Teleport streamlines?
        
         | pzduniak wrote:
         | Teleport completely streamlined all sorts of access management
         | in my projects. Every new feature they release (Kubernetes,
         | Database, App access) work as advertised and the Helm chart +
         | Terraform plugin (which for some reason is not published in the
         | marketplace??) are great for automating the whole deployment +
         | configuration. Great piece of software.
        
       | sjaak wrote:
       | I'm not doing anything wrong.
       | 
       | My sshd is behind spiped + logging in requires me to physically
       | tap my yubikey.
        
       | GekkePrutser wrote:
       | SSH certificates make sense. But can you use hardware backed ones
       | like the OpenPGP applet on yubikey with this?
       | 
       | I currently use this method to store my SSH keys safely. But I
       | don't know how this would work with certificates. If I have to
       | store them in the computer instead of a hardware token it's a
       | huge step back in security.
       | 
       | By the way what do home users use to set up a PKI? Scripting
       | everything with OpenSSL is but very nice. It would be cool if
       | there were an open source PKI platform ideally even with IDP
       | built in. With a nice web interface and easy to install with
       | docker. Never found one though.
        
         | blueflow wrote:
         | Yes, ssh-keygen allows you to use private keys from any PKCS#11
         | backend using the -D option. This includes smartcards and
         | tokens.
         | 
         | Including for the CA key.
        
           | GekkePrutser wrote:
           | Ah Yes I used this method before with PIV cards and
           | OpenSC/OpenCT.
           | 
           | The toolchain is a real PITA though. Unstable, difficult to
           | provision. Proprietary tools for card management. Not all
           | platforms support PKCS modules. As far as I remember openssh
           | on macOS was compiled with support for it. So I had to
           | replace it with one from brew which is much harder these
           | days. Mind you we're talking 5-6 years ago.
           | 
           | OpenPGP is really user-friendly. Nice config menu with gpg
           | --card-edit . SSH agent functionality built into the gpg
           | agent.
           | 
           | I kinda want to retain this level of comfort to be honest.
        
             | RL_Quine wrote:
             | Why use gpg at all? SSH supports FIDO.
        
               | dfshsdfjsfj wrote:
               | FIDO is still of extremely limited availability. I'm
               | still running into hosts I need to access that can only
               | use RSA/DSA keys, as opposed to the ed25519 key on my
               | Yubikey, nevermind FIDO.
        
               | Sharparam wrote:
               | I tried the FIDO2 way of authenticating to SSH when I
               | finally had a Linux system with new enough versions of
               | OpenSSH et.c in the repos.
               | 
               | It works quite nicely, but the PIN has to be entered
               | every time I auth to SSH. This may be a desirable feature
               | to some, but I prefer the GPG way of the PIN being cached
               | until the key is removed from the system.
               | 
               | (This could possibly be a shortcoming of KWallet, as it
               | would pop up the dialog asking for the PIN, but checking
               | the "remember this password" would achieve absolutely
               | nothing, and besides, I wouldn't want to save it
               | permanently, which that checkbox would otherwise do.)
        
               | RL_Quine wrote:
               | I don't have any PIN entry at all with FIDO.
        
               | Sharparam wrote:
               | But then someone with your key just has direct access?
        
               | RL_Quine wrote:
               | It's a second factor, not first. This is fine.
        
               | GekkePrutser wrote:
               | But this is not FIDO2 in CTAP (passwordless) mode, which
               | is what the previous poster referred to. It's FIDO1 and
               | it's not supported by default in SSH, only with some PAM
               | plugins.
               | 
               | OpenSSH supports FIDO2 passwordless mode natively in the
               | latest versions.
        
               | GekkePrutser wrote:
               | Ooh yeah I definitely don't want this either.
               | 
               | I want GPG to ask for the pincode once, and the yubikey
               | to require a physical touch for each authentication
               | (including the first, obviously). This way it can't be
               | automated by malware either (a pincode can be sniffed and
               | replayed through the keyboard driver, the physical touch
               | can't).
               | 
               | The PIN is great against attackers that find your key on
               | the street. Not against a determined attacker that is
               | already on your computer. For that the physical touch
               | thing is a great solution (though the yubikey doesn't
               | require it by default, you can easily turn it on).
               | 
               | Asking the pin upon first use and a touch every time is
               | the perfect compromise between security and usability
               | IMO. There's still some weakness around attackers with
               | physical access but they are more easily mitigated.
        
               | GekkePrutser wrote:
               | Good point but it's not well supported yet. There's no
               | mobile clients I know of that support it (let alone with
               | agent forwarding which I really need for jumpboxes).
               | 
               | And of course once you do fido, you're back to the same
               | issues around SSH keys that certificates are a solution
               | to (as the article demonstrates). So moving to Fido is
               | not a whole lot better than using SSH keys which work
               | very well everywhere.
               | 
               | Also, another important point: I also use GPG a lot to
               | encrypt files. It's great to use the same key (sometimes,
               | for sensitive stuff I use a different OpenPGP card) and
               | toolchain (always) for this. So I need it anyway, might
               | as well use it for SSH authentication as well.
        
       | Cpoll wrote:
       | I wish revocation was covered as well. The article mentions this
       | issue in SSH:
       | 
       | > Keys are trusted permanently, so mistakes are fail-open.
       | 
       | But you can make the same mistake using certificates, by issuing
       | a certificate with a large expiration date. AFAIR you _can_
       | revoke valid certs, but that involves making changes to every box
       | running ssh, much like when revoking a public key.
        
         | blueflow wrote:
         | You can distribute a key revocation list along with your
         | regular configuration management. The key revocation list can
         | be generated and maintained via ssh-keygen.
         | 
         | I used to have a central location with an revocation list. All
         | servers had a cronjob fetching it.
        
           | camtarn wrote:
           | And that's pretty much the exact same infrastructure you'd
           | need for distributing authorized_keys.
           | 
           | Instant revocation seems to be a giant hole in this article's
           | argument which isn't addressed at all. If you fire somebody
           | partway through the work day, end-of-day cert expiry is not
           | good enough to prevent compromises.
        
             | blueflow wrote:
             | Active reconfiguration doesn't work well if your machines
             | aren't unconditionally reachable.
        
               | camtarn wrote:
               | Hm, that is true.
        
               | [deleted]
        
       | pilif wrote:
       | I love ssh certificates for access and indeed we're using this
       | for accessing our production network using a few small pieces of
       | home-grown infrastructure.
       | 
       | However, there's one big issue nobody is talking about: There is
       | zero support for certificate authentication in any SSH clients
       | for iOS and most people who have network access here also have
       | iOS devices.
       | 
       | And even if there were support, just supporting the certificates
       | alone is not enough - there would need to be some automatable way
       | of getting a new certificate into an app as the whole idea of the
       | certificates is that they are very short-lived (days or even
       | hours if possible)
        
       | risson wrote:
       | Anyone have experience with using SSHFP records to avoid the so-
       | called anti pattern of trust on first use?
        
       | emrvb wrote:
       | I would love this to work, as it indeed fixes several issues
       | addressed in the article.
       | 
       | However, I don't think openssh-server supports OCSP natively, so
       | while you might be doing SSH right, you're doing certificates
       | wrong.
        
       | 1970-01-01 wrote:
       | Meh, you're not _wrong_ for skipping SSH certs. They 're mature
       | security, but not mature enough for everyone. And like everything
       | else, they break, predictably and unpredictably. If you're not
       | ready for someone to emergency access their way into prod to fix
       | a broken SSH issue on Christmas morning, you're not ready for SSH
       | certs. Maintenance will get you, one way or another..
        
         | sergiosgc wrote:
         | The SSO flow is way too complex for something that _must_ work
         | on an emergency.
        
         | teknopaul wrote:
         | Is there something wrong with installing client public keys in
         | the server and preventing password based logins?
         | 
         | Seems more secure than handing over the security of your
         | servers to a CA?
         | 
         | Managing the authenticated_keys file is trivial and I like
         | simple file based solutions.
         | 
         | I find simpler is often more secure because its easier to
         | understand.
         | 
         | I have had plenty of ssh servers on the Internet even, (on
         | non-22 ports to prevent logs filling) and not had a problem yet
         | AFAIK.
         | 
         | I have also been on call when certs expired gawd knows how many
         | times.
        
           | aidenn0 wrote:
           | I'm not sure I buy it, but, TFA tries to explain:
           | 
           | Situation: A person's machine dies and takes the SSH private
           | key with it
           | 
           | No Certs: They now need to get IT to distribute the public
           | keys to a set of hosts, where the full list is possibly not
           | known by any one person, so for the next week it will be "oh
           | shit, I forgot to ask IT to put the pub key on server-12, so
           | I'll open a ticket and not get any work done for the next few
           | hours"
           | 
           | With Certs: They get IT to sign the new key.
        
             | michaelt wrote:
             | _> They now need to get IT to distribute the public keys to
             | a set of hosts, where the full list is possibly not known
             | by any one person_
             | 
             | If the team responsible for server security isn't able to
             | identify all the servers they're responsible for, you've
             | got _far_ bigger problems than SSH certificates are going
             | to solve.
        
             | aseipp wrote:
             | The second part is only kind of accurate, because usually
             | the "sign a new key" part is automated as part of some
             | other authorization flow. For instance you can log into
             | your corporate SSO provider, and that can actually
             | provision a short-term (i.e. 1 hour) SSH key that is signed
             | by the CA for you. That short-term key is then used to
             | shell into hosts, and you periodically renew either the key
             | or the SSO session token. There's a bunch of points in the
             | design space you can do to make this transparent. Normally
             | it's all wrapped up in some command that invokes ssh for
             | you; so you just say 'corp-auth ssh username@host' instead
             | of just 'ssh username@host' and it all "Just Works" if
             | you're lucky.
             | 
             | You can also bake some basic rules/policies into the
             | certificate, i.e. the above flow returns a signed
             | certificate that is only valid for SSH'ing into hosts in
             | the 'www' group (not the 'database' group.) So then
             | operations people can add you to groups in some other place
             | (LDAP, Exchange, whatever) and when they add you to the
             | 'database' group, any future short-term SSH keys issued by
             | the central authority allow you to now shell into the
             | 'database' hosts. The hosts themselves can also have some
             | automation to check these permissions are accurate when an
             | SSH login request occurs. So you just file an IT ticket
             | asking to shell into a host or group of hosts, they fiddle
             | with some knobs somewhere, and a few minutes later you're
             | done.
             | 
             | So the actual second part is more like "Get a new laptop
             | and log back in through Okta" or whatever, and you have all
             | the exact same permissions and whatnot you did before. It's
             | easier for both the user and the administrators.
             | 
             | An immediate advantage of this are that keys are easy to
             | immediately audit and revoke (because a central authority
             | issues them so you have a trusted trail). But there are
             | some more subtle advantages; one for example is that you
             | make servers more homogenous and identical. They all just
             | ship some specific sshd_config file, normally identical,
             | rather than each server potentially having a different
             | authorized_keys file; instead the central authority that
             | issues keys is where the mapping of hosts/authorized users
             | is, rather than each server having that knowledge
             | "individually" in its own file. That's easier to
             | understand, track, keep up to date (who has access where?)
             | etc. You can just go modify a single global user and have
             | the permissions flow downwards.
             | 
             | None of this really matters if you only have like 1 or 2
             | people doing everything, or it's your home network, but
             | once you get above like, 5-10 people, or you have at least
             | one sysadmin, it's actually really useful IMO. Also, to
             | some extent, you can mix and match various parts of the
             | above concepts (e.g. running code on every login request,
             | to add extra authorization checks based on out-of-band
             | information.) You have to decide what's appropriate for
             | you. Read the ssh and sshd man pages and you'd be surprised
             | at what you can do.
             | 
             | Source: I basically implemented my own SSH certificate
             | authority infrastructure (server/client automation) "for
             | fun."
        
               | aidenn0 wrote:
               | I guess I don't see how this is better than using LDAP or
               | AD authentication on each host?
               | 
               | Also, users tend to hate having to always relogin to SSO;
               | maybe that's because the implementations have poor UX,
               | and maybe there's no secure way around it.
        
               | aseipp wrote:
               | Sure, you can use LDAP or AD or any other number of
               | things to control server authentication as well as
               | mapping some global database of user IDs to accounts. You
               | could also do other things like combine this with a 2FA
               | solution like Duo.
               | 
               | One thing SSH certificates certainly have going for them
               | is that they're actually easy to script and integrate
               | with, and "piecewise" migrate to, in my experience, while
               | using a flow you already are pretty familiar with. I
               | personally didn't use any sort of LDAP or AD setup to
               | back my design; you can implement a custom backend for
               | all this pretty easily yourself. There's nothing
               | inherently confusing about the concept of cryptographic
               | certificate authorities or anything, anymore than public
               | key cryptography itself. It's a relatively natural
               | extension of the SSH design you know already, is my
               | point. Again, the man page is worth reading to understand
               | it all a bit better.
               | 
               | > Also, users tend to hate having to always relogin to
               | SSO; maybe that's because the implementations have poor
               | UX, and maybe there's no secure way around it.
               | 
               | Well, I'll be honest, people who tend to use SSH and
               | would be impacted by this stuff tend to hate lots of
               | things and not always for good reasons. Put another way,
               | listening to developers or whatever about what they hate
               | and what's actually good isn't something I would factor
               | into something like this. SSO is mandatory for very good
               | reasons at any reasonable scale (and by "reasonable" my
               | opinion is you should have it in place at, like, 10+
               | people.)
               | 
               | Anyway, besides that. There's nothing in theory that
               | prevents you from doing something specific like having
               | the backend refresh the SSO token issued for your SSH
               | certificates every time you log into some server, upto
               | some given interval e.g. logging in at least once a day
               | seems reasonable, but if you login every 5 minutes to a
               | new set of hosts you can refresh the token.
               | 
               | In my case the flow was something like 'my-ssh-ca-wrapper
               | ssh user@bar', which would ask you for a token. I would
               | then get this token by visiting a little webpage I wrote,
               | but in theory it could also just launch the browser
               | itself with xdg-open with a direct link. I just use a
               | password manager to fill out those "SSO" credentials. It
               | isn't ideal or fully integrated but in practice it would
               | only take a few seconds and it's similar enough to
               | corporate SSO setups. But yes, polish is everything for
               | those final few steps. The actual backbone is pretty
               | straightforward, though.
        
             | politician wrote:
             | If the IT team is treating servers like pets, then this is
             | a problem. If they aren't amateurs, then it's not a problem
             | to update the script and push the configuration out to all
             | of the servers.
        
             | renewiltord wrote:
             | You could just get your authorized keys via an
             | `AuthorizedKeysCommand`. It will make it slower to load but
             | even something as simple as `AuthorizedKeysCommand aws s3
             | cp s3://myprivatebucket/authorized_keys - | grep USER` will
             | do the trick. Then revocation is just deletion from the
             | file and adding is just addition to the file.
        
       | IgorPartola wrote:
       | If this is true then why does ssh not get set up this way by
       | default?
       | 
       | Honestly what I would prefer is a better (and faster) version of
       | monkeysphere [1]. That was honestly the most natural-feeling
       | solution to SSH security.
       | 
       | (1) https://www.systutorials.com/docs/linux/man/1-monkeysphere/
        
         | dsr_ wrote:
         | Because it's not suitable for everyone in all circumstances.
        
           | IgorPartola wrote:
           | So if I am not using certificates them I am not doing it
           | wrong? :)
        
       | mnd999 wrote:
       | Aside from being a pain to set up, what's wrong with using GSSAPI
       | / Kerberos? (within an org)
        
         | zengargoyle wrote:
         | Yeah, my org had a Kerberos setup for some DB/middleware/CLI
         | tools. I inherited a bunch of random script tools that took the
         | whole typing in passwords for everything. Screw that, first
         | thing I did was write a Kerberos module and set up a Kerberos
         | keystore. Presto, no more typing passwords for everything.
         | Typically it ends up boiling down to whether or not the backend
         | thing you're talking to supports Kerberos, many don't and only
         | have the whole SSL thing. A well setup Kerberos was much nicer
         | to work with than chained SSL certs.
        
         | YATA0 wrote:
         | This typically still involves typing in your domain username
         | and password, no?
        
         | en4bz wrote:
         | Kids theses days hate using technology that's older than
         | themselves? Kerberos is amazing and it make me sad to see all
         | these people reinventing stuff that was solved 30 years ago.
         | The number of companies selling SSO and doing it poorly (see
         | this weeks okta hack) is unfortunate.
        
           | risson wrote:
           | 21yo here, I think Kerberos is bloody awesome, but then I was
           | introduced to it by a somewhat old timer, who showed me its
           | benefits when properly integrated in the company.
        
       | nix23 wrote:
       | If you trust a third party you doing ssh wrong :)
        
       | temp8964 wrote:
       | There are cases where if you leave SSH enabled then you are doing
       | it wrong. If you only need SSH to do a few things once a while,
       | you should just turn it off, and only turn it on when needed. In
       | these situations, I feel it's OK to just use password to login.
        
         | Enginerrrd wrote:
         | Honestly, I agree. My ssh passwords aren't going to be brute-
         | forced anytime this century. It's also pretty easy to put a
         | fake sshd on port 22 and set the real one to some other port
         | (preferably one low enough to still require root privileges
         | though).
         | 
         | I don't have encrypted drives on all my devices. I don't want
         | to have to worry about what could happen if one of those gets
         | lost/stolen. I'd rather not leave keys or certificates lying
         | around.
         | 
         | Also, things sometimes go wrong and I need to get access to a
         | server from a device I've never been on. It's nice to be able
         | to do that. Passwords do that.
         | 
         | To be fair, I usually have a single VPS which I keep as locked
         | down as possible that has VPN access to the server I really
         | need. The VPS doesn't even need to be running most of the time.
         | So I can spin it up to get access to the VPN, then ssh into the
         | server with a password. If the VPS gets compromised, the VPN
         | alone won't give an attacker immediate access to the server
         | like it would if I left keys / certificates on there. I have to
         | trust the VPS, and if it gets compromised without me noticing,
         | and I then log in to my server, yeah, I'm SOL, but certificates
         | don't solve that problem.
        
         | remram wrote:
         | How do I turn it on if I can't SSH into the machine? In most
         | cases, SSH is what you use to do management tasks.
        
           | temp8964 wrote:
           | Many hosting services have a control panel to do this on VPS.
           | Some hardwares like Synology also have web interface to do
           | this.
        
       | defanor wrote:
       | > This makes it operationally challenging to reuse host names. If
       | prod01.example.com has a hardware failure, and it's replaced with
       | a new host using the same name, host key verification failures
       | will ensue.
       | 
       | A new machine should just have a new name. If one really wants to
       | pretend that it's the old one, they'd better really copy it,
       | including the keys. But even skipping that, sorting this out
       | doesn't seem like a big deal (at least at a small scale; I
       | suspect the article makes more sense in some scenarios than in
       | others).
       | 
       | > Curiously, OpenSSH chooses to soft-fail with an easily bypassed
       | prompt when the key isn't known (TOFU), but hard-fails with a
       | much scarier and harder to bypass error when there's a mismatch.
       | 
       | Seems to me like a sensible behaviour for TOFU, not sure what's
       | curious about it. Sounds like it implies that an unknown key is
       | at least as bad as a different-than-known key, but that sounds
       | wrong in context of TOFU.
       | 
       | > Once the user completes SSO, a bearer token (e.g., an OIDC
       | identity token) is returned to the login utility. The utility
       | generates a new key pair and requests a signed certificate from
       | the CA, using the bearer token to authenticate and authorize the
       | certificate request.
       | 
       | So the weakest point will likely be the SSO and the related
       | infrastructure, instead of SSH and actual keys, and you'll
       | probably depend on third-party services and/or custom/uncommon
       | self-hosted infrastructure. Likely with a SPOF too. Doesn't sound
       | good in general.
       | 
       | It probably does make sense in some organizations, but this
       | particular setup doesn't seem to apply to all SSH uses, and to
       | justify the title.
        
         | tptacek wrote:
         | Again, the alternative to SSO's single SPOF is many SPOFs for
         | each individual engineer with access.
        
           | defanor wrote:
           | How so? Are you picturing that alternative as regular records
           | in ~/.ssh/authorized_keys, or something else?
        
           | otabdeveloper4 wrote:
           | Or you could automate key distribution and revocation instead
           | of giving away the keys to your little kingdom to Google.
           | 
           | (Let's not pretend that "SSO" doesn't mean "let Google or
           | Microsoft handle password storage for me".)
        
             | tptacek wrote:
             | No:
             | 
             | (1) Authentication bypass to your email service is already
             | game-over for almost every serious company.
             | 
             | (2) Google (if that's what you're using) has better MFA and
             | access control than what you're going to roll yourself.
             | 
             | (3) Having multiple sources of truth for authentication is
             | a corpsec nightmare, and companies that don't have that
             | invariably wind up accidentally persisting access for
             | departed team members or contractors, and, worse, no single
             | place to consult for a reliable catalog of who has access
             | to what, which is why if you poll CISOs at large-ish tech
             | companies, they'll universally tell you than one of the
             | first 5 things they did when they took over was get SSO
             | stood up.
             | 
             | (4) The "automated key distribution and revocation" system
             | you roll yourself will be jankier and less safe than the
             | certificate-based systems that already exist.
             | 
             | (5) Because that automated key distribution and revocation
             | system does not in fact exist, what you're really saying is
             | that you're going to live with developers having long-lived
             | keys on their laptops.
             | 
             | If you don't trust Google, set up Shibboleth or something;
             | the Google stuff is a sideshow. But the idea that you
             | should manage SSH authentication separately from the rest
             | of your authentication is pretty unserious. I spent about 4
             | years, recently, parachuting into dozens of mid-sized
             | startups, all of them clueful, and except for the teams
             | that had SSO-linked SSH access, SSH management was
             | invariably a total nightmare. The "just manage SSH
             | directly" approach is, empirically, a failed model.
        
       | tomxor wrote:
       | > What you're supposed to do is verify the key fingerprint out-
       | of-band by asking an administrator or consulting a database or
       | something. But no one does that.
       | 
       | Actually they do where I work, pretty much every time, and I
       | didn't ask them to. Eyebrows will raise even higher if I forget
       | to notify everyone I replaced a server at a domain causing a new
       | signature (resulting in the scary "possible MITM attack"
       | message). This is a good thing, but I should probably make it
       | more efficient by publishing the fingerprints.
       | 
       | Although the article does point out this specific disadvantage of
       | domain reuse with known_hosts. I can see why this solution could
       | make things easier at larger scales.
        
         | toast0 wrote:
         | > I should probably make it more efficient by publishing the
         | fingerprints.
         | 
         | When I last ran a fleet of servers, I published the known hosts
         | files via git, and strongly suggested using that in new user
         | documentation (you needed to setup ssh_config for our
         | jumphost/bastion anyway, may as well link to the known host
         | keys). There's a tool to generate the files that comes with
         | openssh, iirc.
        
       | 0xdeadb00f wrote:
       | > Users are exposed to key material and encouraged to reuse keys
       | across devices. Keys are trusted permanently, so mistakes are
       | fail-open.
       | 
       | What? This just sounds like you're doing it all wrong then
       | blaming the tools.
       | 
       | Maybe I'm just thinking about it from my POV as a mainly hobbyist
       | user of SSH, but I rotate my keys (not as frequently as I should,
       | but I do), and I remove old ones from .authorized_keys files and
       | I use different keypairs for different servers sometimes too. I
       | never copy keys to other devices - I ssh-keygen on every device,
       | and add my pub key to the authorized_keys file manually.
       | 
       | It isn't even "hard" to do it this way. Sure, it doesn't "scale"
       | for big corps - but I don't need it to scale, so I'm not "doing
       | SSH wrong" by not using certificates.
        
         | [deleted]
        
         | metalliqaz wrote:
         | It's more effort than I would like. I should not be bothered
         | with updating hidden config files containing data that is
         | mostly not human readable. I want to prove my identity once and
         | let the system take care of the rest.
        
           | 0xdeadb00f wrote:
           | Well that's fair enough, but I take issue more with the title
           | of the post telling me I do "ssh wrong", because I can't be
           | bothered to setup a CA and fuck around with certificates.
           | 
           | To me, setting up a CA is more effort than I would like.
        
             | blueflow wrote:
             | It becomes feasible when you have a larger number of key
             | pairs that are supposed to have access to the same set of
             | machines. I did it as a private person because I'm an SSH
             | nomad, using several clients with different key pairs each.
             | 
             | I agree with you, for a regular user with a single client
             | device (or two) its not worth it.
        
             | throwaway984393 wrote:
             | It's very much a use-case and risk driven decision. A
             | company should be using Teleport, which is a lot more than
             | just certificates (but they do use certs). For your
             | personal VPS or GitHub account, nobody is going to go out
             | of their way to get your SSH keys.
             | 
             | The biggest "you're doing it wrong" I see is people who
             | disable host key verification because their servers' IPs
             | change constantly. Do you want MITM?! Because this is how
             | you get MITM! Might as well use Telnet for connections.
        
         | chasil wrote:
         | I decided that I wanted to automate my rotation, so I built a
         | shell script to do it, then wrote about it.
         | 
         | If you Google "ssh key rotation" you will find my article as a
         | "featured snippet."
         | 
         | I don't think that I need a CA, as I only have personal and
         | admin keys (two sets) and I'm flipping these once a quarter.
         | Plus, I don't have expired entries in my authorized_keys (do CA
         | users ever clean out authorized_keys?).
         | 
         | https://www.linuxjournal.com/content/ssh-key-rotation-posix-...
        
       | LinAGKar wrote:
       | Too bad you can't do that with dropbear
        
         | ayushnix wrote:
         | Which is why I installed openssh-server on my OpenWrt hosts.
         | However, due to a bug [0] in openssh-server package in the
         | latest 21.02.2 release of OpenWrt, OpenSSH doesn't allow you to
         | login in failsafe mode on an OpenWrt host. Because my OpenWrt
         | host lacked a recovery mode, it was essentially soft bricked.
         | 
         | I was able to recover it using the serial port but even after
         | all this, the comfort of using SSH certificates on all of my
         | nodes was enough to keep making me use it instead of Dropbear.
         | 
         | [0]: https://github.com/openwrt/packages/issues/17833
        
       | michaelmior wrote:
       | What seems to be missing from this is how I assign permissions to
       | individual hosts. When I'm using public keys, I do so by only
       | adding the user's public key to the hosts I want them to access.
       | It seems the way that certificates were presented that adding a
       | user implicitly gives them access to all hosts using the same CA.
       | I'm sure there's a solution to this problem, but does anyone have
       | any pointers?
       | 
       | For example, I want hostA and hostB to use the same CA. But some
       | users should only have access to hostA but others should only
       | have access to hostB. Others may have access to both.
        
         | gouggoug wrote:
         | Simply don't create a user account for these users on the hosts
         | they shouldn't access.
        
         | blueflow wrote:
         | Set the principal name in the Certificates to the name of the
         | users team.
         | 
         | On the server side, set the AuthorizedPrincipals* for root to
         | the list of allowed teams.
        
         | tener wrote:
         | For scalable, centrally managed solution see:
         | https://goteleport.com/docs/access-controls/reference/#rbac-...
         | 
         | Disclaimer: day work.
        
       | rantanplan wrote:
       | Yes, SSH certificates are the way to go and pretty easy to set
       | up. But what these articles fail to address is the user
       | management aspect.
       | 
       | For the SSH certificate to be accepted, the unix user must first
       | be present on the system. As far as I can understand, FreeIPA(or
       | similar LDAP systems) cannot be used in conjunction with SSH
       | certs. Whereas SSH keys are supported by these systems.
       | 
       | Can anyone provide any insight/experience with this?
        
         | blueflow wrote:
         | Its not the username that needs to match, its the principal.
         | You can allow any principal for the root user, for example.
         | 
         | You can define principals when allowing a CA via
         | authorized_keys, or you can configure allowed principals
         | globally using sshd_config directives like
         | AuthorizedPrincipals* .
        
         | dsr_ wrote:
         | In most circumstances, you want these two things (user is
         | authorized on the system, user can be identified and
         | authenticated) to be different. Having a process that creates
         | the user on system in order to authorize them to login is
         | pretty similar to all your other configuration management
         | tasks.
        
         | ehnto wrote:
         | I imagine it's a matter of automating the certificate insertion
         | on the target servers when it's updated on the user's account
         | in the LDAP server. In other words, it depends entirely on your
         | systems and how far your administration is willing to go to
         | automate it.
        
       | withinboredom wrote:
       | > brew install step
       | 
       | The audacity of mac users who think everyone uses a mac.
        
         | [deleted]
        
         | dljsjr wrote:
         | Brew has been available on Linux for a long time and works very
         | very well.
        
           | withinboredom wrote:
           | There are other OSes than Linux and MacOS, in fact, one of
           | those other OSes is far more popular than both of those put
           | together: Windows. It has built-in openSSH support since
           | 2018.
        
             | tomrod wrote:
             | I'd argue windows isn't terribly popular, in that popular
             | means "people like it more than not", but rather is used
             | often, if grudgingly, by folks who have experienced
             | alternatives or are unaware objectively better options
             | exist.
        
               | withinboredom wrote:
               | I use Linux (baremetal on a System76 machine, and WSL)
               | and Windows daily. I can't stand OSX. As in literally
               | want to trash it and the device it's installed on within
               | 10m. It's a beautiful OS, but as a windows manager, it's
               | literally unusable; at least with a QWERTY keyboard. Like
               | what genius put cmd+q right beside cmd+a, with cmd+q not
               | even prompting the user? I could rant for days... so I'm
               | just going to stop.
        
               | tomrod wrote:
               | I also use Linux -- when clients require OSX on furnished
               | equipment I cringe a bit :)
               | 
               | There are orders of magnitudes of UX difference between
               | how poor Windows is versus OSX and Linux, in my mind. OSX
               | is still worse than Linux, in part because of the
               | inconsistency of key mappings and lack of options to make
               | things consistent.
        
             | dljsjr wrote:
             | And they have an open issue for producing a chocolatey
             | package: https://github.com/smallstep/cli/issues/365
        
           | loudmax wrote:
           | Sure, but if you're not on a Mac, brew shouldn't be the first
           | tool you reach for. You can use brew if you've transitioning
           | from Mac and you need to time to get acclimated, or if the
           | software hasn't been integrated into an the local OS's
           | package management system yet.
           | 
           | Having said that, `brew` appears once in passing in the
           | entire article. This article is almost completely about ssh
           | without regard to platform. There's nothing "audacious" about
           | suggesting users use brew, so the anti Apple sniping here is
           | entirely unwarranted.
        
         | ARandomerDude wrote:
         | Bro you almost made me spit out my latte.
        
         | [deleted]
        
         | jdoss wrote:
         | I work at smallstep and we support a wide range of Linux
         | distros already. https://smallstep.com/docs/step-
         | cli/installation
         | 
         | One of the things on my punch list is to get step CLI into
         | upstream Fedora, Debian, and Ubuntu to make installation a bit
         | more easy.
        
         | wintermutestwin wrote:
         | The same audacity of how tos with "apt get" while lacking any
         | distro specific language.
         | 
         | Linux desperately needs a standard for writing how tos. Even
         | better would be definitive how tos for every common thing a
         | user might need to do being posted and maintained on a distro
         | owned site.
        
         | tomrod wrote:
         | Right? Most folks just need dnf.
        
         | [deleted]
        
       | hedora wrote:
       | Ughh. Go read up on ACME (Let's Encrypt). Unless you run your own
       | Certificate Authority root, or configure things very carefully,
       | using TLS certificates grants host level access to your DNS
       | provider, and every organization that reliably routes external
       | traffic to your host.
       | 
       | To what end? The threat model rekeying tries to protect against
       | involves compromised authenticated client machines.
       | 
       | Once you have those, the attacker has shell on the server, and
       | it's game over. There are these things called "advanced
       | persistent threats" that have been in the news a lot already.
        
         | remram wrote:
         | This article has nothing to do with Let's Encrypt, ACME, TLS,
         | or DNS.
        
       | xrd wrote:
       | This article was very illuminating. And, I wish it was written
       | like this:                 To really secure your SSH server, do
       | these three things:       1. Setup a trusted authority       2.
       | Use ssh certificates       3. With ssh certs, you can easily add
       | MFA for logins expire until you reauth.       Now that we've
       | established this, here are the gory details.       Lorem ipsum,
       | lorem ipsum, lorem ipsum.
       | 
       | It's a great narrative, but I wish I knew the big payout in
       | advance.
        
         | zepearl wrote:
         | I agree, something like that. From what I've experienced it's
         | (for me) a common problem in that realm:
         | 
         | a lot of blah-blah that makes my eyes glaze/lose focus before
         | getting to the core/overview.
         | 
         | ~15 years ago, when the company I'm working for first
         | implemented PKI, I needed something like 100 hours of "help"
         | from local security engineers and the external software-
         | vendor's programmers to understand how that worked and what the
         | SW was supposed to do. After that, explaining to colleagues at
         | least on a high level how that works became a matter of
         | minutes.
         | 
         | To be fair towards myself, even the local scrty eng gurus
         | (relaxed, as they themselves didn't have to "deliver" anything)
         | and the external vendor's programming gurus (hardcore, as
         | nothing would be paid if the SW could not be implmemented) were
         | often absolutely not understanding each other, so I ended up
         | becoming their unofficial mediator/translator:
         | 
         | if I felt like an obvious question was not dared to be asked by
         | one of the parties then I sacrificed my self-esteem to dare
         | asking it, if "even I" did manage to understand some concept
         | then both parties were supposed to get it as well and if not
         | then at least I could act as gateway to explain/expand offline
         | :P
         | 
         | It was an interesting time - not nice nor very bad (a little
         | bit bad, as we had of course an implementation deadline), but
         | at least I learned a lot, as well in the area of social skills
         | :)
        
         | michael_j_ward wrote:
         | Is that what I get out of the box using tailscale for ssh?
         | 
         | https://tailscale.com/kb/1009/protect-ssh-servers/
        
           | tptacek wrote:
           | Two examples of things Tailscale doesn't give you for this
           | usage model that SSH CAs can:
           | 
           | * Transcript-level audit trails for what people are actually
           | doing on SSH sessions.
           | 
           | * Differential access to different groups of users to the
           | same machines.
           | 
           | Tailscale and SSH CAs work together nicely: require
           | membership in the right Tailscale group to talk to SSH at
           | all, thus tying access to SSH to your (e.g.) Google login and
           | MFA requirement, and use something like Teleport for the
           | actual SSH login, to get the audit log, group access, and an
           | additional authentication factor.
        
           | tener wrote:
           | Tailscale (and other similar solutions) works on the network
           | level. This is not a bad idea in itself, but SSH certs
           | operate on the application level.
           | 
           | The fact you can ping the server shouldn't mean you are
           | allowed to actually access it.
        
         | AtlasBarfed wrote:
         | MFA for ssh means you can't automate cluster-level ops. Unless,
         | well, you automate the MFA, which defeats the MFA.
         | 
         | This drives me a bit nuts about security people in the age of
         | cloudscale. They assume you don't mind MFA'ing every hour and
         | are manually doing logins and accesses for everything. Yeah,
         | uh, I need to script orchestration on several hundred machines
         | at once, and orchestrate/access on the scale of hours or even
         | days for some things like "Big Data" backups or restores.
         | 
         | If certificates are anything like SSL certs and the horrorshow
         | cli tools / options / management involved in those, no thanks.
         | I'd rather have an automated sshkey switchover, or for
         | stateless just routinely cycle the infrastructure with new
         | keys.
         | 
         | It's been a long time tenet of security that you want an open
         | algorithm that gets broadly and publicly challenged so you know
         | its secure. Well, in the age of the state actors, this might
         | not be the whole truth.
         | 
         | I think layering some klugy not-invented-here obfuscation atop
         | the more battletested methods is a useful and important
         | deterrent/delay. Sure someone will figure it out, but they have
         | to TRY HARD. A lot of the institutional attacks seem to be
         | based on human attacks on standardized systems, which HAVE to
         | allow human access and vectors.
         | 
         | So in AWS land, secrets manager is secure unless you get the
         | permissions. Then you have everything. But if each of those
         | secrets has some whacko obfuscation for the various apps, then
         | that is a big slowdown to the human attack vectors.
         | 
         | And the state actors? Well, even they have budgets. They'll
         | probably move on to easier targets. If a state actor is
         | motivated at targeting your company in particular, well, given
         | that they'll have malware in the firmware of your hard drives
         | and motherboards and the like, you're probably helpless.
         | 
         | Finally, what really bothers me about most security is that it
         | leaves one of the most important "canaries in the coal mines"
         | aspects of security: honeypots. Sure do the diligence on
         | securing the access, but how about some turnkey approaches for
         | setting up honeypots to detect when people are poking around?
         | Honeypots are perfect for that, because the devs only care
         | about the stuff they are working on. The intruders are doing
         | the scanning.
        
           | fishpen0 wrote:
           | Cluster-level ops really shouldn't be done by direct SSH.
           | Tools like ansible, salt, chef, puppet, etc... all can be run
           | in some form of daemon mode with a central management server
           | for a reason. You should be authenticating against the
           | management service and running your configuration or
           | automation scripts from there, not as a massive pool of CSSH
           | or whatever directly from your laptop.
           | 
           | Over the last 6 years I've worked for three different
           | companies that all universally disabled ssh and we never had
           | troubles running management scripts or tooling.
        
             | outworlder wrote:
             | Until the daemon breaks and someone has to get in with SSH
             | anyway? Chef is notorious for doing this. Or at least it
             | was before we got rid of it. Some random script would break
             | and then the run would be incomplete. And we couldn't just
             | fix cookbooks as the run wouldn't complete.
             | 
             | Some deployments make the daemon approach (that phones
             | home) difficult. Such as management in a corporate network.
             | It's easy to configure AWS and the like to accept requests
             | from well known corporate gateways. It's not as easy to
             | make them from the outside the corporate network in. And
             | even when that's doable, different cloud providers and
             | regions make it difficult. You end up having a bunch of
             | chef (or similar) servers scattered around.
        
               | fishpen0 wrote:
               | In the rare case this happens we still don't use raw SSH.
               | We rely on something identity-driven like SSM in AWS or
               | IAP in GCP to initiate the tunnel.
        
       | JonChesterfield wrote:
       | The 'ideal ssh flow' involves a program that I think the author
       | has written and a website login. Do these certificates require an
       | existing system running single sign on or similar to hand out
       | access to other machines, in single point of failure fashion?
        
       | cestith wrote:
       | The article does a pretty good job explaining advantages of
       | certificates. It overlooks the existence of solutions to some of
       | the individual problems it mentions with keys, though. Tools like
       | sssd exist, with which a way for the end user to keep their
       | public key updated in a directory takes care of key distribution
       | and at the same time limits who can use sudo, all without
       | changing any files per-user on the server.
        
       | tomrod wrote:
       | Rant engaged. As a person who feels responsible for ensuring what
       | I build is secure, the security space feels inscrutably
       | defeating. Is there a dummies guide, MOOC, cert, or other
       | instructional material to better get a handle on all these
       | things?
       | 
       | SSH keys make sense. But certificates? Is this OIDC, SAML, what?
       | Is it unreasonable to request better and deeper "how to do {new
       | security thing}" when PKI is a new acronym to someone? Where can
       | I point my data science managers so they can understand the need
       | and how to implement measures to have security on PII-laden
       | dashboards? As so on.
        
         | chefandy wrote:
         | My needs may be entirely different than yours and I don't want
         | to downplay the importance of security, but...
         | 
         | Security writing in "engagement" obsessed media yields lots of
         | people screaming "FIRE!" whenever seeing something even
         | theoretically flammable, and bandwagoneers-- already imagining
         | their hair is on fire-- lambasting everyone not immediately
         | evacuating for being careless about fire safety. It reminds me
         | of politicians being 'tough on crime'-- they reflexively jump
         | at opportunities to tighten the screws regardless of its
         | necessity or efficacy. It's an emotional response involving
         | self-image, peer pressure, and fashion rather than rational
         | cost benefit analysis.
         | 
         | Perfect is the enemy of good. Attacking every theoretical
         | threat like an international bank's network admin yields no
         | practical benefit for most. Not _nobody_ but _most._ If this
         | TLA is new to me, there will be another new one that people
         | will lambast me for not knowing in a couple of years, max.
         | 
         | For me, this problem was a better fit for the Wizard of Oz than
         | a security education resource-- what I really needed was the
         | right frame of mind rather than learning the implementation
         | details of every incremental certificate authority update.
         | 
         | I evaluate my attack surfaces and reduce them if I can,
         | evaluate the real importance of keeping what I'm protecting
         | secret, implement standard precautions and architecture to
         | mitigate those risks, pay attention to the systems, pay
         | attention to new vulnerabilities, and re-evaluate upon changes.
         | The process is technology-agnostic and only requires you deep
         | dive into the stuff you need to know without feeling like you
         | need a new certification ever 6 months to run your company's
         | CalDAV server.
        
           | scottLobster wrote:
           | Relevant article: How I learned to stop worrying (mostly) and
           | love my threat model
           | 
           | https://arstechnica.com/information-
           | technology/2017/07/how-i...
        
         | throwaway984393 wrote:
         | It's not unreasonable, but we need some kind of universal
         | knowledge base for tech stuff. Security is just one of many
         | inscrutable topics in tech where you need weeks of research to
         | understand the best practices.
        
         | blueflow wrote:
         | I can surely recommend reading into SSH certificates using the
         | ssh-keygen manpage. No any extra tools required.
         | 
         | I sign SSH certificates for all my keypairs on my client
         | devices, principal is set to my unix username, expiry is some
         | weeks or months.
         | 
         | The servers have my CA set via TrustedUserCAKeys in sshd_config
         | (see manpages). SSH into root is forbidden per default, i SSH
         | into an account with my principal name and then sudo or doas.
         | 
         | My gain in all of this: I have n clients and m servers. Instead
         | of having to maintain all keys for all clients on all servers,
         | i now only need to maintain the certificate on each client
         | individually. If i loose or forget a client, its certificate
         | runs out and becomes invalidated.
        
           | westurner wrote:
           | `ssh-keygen` #Certificates: https://man7.org/linux/man-
           | pages/man1/ssh-keygen.1.html#CERT...
           | 
           | "DevSec SSH Baseline" ssh_spec.rb, sshd_spec.rb
           | https://github.com/dev-sec/ssh-
           | baseline/blob/master/controls...
           | 
           | "SLIP-0039: Shamir's Secret-Sharing for Mnemonic Codes" https
           | ://github.com/satoshilabs/slips/blob/master/slip-0039.m...
           | 
           | > _Shamir 's secret-sharing provides a better mechanism for
           | backing up secrets by distributing custodianship among a
           | number of trusted parties in a manner that can prevent loss
           | even if one or a few of those parties become compromised._
           | 
           | > _However, the lack of SSS standardization to date presents
           | a risk of being unable to perform secret recovery in the
           | future should the tooling change. Therefore, we propose
           | standardizing SSS so that SLIP-0039 compatible
           | implementations will be interoperable._
        
           | XorNot wrote:
           | Expiries are not protection against compromise.
           | 
           | Compromises happen in seconds - milliseconds, and once they
           | do they will establish persistence. Expiry systems _do not_
           | and have never been protection against compromise. They 're
           | an auxiliary to revocation systems to let you keep revocation
           | lists manageable.
           | 
           | If you don't _have_ revocation lists, or your number of
           | changes is small, you should go ahead and just set your
           | credential expiries to whatever you want - infinity, 100
           | years, whatever - it won 't make the slightest bit of
           | difference.
           | 
           | Particularly in the case when they're protecting sudo user
           | credentials, they're no defense at all.
        
             | digitallyfree wrote:
             | Yeah the lack of mentioning a CRL at all really stood out
             | when reading this. I actually didn't know about SSH
             | certificates until I saw this article (I always assumed
             | that SSH did not support this), but do run my own CA and
             | authentication for internal web services, EAP-TLS, and VPN.
             | The CRL is your first line of defense in the sense that it
             | blocks the use of that credential instantly when it is
             | revoked.
             | 
             | I will argue though that the use of a short expiry produces
             | slightly better protection than no expiry at all. If an
             | employee leaves the company (with no CRL in place) and
             | their certs expire in 16 hours, then unless their
             | credentials are stolen in that timeframe your systems are
             | still safe.
             | 
             | Likewise, if a CRL is in place and credentials are stolen
             | without you being aware of it, the expiry still provides a
             | form of buffer if the stolen credentials end up being used
             | after the cert expires. In this case the expiry would
             | trigger before you realised that credentials were stolen
             | and updated the CRL. Now yes compromises can happen in
             | seconds, but that's not in every single case.
             | 
             | That being said I definitely agree that the expiry is not a
             | subsitute to a CRL and any certificate system should have
             | revocation systems in place. In the end you really should
             | have both a CRL and expiry date if possible.
        
               | blueflow wrote:
               | Rookie mistake: SSH has no CRL, it has an KRL.
               | 
               | And its actually a separate thing since it operates
               | largely independently from the CA.
               | 
               | I have one in place. Used it once to terminate access for
               | someone.
        
               | OrvalWintermute wrote:
               | It sounds like you could be making the rookie mistake
               | instead by not reading what he/she actually wrote.
               | 
               | > Yeah the lack of mentioning a CRL at all really stood
               | out when reading this. I actually didn't know about SSH
               | certificates until I saw this article (I always assumed
               | that SSH did not support this), but do run my own CA and
               | authentication for internal web services, EAP-TLS, and
               | VPN. The CRL is your first line of defense in the sense
               | that it blocks the use of that credential instantly when
               | it is revoked.
               | 
               | This sounds like he/she is running an x509 CA. He/she is
               | generating certs for various use-cases.
               | 
               | It is possible to use x509 certs with SSH of course, and
               | so he/she could leverage his/her pre-existing CA for that
               | function.
               | 
               | Given above context CRL is completely accurate. And, KRL
               | is not.
        
               | egberts1 wrote:
               | No, SSL CA certificate are in no way like OpenSSL CA
               | issued ones.
               | 
               | The format of the certificate have diverged between two
               | ecospheres about a decade ago.
        
               | imwillofficial wrote:
               | If you didn't know about SSH certs, you shouldn't be
               | giving advice. You should study the fundamentals
        
               | OrvalWintermute wrote:
               | I think you may also have missed the context that he/she
               | used, as they described running an x509 CA first.
               | 
               | In an organizational context, many organizations are not
               | going to jump to creating a novel CA type (SSH CA) when
               | in fact regular x509 CAs are well known and the basis for
               | much security, and many in regulated industries are using
               | them already.
               | 
               | Additionally, given that he/she is running an x509 CA,
               | telling someone with that experience to study the
               | fundamentals is not very polite. It assumes the author of
               | the comment is not educated, but the very description of
               | his/her use-cases are not simplistic ones.
               | 
               | Engineering is all about tradeoffs after all.
        
               | imwillofficial wrote:
               | That's a fantastic point. Mea culpa
        
               | blueflow wrote:
               | Your pronoun thing makes your text painful to read.
        
               | jjulius wrote:
               | ... it genuinely pains you to read "he/she"?
        
               | [deleted]
        
               | OrvalWintermute wrote:
               | I just didn't want to assume gender, and didn't want to
               | go through comment history in order to find it.
        
               | digitallyfree wrote:
               | I'm not familar with SSH certificates, but I do know the
               | fundamentals of certificate-based authentication. If you
               | don't have a way to revoke the cert, then the server will
               | assume that your properly signed unexpired certificate is
               | valid. You will need some way to let the server know that
               | the previously issued cert is not valid anymore.
               | 
               | This is how this type of authentication works, and the
               | article did not address the important case of wanting to
               | revoke a user's credentials.
        
             | NovemberWhiskey wrote:
             | I'm not sure I understand the point here: are you saying
             | that a CRL _is_ an effective protection against compromise?
             | If so, how exactly does that work?
        
             | tomrod wrote:
             | To connect back on my rant -- isn't it amazing the
             | disparity of thoughts around security best practices? How
             | does someone who knows next to nothing become a reliable
             | security professional if even the security professionals
             | disagree on fundamentals?
        
               | kafkaIncarnate wrote:
               | The fundamentals are that you need to exceed O(2^N) > 80
               | bits roughly in complexity of your keys. Adding some
               | padding to that is a good idea because some algorithms
               | can be simplified in theory (for instance AES-128 is
               | actually simplified down to like ~118 already through
               | known math).
               | 
               | This is for symmetric encryption, and for asymmetric the
               | equivalent is ~1024-bits, so padding it up to 2048-bits
               | is generally the "minimum" for RSA, and some of that math
               | is advancing too so bumping it to 4096-bits isn't a bad
               | idea. If you want to be quantum proof, RSA will be broken
               | so moving to something else like EC is nice. AES would be
               | halved O(sqrt(N)), so AES-128 becomes the equivalent of
               | AES-64, so if you want to be quantum proof there you need
               | to jump up to AES-256 (unless you are using XTS/tweak
               | mode, in which case AES-512). Keep in mind quantum also
               | is not exactly short term practical to accomplish at the
               | moment.
               | 
               | You can use whatever technology to accomplish that
               | complexity, be it passwords, SSH keys, or SSH certs.
               | Anything else is just technology architecture noise.
               | Passwords absolutely can clear the O(2^N) > 80 bit
               | threshold. It's just about bytes, and how you store them.
               | 
               | Nobody is going to be brute forcing a sufficiently
               | complex password over the network anytime soon unless it
               | isn't actually random but some default password that
               | looks random.
               | 
               | Just look at the title of this post: "If you're not using
               | SSH certificates you're doing SSH wrong". It's just
               | completely devoid of environment issues, user issues,
               | datacenter issues, and reeks of elitism. There is no "one
               | true way" despite people's insistence that they are the
               | arbiters of truth. I keep reading here about "you should
               | just use serial over network instead of SSH!" but fail to
               | read about how those serial over network connections are
               | usually less secure than SSH itself.
               | 
               | Best practices guides have gone off the rails. They are
               | generally good guidelines, but you have to make sure you
               | are taking into account your own environment and user
               | needs and take them with a grain of salt. Learn for
               | yourself, and read raw facts from real cryptographers and
               | people in the field. Don't take best practices guides as
               | absolute truth, but learn from them.
               | 
               | How does one become a security professional? Maybe not
               | with one of those "become a security professional in 30
               | minutes" packages then start a blog about how everyone
               | isn't conforming to their tiny worldview. No matter what
               | it'll take >10 years with actual experience, just like
               | any profession. One has to start from the bottom and make
               | their way up. Most environments are too complicated for
               | any "one size fits all" solution:
               | 
               | https://xkcd.com/927/
               | 
               | EDIT: Further discussion on this here is interesting. The
               | top comments go all in on SSH certificates, then down the
               | line people start questioning why passwords are bad in
               | the same ways. A lot of the "SSL certificate" push
               | theorized here from their perspective seems to come from
               | VPN providers that need it from lesser skilled
               | clients/users (think, people who bought VPNs off YouTube
               | video recommendations):
               | 
               | https://arstechnica.com/information-
               | technology/2022/02/after...
        
               | OrvalWintermute wrote:
               | > You can use whatever technology to accomplish that
               | complexity, be it passwords, SSH keys, or SSH certs.
               | Anything else is just technology architecture noise.
               | Passwords absolutely can clear the O(2^N) > 80 bit
               | threshold. It's just about bytes, and how you store them.
               | 
               | I always try to assume breach in my thought processes,
               | but I recognize that this lead to overengineered
               | solutions because sometimes the mitigation is not worth
               | the cost.
               | 
               | > Just look at the title of this post: "If you're not
               | using SSH certificates you're doing SSH wrong". It's just
               | completely devoid of environment issues, user issues,
               | datacenter issues, and reeks of elitism.
               | 
               | I think this is an excellent point you make. There are a
               | few different ways to use SSH securely and I probably
               | lean a little towards the x509 and other alternatives,
               | given the established base of x509 within my industry.
               | 
               | I don't use SSH certificates at work because they really
               | don't make sense for me when I am using a strong
               | credential already (HSMs)
               | 
               | > There is no "one true way" despite people's insistence
               | that they are the arbiters of truth. I keep reading here
               | about "you should just use serial over network instead of
               | SSH!" but fail to read about how those serial over
               | network connections are usually less secure than SSH
               | itself. Best practices guides have gone off the rails.
               | They are generally good guidelines, but you have to make
               | sure you are taking into account your own environment and
               | user needs and take them with a grain of salt. Learn for
               | yourself, and read raw facts from real cryptographers and
               | people in the field. Don't take best practices guides as
               | absolute truth, but learn from them.
               | 
               | These are some other seasoned points you make.
               | 
               | I like to think about "Security Objectives". In most
               | cases I am concerned about is something secure from a
               | confidentiality, or integrity perspective. But, since I
               | also deal with an ICS/SCADA community, their context is
               | completely driven by "Availability as Paramount", defined
               | performance within an acceptable range being next, and
               | only after that, does the other objectives come into
               | play.
               | 
               | However, given the varying use-cases of machine, mobile,
               | app, connectivity basis or lack thereof (internet,
               | transient, air-gap, etc) and the limitations of each,
               | sometimes a smorgasboard of solutions are needed to
               | satisfy within constraints.
               | 
               | > How does one become a security professional? Maybe not
               | with one of those "become a security professional in 30
               | minutes" packages then start a blog about how everyone
               | isn't conforming to their tiny worldview. No matter what
               | it'll take >10 years with actual experience, just like
               | any profession. One has to start from the bottom and make
               | their way up. Most environments are too complicated for
               | any "one size fits all" solution:
               | 
               | Appreciate the words of wisdom.
               | 
               | I view security as having much in common with other
               | rapidly evolving fields of expertise. The generalists
               | becoming specialists, are now becoming sub-specialties,
               | adding fellowships, etc. When I was a young force-
               | sensitive had the good fortune to fall in with the right
               | community in which to collaborate.
               | 
               | My opinion is that many of the security communities are
               | among the most welcome, diverse, and inviting folks
               | around.
        
             | blueflow wrote:
             | If I'm not using a device for a long time, it ceases to be
             | an authorized client. This is what i want.
        
           | jas- wrote:
           | Now you have a centralized single point of failure. While the
           | ease of use is inherently obvious with the implementation,
           | if/when it does fail you will have to fall back to public
           | key/password auth anyways.
        
             | tptacek wrote:
             | Centralized single points of control are a basic goal of
             | corpsec. They trade availability for security. The
             | alternative model of individual SSH keys is theoretically
             | more highly available, but has many single points of
             | _security_ failure.
        
             | blueflow wrote:
             | Which failure mode do you mean? The CA is accessible via
             | offline means. I can walk to it and sign me a new keypair.
        
               | jon-wood wrote:
               | What happens when the building the CA is in burns down?
        
               | BenjiWiebe wrote:
               | Scan printed QR codes of your private key that you had
               | backed up off-site.
        
               | politician wrote:
               | Ideally, k-of-n key shards, stored in safety deposit
               | boxes.
        
               | tomrod wrote:
               | That's actually pretty brilliant.
        
               | danuker wrote:
               | Provided you keep said papers away from prying cameras in
               | a verifiable way, that is.
               | 
               | For more inspiration, check out the Glacier Protocol.
               | 
               | https://glacierprotocol.org/
        
               | xxpor wrote:
               | The extreme version of this is using an HSM, and putting
               | one in a safe deposit box.
        
               | danuker wrote:
               | It's not so extreme, you have to trust the HSM
               | manufacturer.
               | 
               | Try generating randomness using casino-grade dice, and
               | xor-ing it with the HSM. Maybe then.
        
               | xxpor wrote:
               | Now I'm wondering who's managed to pull off supply chain
               | attacks on dice, since I'm sure it's happened already.
        
               | tomrod wrote:
               | Thanks for the heads up!
               | 
               | I wish I'd thought about this when playing with bitcoin a
               | few months after launch and amassing an integer value
               | larger than zero. That wallet died with the hard drive.
        
               | krnlpnc wrote:
               | ddrescue may be of interest if you still have the disk.
               | 
               | That's `dd` for broken disks. It keeps a log of data it
               | couldn't read, and can keep trying to read it
               | indefinitely, it even supports a save state and can
               | resume trying again later.
               | 
               | I've recovered filesystems from several failed disks
               | using it. It's not fast though!
        
               | aspenmayer wrote:
               | Please tell me you still have the hard drive. There's a
               | chance for recovery, and I have some experience in this
               | area if you want some tips. Step 0 is always keep your
               | drives for future recovery attempts.
        
               | tomrod wrote:
               | It was dumped many, many years ago while BTC was still a
               | novelty paying for pizza in the thousands BTC per. I went
               | to see if I still had a backup of the wallet with a
               | USD:BTC spike a few years back and it was gone.
               | 
               | Life goes on, even when sad things happen :(
        
               | AitchEmArsey wrote:
               | Think of it this way: by starving the supply of that one
               | bitcoin, you have contributed in some small way to the
               | eventual loss of _all_ bitcoins through similar events -
               | speeding up the rate at which the world can move on from
               | this silly fad.
        
               | remram wrote:
               | The CA is a key, not a network service.
        
               | blueflow wrote:
               | The CA is in a gpg-encryped secrets store (pass) and has
               | a password on itself, so it can be backupped like normal
               | data to an off-site location.
        
               | hamburglar wrote:
               | Also, this doesn't apply to most real scenarios
               | (especially not "how I run my personal stuff" type
               | scenarios), but is a fun one to contemplate: what happens
               | when your customer has requirements that specify all keys
               | (including root signing keys) to be rotated at a certain
               | point in the future? Having a process for this is an
               | interesting challenge.
        
         | capn_duck wrote:
         | Practical Cryptography by Bruce Schneier and Niels Ferguson is
         | decent in that it gives a good lay of the land without diving
         | too deep in to the mathematical rigor. The first half explains
         | at a high level the concepts of encryption, key exchange,
         | asymmetric encryption, digital signatures, and lays out the
         | problem statement that PKI solves.
         | 
         | It's nice in that it will list out a bunch of available
         | encryption algorithms or hash algorithms, but at the end of the
         | chapter say "Just use this one, it's considered safe right
         | now." i.e. AES256 and SHA256.
         | 
         | Unfortunately, it mostly avoids the practical steps of web
         | security, like its not going to print out the command to type
         | in to your shell to generate an SSL signing certificate. So I
         | wouldn't recommend it if you're looking for an immediately
         | practical book to help you secure your web server. But it
         | orients you to the landscape so you have a general idea of what
         | you're trying to achieve, and can google yourself the rest of
         | the way there.
        
           | tomrod wrote:
           | I feel this is the _exact_ right thing for me right now --
           | people trusted in industry. I can follow tutorials and
           | documentation. The part where a concept is explained is often
           | missing and can be guessed at (albeit often wrongly).
           | 
           | I'll look into this and perhaps supplement with some good
           | tutorials for my developers and data scientists. I appreciate
           | your input!
        
           | tptacek wrote:
           | I don't think Practical Cryptography is going to give you
           | much of an intuition about why this article is advocating for
           | certificates.
        
           | vngzs wrote:
           | If they're willing to read a book on security design, I would
           | recommend Security Engineering, 3rd Edition [0]. It includes
           | a broad survey of what matters in the security space (rather
           | than just cryptography), and generally in sufficient depth to
           | understand how we may build secure platforms in the face of
           | adversity.
           | 
           | Also, many of the chapters are available to read for free -
           | read author's text under the cover photo.
           | 
           | [0]: https://www.cl.cam.ac.uk/~rja14/book.html
        
         | tener wrote:
         | I feel for you. Security is a complex, evolving topic, with a
         | dizzying array of concepts.
         | 
         | At work, we develop Teleport (https://goteleport.com/) to
         | provide a secure access solution that is also easy to use and
         | hard to get wrong. (Note: you cannot truly have "hard to use"
         | and "secure" access, because people will _always_ develop
         | "backdoors" that are easier to use but _not_ secure.)
         | 
         | If you are interested in some accessible writing about security
         | check out: https://goteleport.com/blog/
         | 
         | On SAML: https://goteleport.com/blog/how-saml-authentication-
         | works/
         | 
         | On OIDC: https://goteleport.com/blog/how-oidc-authentication-
         | works/
         | 
         | I can recommend the YouTube channel too:
         | https://www.youtube.com/channel/UCmtTJaeEKYxCjfNGiijOyJw
        
           | robomc wrote:
           | I'd love to use this product in my organisation, but I don't
           | want to self host, and it's really unclear what it would cost
           | me.
           | 
           | Seeing an "enterprise, call for a quote" type tier makes me
           | assume it's going to be too expensive for agency securing
           | 10-20 servers.
        
           | LambdaComplex wrote:
           | Teleport seems like a genuinely cool product.
           | 
           | With that said, the company really needs to improve its
           | interview process--my experience was downright terrible, and
           | Glassdoor shows that other people had a similar experience
        
             | GeorgeHahn wrote:
             | As a current candidate, I'd be interested in hearing more
             | about your interview experience.
        
             | stusmall wrote:
             | I'm with you. I really like the concept of their product
             | and would be interested in using it. I applied a while ago
             | but bowed out during the phone screen. There were a couple
             | strange things that came up during the short call but there
             | was one that wasn't forgivable. The post was clearly for a
             | rust developer but they were upfront that they don't have
             | any rust and are primarily a go shop. He said they put rust
             | in the job title because it helps attract smart, passionate
             | people.
             | 
             | It really put me off. I'm not dead set on developing in any
             | given language. I like rust and have been working with it
             | for a while but that isn't a deal breaker for me. The thing
             | is that if our introduction starts off with dishonesty I
             | don't have any reason to expect it to get better from
             | there. What will they mislead me about after I'm hired?
        
               | schemingguild wrote:
               | Roles shouldnt be put up as rust, but there is clearly
               | rust in the github repository. Its not a lot but my
               | understanding is the usage is somewhat growing.
               | 
               | https://github.com/gravitational/teleport/search?l=rust
        
               | stusmall wrote:
               | According to the gitlogs this conversation happened about
               | a year before those were added. We talked about this
               | pretty point blank. It was made clear that while they
               | might use rust in the future and they had rust fans
               | internally, it was a go position.
        
               | schemingguild wrote:
               | Ah right, that sucks
        
             | sparc24 wrote:
             | Their pricing is bat shit crazy. Stay far, far away.
        
               | tptacek wrote:
               | You don't need their paid product. The free (open source)
               | version is excellent.
        
               | cmiles74 wrote:
               | Is it really expensive? Their website lists a 14 day
               | trial but I don't see any pricing, just links to "Contact
               | Sales".
        
               | remram wrote:
               | Wow, a pricing page with no numbers:
               | https://goteleport.com/pricing/ Amazing
        
               | mwmaxey wrote:
               | I work at Smallstep
               | 
               | here's one with numbers: https://smallstep.com/sso-
               | ssh/pricing/#pricing
        
               | tailspin2019 wrote:
               | I share the dislike for "call us for pricing" model.
               | 
               | But in fairness there is a de facto number on this
               | pricing page, and that's zero. Their free open source
               | plan.
               | 
               | So I give them a bit of credit for that.
               | 
               | It's the companies that have no free tier or even an
               | advertised monthly cost plan at all and just a "call us
               | for pricing" that I find a real turn off (even in roles
               | where I have been a potential "enterprise" customer). So
               | I'd definitely draw a distinction between the two.
        
               | alexk wrote:
               | Sasha, CTO @ Teleport here.
               | 
               | I agree, our enterprise product is quite expensive. Let
               | me explain why:
               | 
               | * We are going through several security audits by third
               | party agencies several times per year. We are trying to
               | hire the best security agencies to audit our code and it
               | is quite expensive.
               | 
               | * We are recruiting globally and try to place our comp at
               | 90th+ percentile of the compensation as listed in
               | opencomp.com and other sources we have access to.
               | 
               | * Our sales process also takes time, and the sales team
               | employs sales engineers, sales and customer success
               | specialists to assist with deployments of such a critical
               | piece of the infrastructure.
               | 
               | * For all our employees we have wellness benefits for
               | home office improvement, personal development, healthcare
               | packages.
               | 
               | All of these factors above add up and we charge a lot for
               | building a quality security product supported 24/7 across
               | the globe.
               | 
               | However, this might not work for everyone, and we have a
               | completely free and open source version that people can
               | use without ever talking to our sales team:
               | 
               | https://github.com/gravitational/teleport
        
               | gk1 wrote:
               | Hey Sasha :) Price should be justified by value to the
               | customer, not overhead costs of the company. Even though
               | your value/benefits are listed on the site, this is a
               | good opportunity to reiterate them.
        
               | sokoloff wrote:
               | It's an intersection of those two things. Hawks can
               | profitably prey on squirrels, while lions could not.
               | 
               | There's room in the security market for $10/mo/user
               | products and room for <whatever it is that Teleport
               | charges>. If not, they'll find out in an expensive and
               | painful fashion...
               | 
               | Given that they have paying customers, their price is
               | justified to at least those customers.
        
               | alexk wrote:
               | gk1 thanks, this is a valid point!
               | 
               | Teleport solves many quite important problems four our
               | enterprise customers' infrastructure. Our users use
               | Teleport to replace secrets and static keys with short
               | lived certificates, manage certificate authorities, add
               | audit and compliance controls for access to critical
               | data, consolidate access for SSH, Kubernetes, Databases
               | and Desktops.
        
               | sparc24 wrote:
               | You have no idea how much money you are leaving on the
               | table because of your insane pricing strategy. Your
               | expenses do not scale with a customer's use. Amateur
               | mistake.
        
               | sofixa wrote:
               | It's a security product that _could_ be a huge
               | productivity gainer.
               | 
               | All competitors i can think of are also expensive.
        
               | michaelt wrote:
               | If for some bureaucratic reason you can't use SSH, which
               | the industry has been quite happily using for 20+
               | years....
        
               | sofixa wrote:
               | It's not about not using SSH, it's about:
               | 
               | * having an easy way to connect to all machines in
               | environments where not everything is built the same way
               | and on the same cloud or whatever. A big company can have
               | a ton of teams building stuff across a variety of clouds
               | and DCs. Not to mention those machines could be dynamic,
               | so you need to add discovery. Heck, there might be
               | Windows boxes here and there.
               | 
               | * having audit logs of who run what command on which
               | server when
               | 
               | * extra security features like team management, MFA, etc.
               | 
               | You can do all that (minus audit logging) with SSH, sure,
               | but it takes time and effort by the people who care least
               | ( practitioners) about those things ( security teams).
               | Buying something like Teleport or Wallix or Boundary
               | solves all those problems at once.
        
               | tener wrote:
               | Free, extremely capable open source version:
               | https://goteleport.com/teleport/download/
               | 
               | You don't get support and some other things (see:
               | https://goteleport.com/docs/enterprise/introduction/),
               | but this is not a "demo" version where you cannot do
               | actual work.
               | 
               | Kind of crazy indeed.
        
             | mwmaxey wrote:
             | I work at Smallstep
             | 
             | We are hiring and we have a non-terrible interview process
             | (and amazing culture)!
        
             | alexk wrote:
             | Hey, I'm Sasha, CTO @ Teleport. I have designed our
             | interview process and have described it here:
             | 
             | https://goteleport.com/blog/coding-challenge/
             | 
             | We are also trying to be as transparent as possible with
             | our challenges being open source:
             | 
             | https://github.com/gravitational/careers/tree/main/challeng
             | e...
             | 
             | and requirements being published here:
             | 
             | https://github.com/gravitational/careers/blob/main/levels.p
             | d...
             | 
             | I am sorry to hear that you had bad experience. Our
             | interview process is a trade-off and has one big downside -
             | it may take more time and efforts compared to classic
             | interviews. It could also feel disappointing if the team
             | does not vote in favor of the candidate's application.
             | 
             | However, if there was something else wrong with your
             | experience and you are willing to share, please send me an
             | email to sasha@goteleport.com.
        
               | tptacek wrote:
               | I like Teleport. If you're doing work samples, why is
               | your team _voting_ in favor of applications? Part of the
               | point of work samples is factoring out that kind of
               | subjectivity.
        
               | alexk wrote:
               | That's a fair question. The team votes on specific
               | aspects of implementation that can not be verified by
               | running a program, for example:
               | 
               | * Error handling and code structure - whether the code
               | processes errors well and has a clear and modular
               | structure or crashes on invalid inputs, or the code
               | works, but is all in one function.
               | 
               | * Communication - whether all PR comments have been
               | acknowledged during the code review process and fixed.
               | 
               | Others, like whether the code uses good setup of HTTPS
               | and has authn are more clear.
               | 
               | However, you have a good point. I will chat to the team
               | and see if we can reduce the amount of things that are
               | subject to personal interpretation and see if we can
               | replace them with auto checks going forward.
        
               | tptacek wrote:
               | We're a work-sample culture here too, and one of the big
               | concerns we have is asking people to do work-sample tests
               | _and then_ face a subjective interview. Too many
               | companies have cargo-culted work-sample tests as just
               | another hurdle in the standard interview loop, and
               | everyone just knows that the whole game is about winning
               | the interview loop, not about the homework assignments.
               | 
               | A rubric written in advance that would allow _a single
               | person_ to vet a work sample response mostly cures the
               | problem you have right now. The red flag is the vote.
        
               | alexk wrote:
               | That's a fair concern. We don't have extra steps to the
               | interview process, our team votes only on the submitted
               | code. However, We did not spend enough time thinking
               | about automating as many of those steps as possible as we
               | should have.
               | 
               | For some challenges we wrote a public linter and tester,
               | so folks can self-test and iterate before they submit the
               | code:
               | 
               | https://github.com/gravitational/fakeiot
               | 
               | I'll go back and revise these with the team, thanks for
               | the hint.
        
               | tptacek wrote:
               | The good news is, if you've run this process a bunch of
               | times with votes, you should have a lot of raw material
               | from which to make a rubric, and then the only process
               | change you need is "lose the vote, and instead randomly
               | select someone to evaluate the rubric against the
               | submission". Your process will get more efficient and
               | more accurate at the same time, which isn't usually a win
               | you get to have. :)
        
               | wdella wrote:
               | Disclaimer: I'm a Teleport employee, and participate in
               | hiring for our SRE and tools folks.
               | 
               | > A rubric written in advance that would allow a single
               | person to vet a work sample response mostly cures the
               | problem you have right now. The red flag is the vote.
               | 
               | I argue the opposite: Not having multiple human opinions
               | and a hiring discussion/vote/consensus is a red flag.
               | 
               | The one engineer vetting the submission they may be
               | reviewing before lunch or have had a bad week, turning a
               | hire into a no-hire. [1] Not a deal breaker in an
               | iterated PR review game, but rough for a single round
               | hiring game. Beyond that, multiple samples from a
               | population gives data closer to the truth than any single
               | sample.
               | 
               | There is also a humanist element related to current
               | employees: Giving peers a role and voice in hiring builds
               | trust, camaraderie, and empathy for candidates. When a
               | new hire lands, I want peers to be invested and excited
               | to see them.
               | 
               | If you treat hiring as a mechanical process, you'll hire
               | machines. Great software isn't built by machines... (yet)
               | 
               | [1] https://en.wikipedia.org/wiki/Hungry_judge_effect
        
               | tptacek wrote:
               | Disclaimer: this comment ticked me off a bit.
               | 
               | If you really, honestly believe that multiple human
               | opinions and a consensus process is a requirement for
               | hiring, I think you shouldn't be asking people to do work
               | samples, because you're not serious about them. You're
               | asking people to do work --- probably uncompensated ---
               | to demonstrate their ability to solve problems. But then
               | you're asking your team to override what the work sample
               | says, mooting some (or all) of the work you asked
               | candidates to do. This is why people hate work sample
               | processes. It's why we go way out of our way not to have
               | processes that work this way.
               | 
               | We've done group discussions about candidates before,
               | too. But we do them to build a rubric, so that we can
               | lock in a consistent set of guidelines about what
               | technically qualifies a candidate. The goal of spending
               | the effort (and inviting the nondeterminism and bias) of
               | having a group process is to get to a point where you can
               | stop doing that, so your engineering team learns, and
               | locks in a consistent decision process --- so that you
               | can then communicate that decision process to candidates
               | and not have them worry if you're going to jerk them
               | around because a cranky backend engineer forgets their
               | coffee before the group vote.
               | 
               | I don't so much care whether you use consensus processes
               | to evaluate "culture fit", beyond that I think "culture
               | fit" is a terrible idea that mostly serves to ensure
               | you're hiring people with the same opinion on Elden Ring
               | vs. HFW. But if you're using consensus to judge a work
               | sample, as was said upthread, I think you're misusing
               | work samples.
               | 
               | You can also not hire people with work samples. We've
               | hired people that way! There are people our team has
               | worked with for years that we've picked up, and there are
               | people we picked up for other reasons (like doing neat
               | stuff with our platform). _In none of these cases did we
               | ever take a vote_.
               | 
               | (If I had my way, we'd work sample everyone, if only to
               | collect the data on how people we're confident about do
               | against our rubric, so we can tune the rubric. But I'm
               | just one person here.)
               | 
               | Finally: a rubric doesn't mean "scored by machines". I
               | just got finished saying, you build a rubric so that _a
               | person_ can go evaluate it. I 've never managed to get to
               | a point where I could just run a script to make a
               | decision, and I've never been tempted to try.
               | 
               | I'll add: I'm not just making this stuff up. This is how
               | I've run hiring processes for about 12 year, not at crazy
               | scale but "a dozen a year" easily? It's also how we hire
               | at our current company. I object, strongly, to the idea
               | that we have a culture of "machines", and not just
               | because if they were machines I'd get my way more often
               | in engineering debates. We have one of the best and most
               | human cultures I've ever worked at here, and we reject
               | idea that lack of team votes is a red flag.
        
         | hitpointdrew wrote:
         | Check out teleport. It abstracts away the certificate bit and
         | manages it for you. You run 'tsh login' once and you get a cert
         | good for 12 hours (then you can get access to all the teleport
         | resources you are allowed to, weather that is ssh server
         | access, db access, kubernets access, etc.) I am evaluating the
         | product now and am quite impressed.
         | 
         | https://goteleport.com/
        
         | enriquto wrote:
         | > SSH keys make sense. But certificates?
         | 
         | I am equally mystified... Never understood how involving a
         | possibly malicious third party can make communication more
         | trustworthy.
         | 
         | But then again, I was also sure when I first heard about it
         | that public key cryptography was _obviously_ impossible. You
         | just could not have secret communication when everything is on
         | the open! Is there any simple explanation that we ignorant
         | people can read about certificates to get an  "aha! insight"
         | moment? For the case of public key cryptography, the moment
         | where everything snapped together was when I read the
         | mathematical description of the Diffie-Helman key exchange [0].
         | 
         | I'm not interested in how to do certificates with ssh, but on
         | what problem do certificates solve, exactly.
         | 
         | [0]
         | https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exc...
        
           | adrian_b wrote:
           | Using certificates does not provide anything useful for a
           | small private network, with a single administrator, or where
           | all the users are trusted.
           | 
           | On the other hand, they are useful for large organizations,
           | with needs for differentiated access rights and management
           | rights.
           | 
           | The centralized control over the certification authority
           | allows the delegation of restricted rights to other levels of
           | network administration.
        
           | slivanes wrote:
           | I really like: https://www.abc.net.au/news/2017-09-15/how-
           | encryption-works-...
        
             | enriquto wrote:
             | This is a (quite convoluted) explanation of public key
             | cryptography, that I already understand. My question was
             | about certificates.
        
               | HideousKojima wrote:
               | For public key cryptography my go to analogy for non-
               | technical people has always been the mailing padlocks
               | example (where the padlock is the public key, and the key
               | to unlock itbis the private key that stays with the
               | sender).
        
           | yosamino wrote:
           | One example also mentioned in the article:
           | 
           | If you connect to a ssh-server for the first time, ssh will
           | give you a warning and let you know that you have to verify
           | the fingerprint of the host key.
           | 
           | This becomes annoying when you connect to many different
           | servers and I would not trust everyone (including me) to do
           | this check correctly every single time.
           | 
           | SSH certificates solve this by having the ssh-host-key be
           | signed in a way that your ssh-client can verify and you only
           | have to add a key-signing-key to you known_hosts once.
           | 
           | Now you have to sign the ssh-host-key but you only have to do
           | it once per server as opposed to having each user having to
           | do it locally on every first connect.
        
           | tptacek wrote:
           | People talk too much about "certificates" as a good thing,
           | but they're just a means to an end. The two major goals
           | you're trying to solve with SSH:
           | 
           | (1) You want all of your authentication to route through a
           | single point of control where you can enforce group-based
           | access control, MFA, onboarding/offboarding, and audit
           | logging.
           | 
           | (2) You want the actual secrets that allow you access to an
           | SSH server not to live for a long time on anyone's laptop,
           | because it is effectively impossible to ensure that, on a
           | sufficiently large engineering team, nobody's laptop will
           | ever get compromised; there's just too many of them, and
           | developers do weird shit so the machines can't be ruthlessly
           | locked down. You want people to have SSH login secrets for
           | exactly as long as they need them for a specific server, and
           | no longer.
           | 
           | Certificates solve the problem of having dynamic access
           | control to SSH servers without having some weird system that
           | is constantly replacing authorized_keys on all your servers;
           | instead, there's a single root of trust on all the SSH
           | servers (the CA public key) and a single place that mints
           | valid certificates that enforces all the stuff I mentioned in
           | (1) above.
           | 
           | It's worth knowing here that SSH certificates are nothing
           | like X.509 certs; they're far simpler, and you could bang out
           | an implementation of them yourself in a couple hours if you
           | wanted.
        
           | dsr_ wrote:
           | A certificate is just a formatted list of attributes that has
           | been signed by a particular private key. Username, UID, GID,
           | membership in this, privilege for that, good-after date and
           | good-until date, for instance.
           | 
           | Everybody knows the public key associated with that private
           | key, so you can verify that the private key did sign this
           | list of attributes.
           | 
           | An ssh keypair is an actual public/private keypair, but a
           | certificate is just signed and encoded (but not encrypted)
           | formatted data.
           | 
           | If an ssh daemon has knowledge of a public key used to sign a
           | cert, and has been instructed to trust that cert, and all the
           | dates are good, then the ssh daemon can accept that cert as
           | proof of identity and allow a login.
        
             | enriquto wrote:
             | I understand what you mean, thanks for the explanation.
             | 
             | But why would you want to do that? What problem does it
             | solve? Just that you can connect without having a private
             | key yourself? This doesn't sound very safe.
        
               | dfox wrote:
               | The common way of managing ssh keys involves having some
               | central entity that somehow updates the authorized_keys
               | on all relevant hosts, which involves interaction with
               | all the hosts which is somehow triggered by interaction
               | with the user requesting access. With ssh certificates
               | the central trusted node only interacts with the user (by
               | signing the certificate) and does not have to update
               | anything anywhere else.
        
               | tfigment wrote:
               | It solves partly managing authorized_keys files. If you
               | have a team separate keys can be difficult to manage.
               | Shared keys are even worse. Certs can help with this if
               | you properly manage the cert signing server (like
               | hashicorp vault). All of that is currently free and open
               | source. Also can now have short expiry times if desired.
        
               | blueflow wrote:
               | You still need your own private key plus the certificate.
               | 
               | I have n clients, m servers.
               | 
               | On clients, i sign the lokal keypair with the CA key and
               | log-in via certificate. The client-side certificate
               | basically replaces the line in the server-side
               | authorized_keys. The editing stays locally.
               | 
               | On servers, i register the CA key as "Certificates signed
               | by this keypair are trustable", the authorized_keys file
               | stays empty. No further editing required.
               | 
               | During normal daywork, the CA key sits unused and can be
               | shut away.
               | 
               | Key Advantage: I don't need to edit anything on the
               | countless servers anymore.
        
               | tptacek wrote:
               | Because the keys aren't directly coupled to server
               | configurations, but rather indirected through a CA which
               | hosts the only durable key, those "private keys" users
               | have to have can be extremely short-lived, and tailored
               | for each individual access request.
               | 
               | I think people really get into trouble with SSH
               | certificates trying to reason about the properties of
               | certificates versus SSH keys versus passwords. The format
               | isn't the point; making the endpoint keys dynamic is. If
               | you built a secure messaging system that propagated one-
               | time-use SSH keys, it would address the same problem.
               | Nobody will, because certificates are easier and already
               | work, but you could.
        
         | jpgvm wrote:
         | I would say PKI (and especially the associated X509 standards)
         | is by far the least understood (or most misunderstood) part of
         | actually building secure stuff.
         | 
         | It would be nice if there was a dummies guide but I'm not
         | really aware of one. Doesn't help that most of "how to PKI" on
         | the web amounts to a bunch of unexplained cryptic openssl CLI
         | incantations.
        
           | dfox wrote:
           | It is combination of two things. X.509 is arguably overly
           | complex ASN.1/X.500 thing, but that is not the main issue.
           | 
           | Main issue is that most people do not even grasp the concept
           | of a certificate (ie. binding of public key to some
           | additional information that is signed by some other entity).
        
           | brandonmenc wrote:
           | I recommend _Security without Obscurity: A Guide to PKI
           | Operations_ by W. Clay Epstein and _Bulletproof TLS and PKI_
           | by Ivan Ristic.
           | 
           | I started working in this space a year ago (I'm on a project
           | deploying zero trust networking at a large company) and these
           | books have been invaluable.
           | 
           | https://www.amazon.com/gp/product/036765864X
           | 
           | https://www.feistyduck.com/books/bulletproof-tls-and-pki/
        
           | raxxorrax wrote:
           | Also it is a moving space. Browsers don't accept a single
           | certificate for a site anymore, you also have to have that
           | signed by a CA. You can create such a certificate yourself
           | too, but as of today you will need at least two certificates
           | for browsers to fully accept a TLS secured connection. It
           | hasn't been that long since that rule is in place.
           | 
           | So it isn't only the technicalities of asynchronous
           | encryption, there is also specific behavior of applications
           | that use certificates to prove identities.
        
         | diarized wrote:
         | Scalable and secure access with SSH @FB
         | https://news.ycombinator.com/item?id=12482212
        
         | pphysch wrote:
         | I spent an afternoon implementing a SSH cert service in Go
         | using the standard "x/crypto/ssh" package, with little to no
         | prior knowledge of SSH internals.
         | 
         | SaaS like Smallstep and Teleport are trying to middleman and
         | monetize what is actually a simple process that more developers
         | should be comfortable implementing themselves.
         | 
         | This isn't "rolling your own crypto", this is standard SSH key
         | stuff plus a bit more nuance and LOC to make things more
         | secure.
         | 
         | When you pay for those services, you are essentially paying for
         | a wrapped SSH command + dead-simple web app + someone to be
         | your CA (read: store the resulting files of `ssh-keygen`,
         | hopefully securely). And all the potential headaches of relying
         | on yet another SaaS.
         | 
         | Plenty of developers are comfortable writing a script, a simple
         | web app, and securely storing a file on the webserver. This is
         | all that is required to build SSH cert support into your
         | internal apps/tools, plus an afternoon understanding how CAs
         | work (in short, CA private key can sign any SSH public key,
         | then that SSH public key can be validated by anyone holding the
         | CA public key, no TOFU required).
        
         | michael_j_ward wrote:
         | I have the same feeling, and it motivated me to recently
         | purchase this "Bulletproof TLS and PKI" [0]
         | 
         | I haven't read it yet, so I'm posting i hope of someone else
         | giving a quick review.
         | 
         | https://www.feistyduck.com/books/bulletproof-tls-and-pki/
        
       | sigmonsays wrote:
       | Who really wants to run a CA though?
        
       | jimmaswell wrote:
       | I still don't care about any of this nonsense for my personal
       | stuff when I can avoid it. Passwords all day for me. 0 security
       | incidents in my lifetime.
       | 
       | Sucks that Github and some other things force SSH keys which are
       | just passwords except always saved to your disk so that anyone
       | who steals your laptop gets access.
       | 
       | It adds insult to injury when you try to capitulate to this
       | malarkey, generate a key in PuTTy's key generator, then Github
       | whines that the default setting isn't overkill enough and you
       | have to make a whole NEW key with some other setting. I miss the
       | good old days.
        
         | teknopaul wrote:
         | Ever don't the math on the amount of time spent entering
         | passwords? I have. Stopped using passwords for personal stuff.
         | 
         | I multiplied by cost per min for downtime in professional
         | support and the cost of typing passwords was more than my
         | yearly wage.
        
         | tener wrote:
         | I miss the good old days before passwords were a thing and
         | everyone just trusted others to behave ;-)
         | 
         | You can secure access to Github (and other places) with
         | hardware keys, e.g. from https://www.yubico.com/.
        
         | outworlder wrote:
         | > I still don't care about any of this nonsense for my personal
         | stuff when I can avoid it. Passwords all day for me. 0 security
         | incidents in my lifetime.
         | 
         | Even for personal stuff, why would you want to use passwords?
         | Keys are more secure AND more convenient. Sure you don't need
         | certificates but I don't understand how keys are more
         | 'nonsensical' than passwords.
         | 
         | Keys have more flexibility, you can use SSH Agent, you can do
         | SSH agent forwarding, etc.
         | 
         | > except always saved to your disk so that anyone who steals
         | your laptop gets access.
         | 
         | This is wrong. First of all, your laptop should have disk
         | encryption. Always. I don't care what your threat model is,
         | encrypt the disk. Second, SSH keys can (and SHOULD) have a
         | passphrase.
        
         | toolz wrote:
         | not if you encrypt your ssh keys, which is what everyone I know
         | does - then your potentially weak password requires physical
         | access to exploit while things accessible over the internet
         | can't really be brute forced.
         | 
         | further, this is even more convenient when paired with an ssh-
         | agent that will securely hold your private key in memory and
         | not allow anyone to export that key...you could dump the memory
         | but that would require root access, which again should be
         | password protected
        
           | teknopaul wrote:
           | My servers expect an environment variable to be sent too,
           | (new sshd and ssh can do this). This gives basic 2fa without
           | typing anything.
        
         | kenmacd wrote:
         | But you add a password to the key, so it's the same.
         | 
         | And not everyone saves it to disk. My ssh key is my gpg key.
         | It's stored on a yubikey and can't ever leave it. If I do a
         | `git pull` then my yubikey flashes and I have to tap it to
         | allow that connection to happen. Steal my yubikey, well you
         | can't unlock it. Hack my laptop and you can't tap the key.
        
         | raxxorrax wrote:
         | Passwords are safe if you can memorize them. It is not too hard
         | in my opinion. I also think they should always be an option for
         | any kind of auth. Maybe I want to authenticate against a system
         | but I don't want others to know my ID. For that use case a
         | password is the better solution.
        
         | krnlpnc wrote:
         | > Sucks that Github and some other things force SSH keys which
         | are just passwords except always saved to your disk so that
         | anyone who steals your laptop gets access.
         | 
         | This is the reason to encrypt ssh private keys with a
         | passphrase. If the key is leaked it's still protected by the
         | password.
         | 
         | It's a built-in feature of ssh. For an existing key downloaded
         | from a cloud provider, use ssh-keygen -p to add/change the
         | passphrase.
        
           | gizzlon wrote:
           | How hard is it to crack these passwords? Since it's local
           | (unlike the github password) I guess you can run brute force
           | attacks etc at full speed ?
        
             | 9dev wrote:
             | Yes. This is precisely why passphrases are a _bad_ idea -
             | people tend to use their easy-to-remember default password,
             | which gets compromised along the way if an attacker can get
             | their hands on the key file and throw their full processing
             | power at it.
             | 
             | SSH certificates are a solution to that problem.
        
             | confident_inept wrote:
             | That's going to depend on the length of your password.
             | Longer is more entropy and orders of magnitude more
             | difficult to 'brute force' with each character added.
        
             | kenmacd wrote:
             | Yes it's local, but also can be taken away to run on a
             | cluster. Looks like ssh-keygen is using 16 rounds of
             | bcrypt_pbkdf. My laptop just took 185ms to try a password.
             | So I guess I could run less than 10 passwords per second
             | (per core?).
             | 
             | I don't keep an ssh key on disk though. I use my gpg key on
             | my hardware security token, which gives you 3 attempts
             | before you have to unblock it with a separate management
             | password, which again you get 3 attempts at before the key
             | is entirely locked.
        
             | wombatpm wrote:
             | Pass phrase not password. You are going for length to
             | protect from brute force.
             | 
             | Soylent Green is NOT people
             | 
             | Was one of my shorter pass phrases
        
               | teknopaul wrote:
               | How many times a day did you typ that wrong!
        
               | UweSchmidt wrote:
               | I'm sceptical about the entropy of easy to remember pass
               | phrases, including negations and simple capitalizations.
               | Even when going for something like "correct horse battery
               | staple", which requires a memorization technique to
               | remember, the space of words we are realistically drawing
               | from when prompted by a shell is probably not that large.
        
               | ylk wrote:
               | That's what diceware is for:
               | https://theworld.com/~reinhold/diceware.html
        
             | stjohnswarts wrote:
             | sure, but that's why you're using a password manager that
             | lets you generate 24 character mixed everything random
             | passwords and use them easily, right? Right? Guys?
        
             | krnlpnc wrote:
             | The longer the better. A memorable sentence is a good place
             | to start.
             | 
             | ssh-agent will cache the passphrase in memory, which helps
             | avoid needing to type in a long phrase repeatedly.
             | 
             | But it's worth saying that if any private key is leaked
             | (passphrase or not), it's time to revoke it and generate a
             | new one.
             | 
             | Having a passphrase in place raises the bar from "key
             | leaked, 3rd party has access to everything" to "key leaked,
             | 3rd party has to now attempt to crack the passphrase". It
             | mitigates a very bad scenario and buys time.
        
         | jalgos_eminator wrote:
         | The github change also messed up my workflow, which involves
         | pulling/cloning my company's git repo from lots of machines,
         | many of them being short lived or disposable. Now I have to
         | save the password forced on me in a file because I'm unable to
         | memorize it easily and that made our setup less secure. Thanks
         | github...
        
       | krnlpnc wrote:
       | It depends on the scale. If a company has a handfull of hosts I'd
       | argue that deploying the full AAA and PKI systems to back cert
       | auth is doing it wrong.
       | 
       | Traditional ssh-key auth is simple and reliable, it's not until
       | you have a large, complex and diverse user base that you need
       | something more. That's why the huge fang sites use it. Every org
       | doesn't need to mimic fang.
        
         | ayushnix wrote:
         | I'm using SSH certificates to manage a few nodes in my homelab
         | and it's a pleasure to not have to deal with managing the
         | known_hosts file on my clients and authorized_keys file on my
         | servers. There's only 1 line in my known_hosts for my nodes and
         | authorized_keys doesn't even exist on any of my servers. If I
         | add a new node to my homelab, I don't have to make any changes
         | in known_hosts or authorized_keys in the existing nodes and
         | it's easy to bootstrap the same known_hosts and sshd_config
         | that I use everywhere in the new node.
         | 
         | SSH keys would make managing these few nodes a lot more complex
         | that it is.
        
         | fvold wrote:
         | Exactly!
         | 
         | I don't understand this obsession with enterprise-level
         | security on home networks and hobby projects. If you think it's
         | fun and educational to set up, then you're doing it for fun and
         | education, not security. If you're doing it for security,
         | you're basically setting up anti aircraft guns to do what a
         | drone jammer could do with way less resources spent.
        
           | hedora wrote:
           | Anti-aircraft guns can't actually take out most drones.
           | 
           | Good analogy.
        
       | el_duderino wrote:
       | Previous discussion from 2019:
       | https://news.ycombinator.com/item?id=20955465
        
       | andix wrote:
       | As long as it is not as easy as let's encrypt, it won't take off.
        
         | inetknght wrote:
         | I toyed with SSH certificates a few years ago. They seemed cool
         | and secure but, ultimately, the x509 stuff was quite (!)
         | arcane. And trying to get the IT team on board would have been
         | a nightmare.
         | 
         | I quite agree: as long as it's not as easy as Let's Encrypt
         | then it won't take off.
        
           | blueflow wrote:
           | man ssh-keygen lists me this:
           | 
           | > Note that OpenSSH certificates are a different, and much
           | simpler, format to the X.509 certificates used in ssl(8).
           | 
           | Where do you work with x.509 certificates when doing an SSH
           | ca? I use SSH certificates productively and only ever used
           | ssh-keygen commands.
        
             | inetknght wrote:
             | Like I said, it's been a few years so maybe things have
             | changed since then. But if I recall...
             | 
             | x509 supports all of the things that ssh key file format
             | does ... and also a lot more. ssh supports x.509 too so
             | that makes it "easier". When you use `ssh` to connect,
             | you'd specify -I and point to the private key and it will
             | automatically try to find a certificate file whose name is
             | identical to the identity file but with "-cert.pub"
             | appended (see `man ssh_config`). There's another option to
             | explicitly specify the key certificate file
             | `CertificateFile` but there's no short-argument version so
             | you have to use `-o CertificateFile [filename]` or add that
             | to your `~/ssh/.config` file.
             | 
             | You'd need to use `openssl` command (or, of course, any
             | openssl-like command line) to sign the your SSH key. And,
             | of course, openssl doesn't understand ssh key files so you
             | have to use `ssh-keygen` to convert the key to x509 format
             | or else generate the key using `openssl` (but again, at
             | least ssh understands x509 natively so the downside is
             | having the much longer x509 text). And that's quite the
             | arcane part: openssl cli is _fucking awful_. It doesn 't
             | follow normal command line conventions, doesn't have tab
             | autocompletion, and its documentation is obscure/difficult
             | to find, extremely terse, and difficult to even understand
             | if you're not already explicitly familiar with exactly your
             | inputs and outputs.
             | 
             | And then there's the whole process of getting your key file
             | signed. At least that process is (in general) identical to
             | having a web SSL key signed -- because the private key is
             | actually identical but the only nominal difference is the
             | format of the file that you normally think of using for it.
             | But the workflow is different because the certificate
             | doesn't get installed to somewhere that a webserver would
             | want. I had ended up creating my own Certificate Authority
             | to test with and that was yet another rabbit hole of anger
             | management.
        
               | blueflow wrote:
               | Yeah, you should re-educate yourself. Like i said, i used
               | SSH certificates managed exclusively with ssh-keygen, i
               | don't have any connection between the OpenSSL and SSH
               | cryptosystems, nor do i see a point in that. My advice:
               | Disregard any blog posts mucking SSH certs with OpenSSL
               | and take the same time to read the ssh-keygen manpage.
        
               | inetknght wrote:
               | > _Yeah, you should re-educate yourself._
               | 
               | Thanks! That sounds quite condescending. I'd educated
               | myself by reading about ssh keys in ssh and wondering how
               | to use them.
               | 
               | Perhaps you should make your own blog post describing how
               | to do it all using only ssh-keygen then.
        
               | blueflow wrote:
               | Here is my "blogpost": https://www.man7.org/linux/man-
               | pages/man1/ssh-keygen.1.html . The signing process is the
               | `ssh-keygen -I` part.
               | 
               | The source of truth (in sync with your currently
               | installed version) is right on your disk. Invoke `man
               | ssh-keygen`.
               | 
               | Sorry for my tone. I'm pissed that, every second day,
               | some person comes along and demonstrates to me that I'm
               | seemingly the only person in the world who is reading the
               | documentation on the tools we all use. Like, how do you
               | even know what you are doing?
        
               | inetknght wrote:
               | And yet: I did read the documentation and using openssl
               | was the solution I had come up with. Perhaps I was using
               | a nonstandard version of ssh. Telling people to "just
               | read the documentation" is quite condescending when
               | they're demonstrating that they've come up with a
               | different solution. It was years ago and I don't remember
               | what version I was using.
               | 
               | Indeed, using ssh-keygen for the whole process certainly
               | seems easier.
        
               | blueflow wrote:
               | I'm sorry for my harsh words.
        
           | sgarg1 wrote:
           | SSH with x.509 requires patching the SSH client and daemon.
           | It is an unofficial patch to add the functionality.
           | 
           | SSH certificates themselves are similar to x.509 but a
           | lighter version. They are supported out of the box.
        
         | [deleted]
        
         | pxeger1 wrote:
         | The USP of Step is to make setting up a custom CA as easy as it
         | is with Let's Encrypt
        
         | tfigment wrote:
         | Server setup isnt hard. No need for authorized_keys files
         | anymore as you just trust signing server instead. What is hard
         | is user experience. Sshagent doesnt support the certs. Putty
         | (most popular client on windows?) doesnt support certs. I want
         | to use them but need better client support. Vault makes almost
         | easy to create and manage certs but have to use openssh client
         | only with extra argument every time.
        
         | seanw444 wrote:
         | I didn't know people thought Let's Encrypt was easy. My first
         | foray into setting up certificates for a web server let me
         | realize how convoluted the system itself is, not just
         | specifically Let's Encrypt. I guess it's easier compared to the
         | alternatives. But holy cow I hate the process.
        
           | crispyambulance wrote:
           | yeah, it's really confusing.
           | 
           | Many folks are comfortable with jumping through the hoops
           | (following instructions) WITHOUT understanding what's really
           | happening.
           | 
           | Those of us who aren't comfortable with "just works" need to
           | plow through a enormous amount of persnickety jargon and
           | easily forgettable material to reach an understanding that
           | gives us confidence.
        
             | davegauer wrote:
             | Thank you for this reminder! It has slowly dawned on me
             | over the years that when "everyone" thinks something is
             | easy and I don't, it's often because we're comparing a
             | surface level understanding vs. a full grok.
        
       | cj wrote:
       | I'm tempted to write a competing article: "If you're using SSH in
       | 2022, you're doing it wrong"
       | 
       | There's no need to open port 22 if tools like AWS Session Manager
       | (and GCP's equivalent) are available to you.
        
         | core-utility wrote:
         | Don't forget about the no-ops "If you're opening a shell on any
         | server, you're doing it wrong (2022)" method.
        
           | dtgriscom wrote:
           | What's next? "If you're using a computer, you're doing it
           | wrong"?
        
             | kafkaIncarnate wrote:
             | "If you have data, code, or processor execution, you're
             | doing it wrong."
        
               | jandrese wrote:
               | That's really not that far off from what some people
               | think. "You should never touch your own data, that is
               | what the cloud is for."
        
             | [deleted]
        
             | geodel wrote:
             | Now this is where I want to go! I mean using technology in
             | 2022 feels so outdated. Don't we have cloud to do
             | everything for us.
        
         | GekkePrutser wrote:
         | But even if you do it through that, SSH is a much nicer
         | protocol than typing on a remote console. You get file
         | transfer, X, agent and port forwarding, terminal window scaling
         | and much lower bandwidth.
         | 
         | Also, not all servers are on cloud platforms.
        
           | twic wrote:
           | Is it possible to run Zmodem over this AWS pseudo-console the
           | way you can over SSH?
           | 
           | Not all servers are on cloud platforms, but there are
           | somewhat comparable ways to shunt a serial or out-of-band
           | management console over the network.
        
             | GekkePrutser wrote:
             | Not sure, I haven't used AWS much. Most of the web consoles
             | I have used are like VNC. So no, not really possible to run
             | anything like Zmodem over.
             | 
             | Not that I'd want to either, of course. Bringing an 1988
             | solution back to fix a conceptual 2022 problem does not
             | sound like a great fix :)
             | 
             | I understand that in some workload types you want to have
             | full autodeployment on servers, using ansible, kerberos,
             | whatever. In that case interactive login is never needed.
             | 
             | But this is a very specific subset of 'servers' in my
             | opinion. A lot of HN contributors work in this so this
             | approach may work for them but it won't everywhere.
        
         | [deleted]
        
         | onpensionsterm wrote:
         | There are actually people working on things other than CRUD web
         | apps!
        
         | martopix wrote:
         | And can I access my university's server with that? My office
         | computer from home?
        
         | throwaway984393 wrote:
         | I think you're being downvoted by the same nerds obsessed with
         | self-hosting. "But how can I self-host without ssh?!"
        
       | tacostakohashi wrote:
       | This is definitely true... and a lot of people are doing it
       | wrong.
       | 
       | This is one of those things that drives me nuts as an
       | experienced/old developer, seeing people type passwords for
       | ssh/git/whatever several times per day. Sometimes there are tasks
       | that require copying / checking some file on N servers, and these
       | people seem to think that cannot be done in a shell script
       | because the password needs to be entered interactively.
       | 
       | Then there's ssh port forwarding, X11 forwarding, etc... but its
       | amazing how many people use ssh for years without so much as
       | glancing at the man page.
        
         | Symbiote wrote:
         | It is amazing how many people use any CLI tools for years
         | without reading the man page.
         | 
         | The man page for the shell (man bash; man zsh) is a good place
         | to start.
        
           | tacostakohashi wrote:
           | Yes that too... "the kids of today" seem to regard basic
           | shell tools and scripting as a dark art rather than an
           | everyday part of using a computer and doing development
           | productively. It's kind of sad.
        
             | klabb3 wrote:
             | I can't speak for others but personally I've never built up
             | enough motivation to learn shell scripts (and related
             | hackery like awk and sed) properly, even though I've learnt
             | maybe 5-10 programming languages quite well and use the
             | shell interactively all the time.
             | 
             | I can't explain with certainty why, but I think it's due to
             | lack of discoverability, inconsistent conventions for flags
             | and positionals, esoteric syntax for simple control flow,
             | lack of errors/feedback for things like undefined
             | variables, no scoping/namespaces, unclear type system (not
             | asking for much, strings, bools and ints would suffice).
             | That said, piping/streaming is amazing and often better
             | than in modern languages.
             | 
             | In short, it's quite different from other imperative
             | languages - the design feels arbitrary and the learnings
             | non-transferable, even though I know it is useful and
             | ubiquitous.
        
             | jessaustin wrote:
             | IIRC, one could have said the same of "the kids of" twenty
             | years ago. It still blows my mind that I didn't learn
             | scripting at university. Could it have been because all the
             | CS classes insisted on tcsh? My first job, I was using an
             | internal tool and thought to myself "this could be better".
             | My boss said "call this guy", and a brief phone call
             | completely changed my understanding of Unix.
        
         | toast0 wrote:
         | > these people seem to think that cannot be done in a shell
         | script because the password needs to be entered interactively.
         | 
         | That's what autopw[1] is for. ;p
         | 
         | [1] https://github.com/jschauma/sshscan/blob/master/src/autopw
        
       | throw7 wrote:
       | The comparison to public-key authentication is wrong. Introducing
       | cert authentication brings a third party: public key
       | infrastructure (which isn't addressed at all and is described as
       | some type of security panacea).
       | 
       | The more apt comparison should be made to other authentication
       | technologies like, in particular, kerberos.
        
       ___________________________________________________________________
       (page generated 2022-03-24 23:00 UTC)