[HN Gopher] Our Dumb Security Questionnaire
___________________________________________________________________
Our Dumb Security Questionnaire
Author : zdw
Score : 96 points
Date : 2021-01-15 17:13 UTC (5 hours ago)
(HTM) web link (hangar.tech)
(TXT) w3m dump (hangar.tech)
| jamiesonbecker wrote:
| > If you don't use a cloud provider: why not?
|
| This makes it sound like it's a negative, but a solid argument
| can be made controlling your application all the way down to the
| metal is a _more_ , not less, secure way to go.
|
| We can all make opinionated assumptions about whether a cloud
| provider is "more" or "less" secure than doing it (whatever "it"
| is, exactly) on your own, but even the fact that it's so
| debatable proves that it's not really a defining point that
| should be on this list of questions (although it might be to an
| individual team, especially if you're trying to bring things in-
| house in the first place, but of course this set of questions is
| really relevant to SaaS.)
| count wrote:
| I would suggest that for the VAST majority of organizations, a
| cloud provider[1] will be an unequivocally more secure
| infrastructure than running it themselves 'down to the metal'.
| The big clouds have security research security ops teams larger
| than many companies. I don't think anyone who has seriously
| spent time looking at this would debate it at all. Nb. this has
| no bearing on if it's the right _business_ decision, mind you.
| That absolutely could end with you being better on your own
| metal! But you won 't be more secure.
|
| Source: spent a decade doing security evaluations of cloud
| providers for the DoD/IC, who also had this misconception going
| into things.
|
| [1] to be fair, I'm only talking about the large cloud
| providers (AWS, Azure, GCP, etc.). There are tens of thousands
| of SaaS/smaller cloud providers that are absolutely debatable
| around security posture.
| jamiesonbecker wrote:
| > But you won't be more secure.
|
| I respectfully disagree. A cloud has many more vectors for
| attack, and even the largest clouds have blind spots. For
| example, check out AWS's (lack of) response to a vuln report
| that I filed with their security team:
| https://github.com/aws/aws-codedeploy-agent/issues/30
|
| Did you know that AWS does NOT pay for security vulns and
| does not operate a bug bounty program? It seems a common myth
| that AWS is more secure than any particular company. Your
| security inside or outside of a cloud has to do more with
| your internal security posture than what AWS brings to the
| table (and leaves behind).
|
| The public cloud is not magic security pixie dust. It's a
| virtualization platform with extra services attached. You are
| still 100% responsible for securing your servers (meaning
| everything down to the network routing layer, and some beyond
| that, too). Obligatory reference to the AWS Shared
| Responsibility Model[0]
|
| This is not a bad thing, but it's still a thing. Anyone who
| migrates to the cloud and magically expects their security
| posture to improve one iota is simply wrong unless they take
| active steps to integrate as tightly as they can into that
| cloud's security tools. As someone who has done a ton of AWS
| security audits, I can tell you that people take multiple
| approaches on that: either they don't integrate into it
| fully, or they submit to vendor lock-in.
|
| This is perhaps too deep of an argument, since clearly
| intelligent and skilled people can reasonably disagree. FWIW,
| lapsed AWS SA, recovering Fortune 50 cloud security
| architect, and please don't take my argument about the cloud
| to indicate that I'm anti-cloud: my security-focused SSH key
| management company, https://userify.com, is _strongly_ cloud
| focused: AWS Advanced Tier, GCP Partner, SOC-2, PCI, HIPAA. I
| 'm very much pro-cloud, while recognizing that it enlarges
| your threat model but brings other security advantages.
|
| Where you're hosted doesn't matter as much as whether you're
| paranoid and clueful, and perhaps have former blackhats on
| your team ;) So my point is simply that the cloud question
| isn't that relevant, and the thrust of the other questions
| should be used to determine security competency instead of
| where they're running their operations.
|
| 0. https://aws.amazon.com/compliance/shared-responsibility-
| mode...
| teddyh wrote:
| > _The big clouds have security research security ops teams
| larger than many companies._
|
| That's like claiming that a company is on safe legal basis
| beause of the army of lawyers they employ. It might be true,
| but the inverse correlation also holds.
| count wrote:
| OK, I should have added in qualifiers 'highly competent and
| respected'. More bodies doesn't necessarily equal better
| product, for sure.
|
| They can devote teams of researchers and engineers to
| topics like CPU side channel attacks, entropy draining in
| virtualized environments, and other stuff that your average
| firewall/AV admin isn't even aware are attack vectors.
| jamiesonbecker wrote:
| > OK, I should have added in qualifiers 'highly competent
| and respected'. More bodies doesn't necessarily equal
| better product, for sure.
|
| Agreed.
|
| > They can devote teams of researchers and engineers to
| topics like CPU side channel attacks, entropy draining in
| virtualized environments, and other stuff that your
| average firewall/AV admin isn't even aware are attack
| vectors.
|
| True. Of course, side channel attacks aren't even
| possible threat vectors on bare-metal/non-
| virtualized/single OS/tenancy environments. From that
| perspective, virtualization can be seen as having two
| primary benefits: potentially higher availability and the
| cost savings of combining multiple functions onto a
| single piece of hardware, but if costs were not a concern
| and all else being equal, separate bare-metal servers
| will have better isolation.
|
| One thing that concerns me is that KPIs at some large
| cloud providers seem to be focused on creating _new_
| features and not fixing bugs, especially security bugs,
| so there 's no incentive to do the drudgery of curing
| security flaws for a developer. It's hard and detail-
| oriented work, which makes me even more concerned when I
| see a trillion-dollar company that won't pay to find
| about bugs in its own products.
|
| Lots of huge companies with tons of money have big
| security teams. This does not always give the best
| results, as we've seen.
|
| Of course, having great arguments (like this one!) helps
| in raising awareness and comprehension among the less
| technical stake holders. Iron sharpens iron!
| gneray wrote:
| nice, thanks for sharing
| _wldu wrote:
| Some industries have already matured and standardized this
| concept. The HECVAT is one example.
|
| https://library.educause.edu/resources/2020/4/higher-educati...
|
| The benefit of this is that if you (as a vendor) wish to sell in
| this industry, you probably only need to complete this once,
| rather than one for each potential customer.
| orestis wrote:
| This is indeed much shorter than what I've seen around.
|
| As a developer/team lead that might need to answer these
| questions to a satisfactory degree, what are resources that would
| actually help implementing this kind of security infrastructure?
| jacobian wrote:
| (Author here.) Well, you could always use this questionnaire as
| a starting point itself: ask yourself these questions, and if
| you're not happy with the answers, do something about it.
|
| Another reasonable security practices starting point would be
| another article by Latacora:
| https://latacora.micro.blog/2020/03/12/the-soc-starting.html
|
| It's semi-oriented towards SOC2, but every item on that list is
| practical, doable even for small teams, and has real solid
| security impact.
| orestis wrote:
| Ooh, I've done that, and I'm doing it with many such
| questionnaires I receive :) sometimes it makes sense and we
| do something about it, but many times you just don't know
| what you don't know, or you don't know where to start, and
| it's not a topic that comes up often on the various public
| fora.
|
| I was looking for books, talks, guides - anything. I just
| read the latacora soc2 guide and it's at least a starting
| point.
| karmakaze wrote:
| This is great. I like the open ended phrasing, there's no on a
| scale of 1 to 10 thoughtless responses possible.
| tr0ut wrote:
| Are there any legal issues if one were to be untruthful/lie on
| security questionnaires?
| munchbunny wrote:
| What I'm familiar with is that you would write into any
| business contracts signed with that vendor that all of the
| [insert scoping modifier here] representations the vendor has
| made are truthful or else the customer may cancel the contract
| and seek [consequences of contract violation]. Just make sure
| the scope includes that security questionnaire.
| Wowfunhappy wrote:
| If nothing else, I imagine if the vendor _were_ to get hacked,
| the client would have obvious grounds for a lawsuit. Obviously
| you 'd rather it not get to that point, but, still.
| ndiscussion wrote:
| IANAL but wouldn't that be straightforward fraud?
| tr0ut wrote:
| I asked because I've heard more than once that a company
| either stretched the truth or outright lied on these
| questionnaires.
|
| So stretching the truth could be:
|
| Do you adhere to NIST?
|
| The truth could be: "well not exactly but that's on our
| roadmap,we do somethings that are close enough." That would
| get a 'YES' check.
|
| Or something like end to end encryption. The answer could be
| a 'YES' because a company uses front-end TLS and pretends to
| not completely understand the ask.
|
| In this case it is mostly the business either forcing
| security to bs or another group (Sales?) filling out the
| response untruthfully because they are loosing revenue if
| they're honest.
| the8472 wrote:
| In my experience people don't outright lie about these things,
| they sell a piggy bank as fort knox.
|
| E.g. someone running an automated vulnerability scanner that
| may not even be entirely appropriate for the application being
| scanned could be considered a pen test or perhaps OWASP
| mitigation.
|
| TOTP software authenticator on the same machine as the password
| safe? Totally 2FA.
|
| Security training for employees? Some mind-numbing videos of a
| consultant reading the OWASP list from 2011 over some
| powerpoint slides and mentioning some buzzwords, employees
| self-certify having watched these videos.
| theevilsharpie wrote:
| If you're hacked and leak your customer's data, and it turns
| out that you materially lied on your customer's security
| questionnaire/due diligence, you could be sued by your customer
| for damages, and your insurance company could refuse to defend
| you.
| count wrote:
| These are pretty good, but one problem for 'general' adoption of
| something like this:
|
| Most customer orgs aren't qualified to judge the vendors answers
| because _they aren 't good at security either_.
|
| Are you confident you know what a good answer to 'how do you push
| code to production' looks like?
| jacobian wrote:
| I tried to get at that in the "who this is for" section:
|
| > [This assumes that] you have a security person (or team) who
| can evaluate the answers and is part of the decision-making
| process. If nobody's going to read the answers, don't waste
| your vendor's time.
|
| So yeah, totally agree. If you can't adequately evaluate the
| answers, it's not worth anyone's time to ask 'em.
| count wrote:
| I missed that line, sorry!
| gwbas1c wrote:
| This also looks like a good way to evaluate a potential employer.
| tolstoshev wrote:
| One reason to us SaaS - if there's a hack you can blame them
| rather than taking the heat yourself. It's outsourcing of
| culpability.
| scooble wrote:
| To a point. I've been asked about the criteria used to evaluate
| third party suppliers, for example. Or the physical security
| measures at outsourced data centres.
| jamiesonbecker wrote:
| In an ideal world, these are mostly reasonable questions, even if
| a bit specific to the OP's requirements, but unless you are
| purchasing SaaS software on a large, enterprise-style license
| (i.e., 6 figures or more), it probably won't be cost-effective
| for most vendors to gather and collate this documentation, as
| well as answering these questions, just like it's not worth it to
| ink a custom contract on a smaller deal, where legal fees alone
| might eat up the entire profit margin or even push it negative.
| Engineering time is expensive and some of these questions are
| technical or "special" in nature.
|
| There's a reason why standardized third-party audits like SOC-2's
| and ISO-27001's exist: to reduce the time required to document
| your veracity and security as a vendor for a potential customer.
| Since even large customers rely on the statements (independently
| audited, unlike in these questions) of a security attestation to
| make purchases, why should a customer request that extra time
| should be taken away from other responsibilities, like making
| better products or providing customer support?
|
| I freely admit that I'm a bit biased against ad hoc security
| questions, even though I used to do it myself when working on
| large security teams. ;) My security-focused SSH key management
| company, https://userify.com, went through the time and expense
| to achieve AICPA SOC-2 certification from an independent third-
| party auditor to reduce the time and costs involved in responding
| to smaller RFP's, and to provide fully documented, standardized,
| and legally binding proof of our security bonafides. We still try
| our best to intelligently respond to any and all questions,
| especially about security, from any customers at all, even free-
| tier customers and hobbyists, but it's harder to do that when
| presented with a big list of questions that are mostly answered
| in our SOC-2 audit already.
| tothrowaway wrote:
| > it probably won't be cost-effective for most vendors to
| gather and collate this documentation, as well as answering
| these questions, just like it's not worth it to ink a custom
| contract on a smaller deal, where legal fees alone might eat up
| the entire profit margin or even push it negative.
|
| Indeed. I get these security "wall of questions" for my
| $10/lifetime SaaS product (it's a dumb little puzzle maker). I
| can't fathom why anyone thinks a vendor would spend the time
| answering their questions for that kind of money.
|
| In their defense, it's probably a standard part of the
| procurement process and they don't even look at the app before
| sending the questionnaire.
| [deleted]
| bostik wrote:
| At the risk drawing fire from everyone, I don't really think
| ISO27001 is a security certification. We get audited annually
| against 27k Annex A (basically ~95% of full one) + UK Gambling
| Commission extras.
|
| The audit focuses far more on technical aspects of business
| continuity than actual security. There's certainly plenty of
| overlap, but other than the parts about access controls and
| "who watches the watchmen" aspect, ISO27k is almost entirely
| about your ability to recover from even the most devastating
| disaster. The pragmatic security parts have a bolted-on feeling
| to ensure the recovery path remains largely uncorrupted.
| jamiesonbecker wrote:
| Annex A is a subset of 27001, but 27001 _is_ focused on
| security.
|
| https://www.iso.org/isoiec-27001-information-security.html
|
| (Even so, your point is well taken. The overall 27000 series
| is more business continuity (DR etc) focused rather than
| purely security focused.)
| quadrifoliate wrote:
| There is a value to just human responses, answering someone's
| questions can help you look inwards to your service/product as
| well.
|
| For example, as far as I can tell, "HIPAA" is spelled "HIPPA"
| on your website - this indicates a lack of familiarity with the
| Act (and potentially professionalism) to me as a first-time
| user, regardless of the multiple kinds of certifications you
| might have.
|
| Is it possible you could have caught that when replying to a
| simple set questionnaire like the one in the link but focused
| on HIPAA?
| jamiesonbecker wrote:
| > There is a value to just human responses, answering
| someone's questions can help you look inwards to your
| service/product as well.
|
| We do respond to every single email, and hopefully fairly
| intelligently most of the time ;), even if the customer is a
| hobbyist or free tier user, especially with regards to
| security questions. (Although internal stats are slightly
| meaningless, and survey responses are relatively uncommon for
| the volume of emails we receive, we do have >99% customer
| support satisfaction based on our internal tracking.)
|
| > Is it possible you could have caught that when replying to
| a simple set questionnaire like the one in the link but
| focused on HIPAA?
|
| Ha, looks like we mispelled it in one place and spelled it
| correctly in another! Are you asking if we'd have caught a
| misspelling or other oversight on our website on the basis of
| an emailed questionnaire? Good question, but, to be honest,
| doubtful (but thank you for bringing it to my attention --
| fixed!)
|
| We've had wall-of-questions requests like that before, and we
| either respond with our other certs/attestations, or
| negotiate a BAA, once we get down to more specific questions.
| For HIPAA, we're providing on-premise security software, and
| not working with PHI in any way (we're deployed in some
| hospital systems in the US), so generally customers are just
| looking to understand our general security stance and how we
| can help them meet the Security Rule with login accounts.
| cinbun8 wrote:
| I really loved this list and think its a great place to start .
| Add to this a network diagram; arch diagram; Tenancy; WAF + FIM +
| SIM; Bring your own keys; and you can start bringing in the
| enterprises into your deals. Not that you need all of those, but
| the add-ons usually build on the struts mentioned on that list.
| alg0 wrote:
| Basic information about these things is alright to pass on but
| this is kinda information that should be a bit private. Even if
| here ( and anywhere on Internet ) is a lot of people that think
| obsecurity is bad it is important tool in any security stack.
| "Security by obsecurity is bad" means that there is no other
| security but a obsecurity. Even NIST gives a recomendation for
| obsecurity if your other security stack is good
| blakesterz wrote:
| "But unfortunately, they're also kinda the best we can do. Huge
| companies might be able to afford real human-powered security
| audits and convince vendors to allow them, but small startups
| like ours don't have a better option than relying on DSQs."
|
| Not what I expected based on this title. I liked this.
|
| I'm not sure how many people know about the Higher Education
| Community Vendor Assessment Toolkit, but if you're into Dumb
| Security Questionnaires, this is worth checking out. "The HECVAT
| is a questionnaire framework specifically designed for higher
| education to measure vendor risk." I know it says Higher Ed, but
| it works just about anywhere.
|
| https://library.educause.edu/resources/2020/4/higher-educati...
| motohagiography wrote:
| Can do it in 4 questions:
|
| 1. What do you need to protect?
|
| 2. What is the complete list of tech stack technologies it
| depends on?
|
| 3. What are you doing to protect that important thing today?
|
| 4. Who do you need to protect it from?
|
| If you can't answer these, the answers to the rest of the
| questions really don't matter.
| coldcode wrote:
| I worked for a HIPAA covered healthcare company that routinely
| passed audits. They also stored all the production database and
| server passwords in a plain text file accessible to half the
| company and no audit trail of any kind on logins.
|
| Clearly no auditor ever asked or looked.
| jpcosta wrote:
| where would one hire an auditor like this? asking for a friend
| et-al wrote:
| I know you're being facetious, but any SOC2 audit would
| overlook this because they're just going down a checklist
| making sure specific controls are in place and not actually
| probing for the various possibilities to bypass these
| controls.
| jamiesonbecker wrote:
| > any SOC2 audit would overlook this because they're just
| going down a checklist
|
| From the GP, "They also stored (1) all the production
| database and server passwords in a plain text file
| accessible to half the company and (2) no audit trail of
| any kind on logins."
|
| Any competent SOC-2 auditor would _not_ overlook this.
| Large swaths of criteria are specifically geared to uncover
| both of these (unencrypted credentials and audit trails).
|
| SOC-2 audit firms have a strongly vested interest in hiring
| competent auditors, because if a SOC-2 auditor did not ask
| questions covering these areas, then failure to complete
| the audit would be actionable malpractice and the auditor
| would be liable, unless the company lied (committed
| demonstrable fraud) in its written responses.
| DiabloD3 wrote:
| Given the current political climate, such a SOC-2 auditor
| should be outed by name.
| jamiesonbecker wrote:
| > Given the current political climate,
|
| Not sure what politics have to do with it.
|
| > such a SOC-2 auditor should be outed by name.
|
| That would be defamatory, and potentially extremely
| unfair, especially if someone lied or was just incorrect,
| in an anonymous internet forum.
|
| If someone experienced a loss because they relied on
| untrue statements from an auditor, they would have
| grounds to bring a case, and the auditor would have a
| chance at due process to respond to that case.
|
| It probably seems slow and unwieldy when we all want
| justice _now_ , but this is how we can be assured that
| justice is, in fact, done, and not injustice.
| raverbashing wrote:
| > Any competent SOC-2 auditor would not at all overlook
| this
|
| I think this statement is tautological, in a way
| jamiesonbecker wrote:
| I see your point: if they were competent, they wouldn't
| overlook it. My point was more that because SOC-2 is more
| pedantic (than, say, HIPAA), it's harder for a SOC-2
| auditor to mess this up.
| raverbashing wrote:
| No, I'm saying their audit process focus more in box
| checking than actually seeing "you left the key under the
| mat"
| jamiesonbecker wrote:
| The "boxes" to be checked are actually asking open-ended
| questions about that, and asking you to provide copious
| written documentation backing up what you are claiming.
| This process takes weeks or months and is quite
| expensive.
| [deleted]
| [deleted]
| Aardwolf wrote:
| > If you use a cloud provider (GCP/AWS/Azure/etc): how
| credentials are provisioned, managed, and stored. If an attacker
| gained access to an individual developer's cloud credentials,
| what actions could that attacker perform, and how would you
| detect and respond to the breach?
|
| N/A
|
| > If you don't use a cloud provider: why not?
|
| We don't need it because the entire database fits in a TXT file
| on the desktop
|
| > Describe how staff authenticate to company services (e.g.
| servers, email, SaaS products), particularly highlighting your
| use of password managers, 2FA, and SSO.
|
| Our windows 98 login is password protected!
|
| > What development practices do you use to protect against the
| OWASP Top 10?
|
| The what now?
|
| > Describe the steps a developer or operations person takes to
| push new code to production.
|
| Save the updated visual basic code and then run it!
|
| > Have you had any security breaches in the last two years?
|
| No, because of our 1337 skills!
___________________________________________________________________
(page generated 2021-01-15 23:01 UTC)