[HN Gopher] DigiCert: Threat of legal action to stifle Bugzilla ...
___________________________________________________________________
DigiCert: Threat of legal action to stifle Bugzilla discourse
Author : DanAtC
Score : 567 points
Date : 2025-02-25 01:40 UTC (21 hours ago)
(HTM) web link (bugzilla.mozilla.org)
(TXT) w3m dump (bugzilla.mozilla.org)
| nightfly wrote:
| Can someone link to (or post copies of) the offending
| questions/statements?
| infogulch wrote:
| In the second paragraph:
|
| > Contrary to this statement, I received a letter from
| DigiCert's lawyers, Wilson Sonsini, regarding posts made by
| Sectigo's Chief Compliance Officer in bug 1910322.
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322
| mdaniel wrote:
| > The code worked in our original monolithic system but was
| not implemented properly when we moved to our micro-services
| systems
|
| :chefs_kiss:
| crooked-v wrote:
| As we all know, software isn't software unless it can
| immediately scale to having 12 billion users.
| evil-olive wrote:
| it gets even better:
|
| > We also found that the bug in the code was inadvertently
| remediated when engineering completed a user-experience
| enhancement project that collapsed multiple random value
| generation microservices into a single service.
| tpxl wrote:
| > multiple random value generation microservices
|
| Wot. They had _multiple_ services generating random
| numbers?
| aragilar wrote:
| It might mean multiple instances of the same code (which
| would explain why there might be commonalities)?
| intelVISA wrote:
| Some might see horror but I see a few genius level
| engineers 'working' 1hour a month rebuilding the same
| getrandom syscall.
|
| They hiring? :]
| infogulch wrote:
| This is shocking to read. Even attempting to choke the legal
| speech of web PKI contributors with legalistic bullying is a
| gross inversion of the purpose and goals of the organization, and
| IMO warrants revoking everything to do with DigiCert on the spot.
| terminalbraid wrote:
| > IMO warrants revoking everything to do with DigiCert on the
| spot
|
| This is pretty heavy handed and I don't think you've thought
| through what the consequences of that may look like. The
| historic way to deal with a problematic CA is to prevent them
| from issuing new certificates or renewing certificates (after
| taking care of immediate damage, of course). There are _lots_
| of legitimate companies that use Digicert and they should have
| an expectation of being able to continue business in the short
| term while they find a different certificate provider.
| DeepPPP wrote:
| Are you saying it was okay to distrust EnTrust but DigiCert
| is too big to fail?
| terminalbraid wrote:
| No, and I'm deeply confused how you drew this conclusion
| from what was written.
|
| I specifically say the historical way to deprecate a
| certificate authority is to prevent them from issuing new
| certificates. Since certificates have a finite lifetime,
| the certificate authority would as well and would have
| immediately no further revenue from certificates.
|
| I _am_ saying "pulling the plug" without notice is going
| to take down tens of thousands of bystanders and anyone
| using those certificates for business would be unable to
| conduct business securely online, which would be
| disastrous. For example amazon.com has a DigiCert-issued
| certificate.
| infogulch wrote:
| I agree. "Revoking everything digicert" is a bit imprecise,
| "preventing digicert from issuing new certificates" is a more
| reasonable measure that actually only damages the parties
| that deserve it (digicert) and spares innocent bystanders
| (their customers).
| jchw wrote:
| Web PKI drama is always astonishing to me because it is one of
| the only areas of the entire world where a corporation's "fucking
| around" is seldom ever _not_ followed by a sobering "finding
| out" period. The various entities that decide what CAs to trust
| can effectively dismantle any CA business in the world, basically
| at the drop of a hat. If DigiCert decides to play this game and
| lose, it would make them the biggest such loser so far. DigiCert
| is, as far as I know, the largest CA on the Internet. It would
| certainly send a strong message, and cause a lot of chaos, if the
| biggest CA on the Internet found itself being removed from trust
| stores. Yet, there's no particular reason it couldn't happen. How
| exciting.
|
| Of course, I think that is unlikely, but on the other hand, it's
| just as cathartic to imagine whatever idiot at DigiCert thought
| it was a good idea to engage legal here to have the dressing down
| of a lifetime. I read the thread in question. It doesn't make
| DigiCert look good, but this action definitely is more damning to
| them than anything Collan said in my opinion.
| userbinator wrote:
| _It would certainly send a strong message, and cause a lot of
| chaos, if the biggest CA on the Internet found itself being
| removed from trust stores._
|
| How many will agree to that removal? How many will see one more
| reason to forever turn off automatic updates and decide what to
| trust themselves, having seen yet another way some faceless
| entity they never knew about can break things?
|
| It will certainly send a strong message, but likely not the
| intended one. All it will do is increase the lack of trust in
| centralised PKI in general.
| jchw wrote:
| I'm not sure how to put this nicely, but I'll try my best:
| The normal people, who account for 99% of the end users of
| DigiCert's customers, are not going to disable updates on
| their web browser or their OS to "take a stand" against this
| decision. (And corporate users can't do this anyways, since
| it's not in their hands, and keeping things out of date is
| not a reasonable option for an organization that has security
| standards.) Most of them won't know what is going on here,
| and if they hear about it, probably won't care or know what
| to do about it. That's the Web PKI infrastructure working as
| intended, because it would be infeasible for billions of
| people to properly understand the gravity of all of these
| decisions. In order to ensure that TLS connections are at
| least minimally safe, it's pretty much necessary for things
| to work this way.
|
| I'm not arguing that it's good that a relatively small number
| of entities (mainly Google, Microsoft and Mozilla) decide
| which CAs are trustworthy, but that's all the more reason
| that it's important for all of this Web PKI work to happen
| completely in the open, so that the few who can spare the
| time and effort to scrutinize what is going on can help the
| rest of us have a safer Internet. We don't have a better
| solution. That's also why DigiCert issuing legal threats
| because they don't like how one of these issue reports makes
| them look is a serious problem that can't be tolerated.
| LegionMammal978 wrote:
| > The normal people, who account for 99% of the end users
| of DigiCert's customers, are not going to disable updates
| on their web browser or their OS to "take a stand" against
| this decision.
|
| I'd imagine that they wouldn't do it to "take a stand" so
| much as "avoid the risk of getting their stuff broken in
| the short term" in this scenario, regardless of whichever
| party loses the blame game. See: the recent WordPress
| drama, which has turned customers away from both parties
| involved.
| jchw wrote:
| As I understand it, the way certificate authorities are
| removed from the trust store is progressive: they will
| announce a date after which new certificates from a given
| CA will no longer be trusted. I think this can be made
| even more progressive, by limiting the validity period of
| new certificates that will be trusted. DigiCert will have
| little recourse other than to let their customers know,
| and/or start providing certificates issued by another CA
| that follows the web PKI procedures and remains trusted.
| (They _can_ still do that, of course, it 's just that
| their own certificates issued by them directly won't be
| trusted anymore.)
|
| On the flip side, for user impact, it will play out like
| this: Some bank or other important entity could possibly,
| for whatever reason, continue using a (presumably
| expired? unless DigiCert continues issuing anyways; note
| that most likely, they will _not_.) DigiCert certificate
| after the cut off date, which will lead to users
| receiving errors. Some of them will have HSTS setup,
| which will lead to an emergency situation where they
| _have_ to issue a new certificate ASAP, as it will
| basically halt their business until they do. For places
| where there is no HSTS, users may be instructed to simply
| bypass the certificate warning temporarily, and support
| lines will be absolutely swamped until they actually fix
| the problem.
|
| The WordPress situation is quite different. You don't
| _have_ to use WordPress. Users don 't even know what the
| Web PKI _is_ to find an alternative to it, not that there
| is one or will be one.
| snailmailstare wrote:
| Sure some users would keep the CA active out of fear,
| while everyone paying digicert would also begin to move
| out of fear.. What portion of the strange sort of sites
| that couldn't figure out how to get off but could figure
| out how to renew would bother with paying for a different
| error message?
| cortesoft wrote:
| > mainly Google, Microsoft and Mozilla
|
| They don't have free rein to do whatever they want. If they
| chose to remove them, they are going to have to be ready to
| defend themselves in court.
| jchw wrote:
| Why exactly do you think certificate authorities have a
| legal right to appear in trust stores?
| saurik wrote:
| Is that really the claim, though? If someone chooses to
| punish DigiCert for compliance with the TRO then DigiCert
| might have grounds to file for declarative relief from
| the court, as the court could very well decide to help
| protect DigiCert from external consequences of their
| demand.
| nickf wrote:
| Perhaps, but only in the case that the delayed
| revocations were scoped to those certificates covered by
| the TRO, and not over 1000x more from subscribers who had
| nothing to do with the company who filed the TRO.
| jchw wrote:
| Well personally, I think that'd be a terrible position
| for them to take. Honoring the TRO is, as far as I can
| tell, not the issue that is being raised.
| cortesoft wrote:
| This case literally involves the courts getting involved
| in what should appear in trust stores. DigiCert was
| ordered by the courts to not revoke certificates via a
| TRO.
|
| If the browsers removed DigiCert, DigiCert could
| certainly sue based on the harm to their business. They
| could argue that the browser vendors were inconsistent
| with the application of their rules. They could argue
| that the browser vendors were unfairly competing by
| kicking them out.
|
| Not saying they would win all these, but they would
| certainly fight it - the alternative would be the end of
| DigiCert, they aren't going to go down without a fight in
| whatever venue they can find.
| SkiFire13 wrote:
| > DigiCert was ordered by the courts to not revoke
| certificates via a TRO.
|
| ... and DigiCert did not try to appeal the TRO.
| cjbprime wrote:
| Those companies are always ready to defend themselves in
| court, and this is a country with extremely strong free
| speech rights.
| chgs wrote:
| What country?
| justinclift wrote:
| Both Google and Microsoft tend to act like they have
| enough money and lawyers to defend whatever behaviour
| they feel like engaging in.
| userbinator wrote:
| _The normal people, who account for 99% of the end users of
| DigiCert 's customers, are not going to disable updates on
| their web browser or their OS to "take a stand" against
| this decision._
|
| ...unless they notice the breakage, and you tell them to do
| so to stop it.
| jchw wrote:
| Ignoring the policy reasons that this won't happen, not
| all devices can be downgraded, so by then it's too late
| anyway. Best you can do is bypass a non-HSTS certificate
| warning.
|
| That all assumes DigiCert continues issuing certificates
| they know won't be trusted. I assume most likely what
| will actually happen is they have to start selling
| certificates from another CA, but failing that they're
| probably just going to stop issuing certificates before
| the cutoff.
| edelbitter wrote:
| Many systems do not fetch updates from the Mozilla root
| store, but from their (possibly Debian-derived) stable
| distribution of it. Meaning two highly respected entities,
| known for being well aware of the wider impact of their
| careful enforcement of strict policies, need to agree to
| cause any major breakage. When that happens, I can blindly
| trust they did the thing needed to keep the unaffected parts
| of that weird system working as intended.. and then still
| head to bugzilla and read about the background - to laugh at
| whoever triggered the mess.
| SkiFire13 wrote:
| > Meaning two highly respected entities, known for being
| well aware of the wider impact of their careful enforcement
| of strict policies, need to agree to cause any major
| breakage.
|
| Note that this is not a "both" but a "any of them". One can
| disagree with the other on this and still cause wide
| breakage.
| pilif wrote:
| If DigiCert were to lose browser trust (at this point, that's
| still a big if), it would happen the same way it happened
| with prior CAs, some of which were pretty big themselves
| (Symantec): all certificates issued after some date would not
| be trusted, yes. But all existing certificates would remain
| valid.
|
| This gives certificate owners ample time to look for a
| different issuer and no certificate buyer would deliberately
| purchase a certificate from an issuer when they know some
| percentage of users will not trust that cert.
|
| So for the end users, everything will keep working: the
| existing digicert certs stay valid and newly refreshed certs
| will be signed by a different authority. There is no need to
| turn off automatic updates over this.
|
| Between Entrust and Symantec, we've already seen this happen
| to large well-known CAs and everything remained fine (not for
| the offenders, but, hey, that's the system working as
| intended)
| woodruffw wrote:
| > All it will do is increase the lack of trust in centralised
| PKI in general.
|
| "In general" is a broad statement, given that the
| overwhelming majority of people using the Web PKI have no
| idea that they're doing so. DigiCert is not a legible part of
| the value chain for the average users; they won't notice that
| the sites they use switch CAs. This strong disfavoritism
| towards vendors is arguably one of the Web PKI's greatest
| strengths.
| twisteriffic wrote:
| It's refreshing to see folks who're obviously used to hiding
| behind clouds of bullshit get skewered by people who both know
| enough to see through it and have the time and energy to follow
| every thread to completion.
|
| The most recent digicert thread smells suspiciously similar to
| those that lead up to the Entrust debacle.
| chicom_malware wrote:
| > It would certainly send a strong message, and cause a lot of
| chaos, if the biggest CA on the Internet found itself being
| removed from trust stores
|
| Their customers would be forced to give their business to
| Honest Achmed[1].
|
| [1] https://bugzilla.mozilla.org/show_bug.cgi?id=647959
| skissane wrote:
| Someone should start a real CA business, complete with all
| the proper audits and everything to get it into the browser
| trust stores... and call it "Honest Achmed's Used Cars and
| Certificates" (have it buy some random used car dealership in
| the middle of nowhere so the name is not a lie)
|
| Where's a billionaire troll when you need one? (And a form of
| billionaire trolling that would upset a lot less people than
| Musk's version of it.)
|
| Would be even funnier if the billionaire's name actually was
| Achmed
| justahuman74 wrote:
| What has prevented this (a good legit CA co) from happening
| before now?
| SpicyLemonZest wrote:
| Let's Encrypt has been around for a decade and is doing
| pretty well for itself. The manual validation required
| for EV and OV certificates precludes it from being a
| passion project, unless managing a call center and a
| squad of document reviewers is your dream job.
| Thorrez wrote:
| Let's Encrypt doesn't issue EV and OV. Honest Achmed's
| Used Cars and Certificates wouldn't have to either.
| boricj wrote:
| It wouldn't be fun if Honest Achmed's Used Cars and
| Certificates didn't also issue EV and OV certificates. It
| wouldn't be on brand to miss out on entire segments of
| the certificate market.
| nickf wrote:
| Time and money. Plus right now even if you bought the
| infra, the staff, paid for and passed the audits, and
| then waiting while Apple, Mozilla, Google and Oracle (at
| least) included your roots...Microsoft aren't taking more
| right now. So you have to wait for some unknown time in
| the future when/if they start doing that again. You could
| purchase a root off an existing CA, subject to the trust
| stores approving it, and the boatload of cash you'd need
| to buy it (plus still having the staff and infra to
| operate it).
| skissane wrote:
| > Microsoft aren't taking more right now. So you have to
| wait for some unknown time in the future when/if they
| start doing that again.
|
| Interesting, I hadn't heard that, is there anywhere I can
| read more about it?
|
| > You could purchase a root off an existing CA,
|
| As well as a sub-CA, I remember in theory you can have
| two independent CAs with cross-signing... but do browsers
| actually support that?
| nickf wrote:
| Nothing public to point to, sorry.
|
| Sub-CAs: Not really. Operational risk to the parent CA is
| huge, you'd be hard pressed to get any current public CA
| to sign an issuing CA to be operated externally. Cross-
| signing still works (though it is the stuff of nightmares
| in many cases) but again you have to have money and a CA
| willing to do it!
| zinekeller wrote:
| "Nothing public to point out to" is probably accurate but
| it is noted publicly here:
| https://learn.microsoft.com/en-us/security/trusted-
| root/new-...
| rocqua wrote:
| Where's the value?
|
| Anyone can get a cert from lets-encrypt. No users of your
| website care how trustworthy your CA is. And CA's are too
| big to fail, so you barely need to worry about your CA
| being removed by the trust store. So you can only compete
| on price.
|
| If we want to change this, we need a way for a
| certificate to be co-signed by multiple CA's. So that the
| certificate can be presented to the user, and they can
| figure out if any trusted CA of theirs has signed the
| certificate _. This way, revoking trust in a CA becomes
| easier, because people should have multiple signatures on
| their certificate. That means, all of a sudden, that the
| quality of a CA actually matters.
|
| _ Whilst it might seem this is already possible, it is
| not. Cross-signing is only a thing for intermediate
| certificates, and does not work. You can also have
| multiple certificates for the same key, but when starting
| a TLS session, you must present a single certificate.
| (This seems to have changed for TLS 1.3, so perhaps this
| is already possible?)
| toast0 wrote:
| I think TLS 1.3 still requires the end entity
| (server/client) cert first. All the other certs can now
| be in any order and the verification is suppoaed to
| figure out a valid path.
|
| In theory, you could make an intermediate CA and get
| cross signed certs from multiple CAs (hopefully with Name
| Constraints), your intermediate CA signs your server
| cert, and you include all your cross certs and the
| intermediate certs for those. And the client figures out
| which chains it can make and if it likes any of them.
|
| But experience has shown, verification may find _a_ chain
| with signatures that line up, but the CA is expired or
| revoked in the local trust store, and reject the
| verification, even though another chain could have been
| found with the provided certificates.
|
| And, because of the limited information in tls handshakes
| from real world clients, it's difficult (maybe
| impossible) to know which CAs a client would accept, so
| that the server could present an appropriate chain.
| justahuman74 wrote:
| I'm actually curious why that wasn't approved
| woodruffw wrote:
| To my understanding, the point of Honest Achmed was to
| demonstrate that it was possible to write a facially
| reasonable and compliant CA inclusion request that was also
| intuitively unacceptable. It successfully demonstrated that
| the inclusion policies at the time needed to be made more
| rigorous, which they were.
| bigfatfrock wrote:
| Thank you, that was an awesome laugh this morning, that
| request was genius:
|
| > 2. Sub CAs Operated by 3rd Parties
|
| > Honest Achmed's uncles may invite some of their friends to
| issue certificates as well, in particular their cousins Refik
| and Abdi or "RA" as they're known. Honest Achmed's uncles
| assure us that their RA can be trusted, apart from that one
| time when they lent them the keys to the car, but that was a
| one-off that won't happen again.
| account42 wrote:
| > It would certainly send a strong message, and cause a lot of
| chaos, if the biggest CA on the Internet found itself being
| removed from trust stores. Yet, there's no particular reason it
| couldn't happen. How exciting.
|
| Trust stores and Browsers specifically have more options than
| simply remove a CA. A more reasonable approach for distrust
| process of a CA this size would be to stop accepting any new
| certificates issued after a certain date. Then existing
| customers will have a heads up and even if asleep will find out
| the bad news during scheduled renewal rather than while the
| responsible employees are on vacation.
| axus wrote:
| Is there any option to turn off trust for all certificates
| generated after a certain date? The ideal would be to let old
| certificates keep working, and not trust new ones.
| duskwuff wrote:
| It's been done previously, e.g. for Symantec:
|
| https://security.googleblog.com/2017/09/chromes-plan-to-
| dist...
| Arnavion wrote:
| Even taking the (one-sided) depiction of the conversations in
| DigiCert's letter at face value, maybe Sectigo's guy was being a
| git at best and intentionally trolling at worst. (I don't think
| he was, but let's play devil's advocate.) But even then, how did
| DigiCert think getting legal involved would possibly go well?
| Sectigo stands to gain publicity and lose nothing by going public
| with it to the CAB as they did here, and it's not like the CAB is
| going to play marriage counselor and get the two companies to
| make up because one of them got their feefees hurt.
|
| Besides, this kind of hyper-polite passive-aggressive "erm
| akchually" conversation happens in every CAB incident discussion.
| I don't know why DigiCert got particularly upset about this one.
| akerl_ wrote:
| > Besides, this kind of hyper-polite passive-aggressive "erm
| akchually" conversation happens in every CAB incident
| discussion.
|
| As somebody who doesn't spend much time scrolling CAB reports,
| this was jarring to me.
|
| Digicert's legal action seems nuts, and there seems like a
| real, risky issue in the idea that a company's customers can
| use the legal system to block the company from complying with
| its obligations to other entities, but it's hard to see any way
| that could be productively addressed given the back and forth
| in the thread. It's like I'm watching a theatrical production
| staring the most stereotypical corporate drones trading
| comments with the most stereotypical IRC nerds, both sides
| doing circles around an interesting topic but too busy trading
| blows to ever really get to it.
| SpicyLemonZest wrote:
| > there seems like a real, risky issue in the idea that a
| company's customers can use the legal system to block the
| company from complying with its obligations to other entities
|
| As Digicert has repeatedly explained, this is simply how the
| United States legal system works. Courts have broad and
| indisputable power to issue temporary restraining orders, and
| the parties to a case must comply even if doing so violates
| some promise they made to a third party. (The point of the
| TRO is to maintain the status quo while the court figures out
| details like what promises have been made to who.) People in
| the PKI community who believe that some carefully written
| policy would enable CAs to reject an invalidation TRO, or
| convince a court that they cannot issue it, are wrong.
|
| The reason it's never come up before is that no CA had
| previously attempted to enforce a widespread 24 hour
| revocation caused by its own error.
| cjbprime wrote:
| (I think Sectigo's argument is that Digicert did not even
| attempt to convince the court that it should be allowed to
| revoke those certificates in the mandated timeframe. If
| they had attempted and failed, I don't think they would be
| receiving criticism.)
| SpicyLemonZest wrote:
| That's their argument, yes, but it was clearly based on
| the incorrect belief that there's some emergency button
| you can press to demand that a court consider your
| arguments ASAP. As Digicert explained:
|
| > The legal world does not move as fast as the demands of
| our CA ecosystem. Our legal approach was to work with the
| complainant's legal team to get the TRO dismissed in 5
| days.
|
| The court would not have dissolved the TRO in anywhere
| close to 5 days without the complainant's cooperation,
| even if Digicert had an ironclad argument for doing so.
| Digicert made the right choice to get the certificates
| rotated as fast as possible - and I don't think Sectigo
| intends to argue it would be better to stand on principle
| even if that makes the revocation slower.
| cjbprime wrote:
| > the incorrect belief that there's some emergency button
| you can press to demand that a court consider your
| arguments ASAP
|
| ... isn't that exactly the legal button the complainant
| pushed to prevent the revocation?
| SpicyLemonZest wrote:
| Should've been more specific. You can _ask_ a court to do
| all kinds of things, but the individual judge who reads
| your filing doesn 't have to (and in most cases can't)
| carefully analyze your arguments that day. They need time
| to think it over, and probably hearings where you and the
| other party can explain all the arguments for why certain
| rulings should or shouldn't be made. A contract dispute
| like this, where one party says they have a right to do
| something and the other party says they don't, is almost
| always going to take longer than 1 or 5 or 30 days for a
| court to figure out.
|
| Temporary restraining orders are the biggest exception.
| If DigiCert is about to do something crazy like take down
| all your websites, courts are generally willing to put a
| temporary stop to it without understanding all the
| details. "Preserve the status quo" and "prevent
| irreparable harm" are the buzzwords.
| darkarmani wrote:
| > If DigiCert is about to do something crazy like take
| down all your websites, courts are generally willing to
| put a temporary stop to it without understanding all the
| details. "Preserve the status quo" and "prevent
| irreparable harm" are the buzzwords.
|
| So if DigiCert's irreparable harm was great would that
| prevent it? Like legally requiring CAs to follow their
| revocation policies or pay millions in damages?
| SpicyLemonZest wrote:
| Are there actually millions in damages being caused by
| delaying revocation of these certificates? Courts are
| generally averse to "penalty clauses" where you make up a
| nonsense number and call it damages. (Irreparable harm
| means that ordering monetary compensation can't remediate
| it, so a more reasonable fee would probably not count.)
| nickburns wrote:
| You're conflating DigiCert's argument against issuance of
| the TRO, with the irreparable harm the complaintant
| (Alegeus) is alleging will occur if the TRO is _not_
| granted.
| teruakohatu wrote:
| Could they not have had a clause in the contract saying
| "if you delay a revocation in any manner you owe us $100
| million."?
|
| So a customer could go right ahead and get a TRO but long
| term it will cost them less than making sure their
| infrastructure can handle this rare event?
| SpicyLemonZest wrote:
| Liquidated damages are possible, but they have to have
| some relationship to the damages suffered, so $100
| million is probably far too high. In any case, I think
| the complainant was correct that it was _DigiCert_ who
| caused the entire scenario to happen by issuing
| certificates which they should have known were invalid.
| lelandbatey wrote:
| One customer filed a TRO saying "don't revoke my certs".
| DigiCert then said "well, I guess we can't revoke any of
| the certs for the 80,000+ other customers either". That's
| stupid, and not acceptable. Instead, revoke the other
| 79,999 customers and communicate to the CABF saying
| "we've got one holdout that we are legally prevented from
| acting on." DigiCert didn't do that. They're acting not
| on behalf of transparently representing the interests of
| the CA/Browser Forum, instead they're trying to save
| their own skin from both sides. That's _not good enough_.
| akerl_ wrote:
| I don't think I disagree with you about what courts can do.
| And (as shown by all the back and forth in the CAB thread,
| and now here), it is risky: we currently have a situation
| where CAs can find themselves in a position where taking
| court-mandated actions (or lack of actions) puts them in
| jeopardy with the CAB, and violating court orders puts them
| in jeopardy with the government.
|
| For all the back and forth in the CAB thread, it doesn't
| seem we're any closer to finding an escape valve for that,
| which seemingly would have to come from the CAB because
| there isn't even really a mechanism for courts to decide
| not to issue TROs about cert revocation, short of somehow
| taking a case around this to the Supreme Court (which isn't
| going to happen).
| joshuaissac wrote:
| > but it's hard to see any way that could be productively
| addressed given the back and forth in the thread
|
| Another commenter mentioned in a sibling thread the
| possibility of using future-dated revocations. The CA could
| be mandated to publish such a revocation immediately, such
| that it takes effect by the 24-hour deadline. Once published,
| the revocations themselves should be irrevocable. This would
| also need to be accounted for in the CA's customer contracts.
|
| https://news.ycombinator.com/item?id=43169867
| akerl_ wrote:
| As the comment you linked to notes, that doesn't really fix
| the problem so much as it ensures that the CA's legal team
| gets to work nights and weekends for the next year.
|
| The legal system is almost certainly going to view future-
| dating and immediately publishing revocations poorly in any
| civil action where a customer claims harm.
| webprofusion wrote:
| Always two sides to a story but the guy who caused the validation
| bug at DigiCert already _resigned_ because of it (which is
| extreme), the Sectigo guy wanted to prevent the bug being closed
| so he could keep pushing them (in a subjectively prickly fashion)
| for more answers about their general responsiveness.
|
| A bit of back and forth discourse is fine and expected, but if
| you keep pushing someone who has their own legal dept they're
| eventually going to wander over to the coffee machine and have a
| chat about it with them, then they're going to take a look and it
| becomes their problem.
|
| So the number one rule would be don't even breath the word
| "legal" unless you want to invoke them. This particular response
| is just a letter telling them to back off and it's why you have a
| legal dept, so they can argue with each other. This one has just
| found it's way into the open.
|
| There is a understandable perspective that says CAs shouldn't be
| burdened with legal risk in their discussions, but that's
| contrary to the fact these guys are commercial entities
| protecting their interests, so you don't get it both ways unless
| all your CAs are non-commercial, and even then that would only
| extend so far.
| willvarfar wrote:
| You've given a detailed rundown like you've been following, so
| what is your sense about the resignation? Did the guy resign
| willingly, or is Digicert the kind of management that likely
| scapegoated him?
| braiamp wrote:
| He assumed personal responsibility rather than institutional.
| At the time, everyone in the community expressed alarm that
| that happened, because 1) and this is obvious, there's no way
| a single guy would be responsible for the entire fallout and
| 2) because it sets a bad precedent about what companies can
| do with their PoC wrt web PKI. It also meant a loss in
| institutional knowledge about how things worked.
| smithcoin wrote:
| Do you have any resources regarding the resignation I could
| read?
| lelandbatey wrote:
| Here's the long message he left stating in part his
| resignation. The comments which follow spend quite some
| time speaking about his resignation:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c17
| account42 wrote:
| Perhaps they should have also had a chat with their PR
| departement. And anyone responsible for company strategy.
| Becose the actions of the legal department just backfired.
| nneonneo wrote:
| In a nutshell:
|
| DigiCert has delayed revocation beyond what's allowed in the
| Baseline Requirements a few times; most recently,
| https://bugzilla.mozilla.org/show_bug.cgi?id=1896053 and
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910805. In the
| former case, it seems DigiCert chose to delay revocation to
| appease certain clients; in the latter case they were prohibited
| by a Temporary Restraining Order (TRO) from performing timely
| revocation.
|
| Tim Callan from Sectigo has publicly lambasted DigiCert for these
| delays, since in both cases it seems DigiCert hasn't pushed back
| hard enough on its clients. In the latter case, there's concern
| that measures like TROs might be employed more often to stall
| revocation. Sectigo (and others in the WebPKI ecosystem) seem to
| want DigiCert to make the revocation policies very clear to
| clients and to ensure that clients can actually replace their
| certificates in a timely manner.
|
| Sectigo is clearly the most vocal but they don't seem to be the
| only ones telling DigiCert to get their delayed revocation under
| control. So the escalation to legal threats is really uncalled
| for, and DigiCert could face some very significant pushback for
| trying this tactic.
| RHSeeger wrote:
| What is the correct action when a TRO says a company cannot
| revoke the cert? Is it that the company will delay revocation,
| but will push the judicial system to resolve the issue as fast
| as possible?
| nneonneo wrote:
| DigiCert probably should have revoked every cert they _could_
| within 24 hours. Instead they just pushed the revocation of
| all 80,000+ certs out to five days.
|
| It's quite likely that many of their other clients pushed
| back on the 24-hour timeline (similar to what happened in
| their previous incident); I believe the delayed revocation
| issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1910322)
| hints at this. The TRO gave them a convenient excuse to delay
| all revocations without having to explain all over again why
| they made exemptions for their special clients.
|
| Heck, their status page
| (https://status.digicert.com/incidents/3sccz3v31lc9) even
| gives instructions for how to request a delayed revocation -
| even though the initial incident page
| (https://www.digicert.com/support/certificate-revocation-
| inci...) says clearly:
|
| "Any issue with domain validation is considered a serious
| issue by CABF and requires immediate action. Failure to
| comply can result in a distrust of the Certificate Authority.
| As such, we must revoke all impacted certificates within 24
| hours of discovery. No extensions or delays are permitted. We
| apologize if this causes a business disruption to you and are
| standing by to assist you with validating your domain and
| issuing replacement certificates immediately."
| Dylan16807 wrote:
| I think the eventual correct option is pretty clear.
|
| A TRO like that is based on the company loudly declaring that
| revoking will cause real damage. That means their use of
| certificates is incompatible with the web PKI rules and
| ecosystem. That means they need to be migrated out ASAP, with
| _every_ certificate authority refusing to take their
| business.
|
| Make that series of consequences clear, and companies will
| stop trying that trick.
| asmor wrote:
| I think that's pretty unrealistic, considering that X.509
| is a de-facto and de-jure standard in a lot of places that
| also ignore that requirement. It's not always up to that
| company to make it possible to replace certificates easily,
| it's an entire chain of vendors (I know at least
| Salesforce's process would fail here). Unless you want
| those people to run self-signed/private CA certs.
|
| The other option would be building in a way to revoke other
| CA's individual certificates if there's some consensus on
| them being compelled to not revoke them. Not sure if the
| status quo or this would be more dangerous, but if a TRO
| can compel a CA to not sign a revocation, can it also
| compel to sign a certificate?
| nickf wrote:
| The parent was correct - it's not about the company not
| using x509 certificates, but not using _publicly_ trusted
| certificates. There are myriad private /internal PKI
| solutions available from OpenSSL & bash to millions of
| dollars of solutions from Big Vendor. If you can't
| replace the publicly-trusted certificate quickly, you
| probably don't need it to be publicly-trusted in the
| first place.
| jeroenhd wrote:
| A company chooses its vendors. If its vendors turn out to
| be incompetent, they should choose again once the damage
| has been done. Just because Salesforce can't get their
| stuff together doesn't mean the CA industry needs to bend
| the knee. Let Salesforce figure out how to automate
| certificates, they've been in the business for long
| enough.
|
| If running a private CA or self-signed certificates are
| even viable, then there are plenty of other workarounds
| that can be put in place (i.e. not updating the CRL so
| the software doesn't know about revocations).
|
| If a CA's terms and legal documentation aren't tight
| enough to prevent a court from compel them to sign a
| certificate, that CA clearly cannot be trusted. Hopefully
| that issue can be fixed by writing clearer terms and
| better agreements, like people have been telling Digicert
| to do, but perhaps that's not possible at all. In that
| case, either the company should move to a jurisdiction
| where that kind of nonsense isn't possible, or it should
| be removed from global trust stores all together.
| asmor wrote:
| > A company chooses its vendors. If its vendors turn out
| to be incompetent, they should choose again once the
| damage has been done.
|
| You say that and yet Teams has 320 million people using
| it, and I bet almost none of them enjoy the experience.
| Sometimes you just have to work with what you're given.
| Given "incompetence", we might as well throw in all of
| Azure.
|
| > Let Salesforce figure out how to automate certificates,
| they've been in the business for long enough.
|
| That might be true for DV, but there's classes of
| certificates that take at least weeks to obtain, think
| BIMI VMC or codesigning.
| roblabla wrote:
| > You say that and yet Teams has 320 million people using
| it
|
| Yeah, there are tons of companies who chose Teams as a
| vendor (because it's bundled with other stuff), and
| inflict it on their employees. It was still absolutely
| their choice.
| hmmm-i-wonder wrote:
| DigiCert and other CA's need to write in their terms that
| failure to follow the policies and revocation timelines
| can/will result in termination of the contract. Alegeus
| should have been dropped as a customer as soon as the
| incident was resolved and refused further
| products/renewals.
|
| Use of a TRO protects you short term, but results in having
| to migrate to a new CA medium term. You can't stop them
| from using TRO's but you can make it not worth it.
| HelloNurse wrote:
| Is the CA allowed to stipulate that the customer (before
| being dropped) needs to pay a significant sum for, say,
| "expenses" if they resort to this kind of TRO?
| hmmm-i-wonder wrote:
| Possibly, I'm not sure if there is precedent for charging
| a company for taking valid legal actions, only vexatious
| in contract law. Probably depends on the legal
| jurisdiction and courts.
| kevin_thibedeau wrote:
| How can a court inhibit revocation when every CA declares
| their right to do so when you purchase a certificate?
| saurik wrote:
| I can declare my right to do something as much as I want...
| that doesn't mean I get to do it, certainly not if a judge
| decides I can't.
| AnthonyMouse wrote:
| The better question is, how do you prevent this tactic from
| working?
|
| For example, suppose there were required to be multiple
| parties who could issue a revocation, each in a different
| jurisdiction, and if any of them was ordered _not_ to do it
| then the others would be required to do it, and would have
| the technical capacity to do it but not be subject to the
| jurisdiction of that court.
| tadfisher wrote:
| Well, you can contest the motion for a TRO, for a start.
| Digicert failed to do so.
|
| You can stick with your policies and revoke the
| certificate within 24 hours, instead of delaying
| revocation until a case is open and a motion for a TRO is
| filed. Digicert failed to do so.
|
| You can stick with your policies and revoke the cert in
| face of the legal consequences, and deal with them
| accordingly. Again, Digicert failed to do so.
| tkfu wrote:
| Correction, the petition for the TRO was filed ex parte.
| Digicert did not have any opportunity to respond before
| it was granted.
|
| They certainly could have filed a response contesting the
| TRO. Then their customer could have filed another motion,
| and eventually (7 days later in this case) the judge
| would have ruled on the substance of it. Their judgement
| was that it would be preferable to work with the customer
| to resolve the technical issues with revocation, and
| submit a joint request to dismiss the TRO. The stated
| reasoning behind this was that it would be significantly
| faster than contesting the TRO. This is true: the certs
| were revoked and the TRO dropped within 3 days.
|
| I think the communication on that point was severely
| lacking, as they only clarified it three months later and
| after significant hectoring in two different bug threads:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910805#c43
|
| I also think it's reasonable not to take Digicert's
| statements at face value, given their history. But I
| think both of the points you made here are wrong:
|
| > You can stick with your policies and revoke the
| certificate within 24 hours, instead of delaying
| revocation until a case is open and a motion for a TRO is
| filed. Digicert failed to do so.
|
| Let's be clear about the timeline: Digicert notified
| their customers that the certs would be revoked. In
| between the time they notified the customer and the time
| of revocation (less than 24 hours), the customer got the
| ex parte restraining order. Are you suggesting that
| issuers should revoke certificates without notifying
| their users, so that the users don't have time to get an
| emergency TRO? I believe that would be in violation of
| the BRs.
|
| > You can stick with your policies and revoke the cert in
| face of the legal consequences, and deal with them
| accordingly. Again, Digicert failed to do so.
|
| By "revoke the cert in face of the legal consequences" do
| you mean "openly defy a valid and legal court order"?
| Because that would also violate the BRs.
| nickf wrote:
| Just to be clear, the whole incident covered over 80,000
| certificates. The TRO was applicable to only those of one
| subscriber - just over 70 certificates, yet caused the
| revocations of all 80k+ to be delayed.
| hmmm-i-wonder wrote:
| To add to this, 3 days after the TRO was filed both
| parties moved to vacate the TRO.
|
| DOCKET TEXT ORDER. 9 Joint Motion to Vacate 3 Order
| Granting Ex Parte Motion for TRO is GRANTED
|
| I'm not sure DigiCert could have done anything about the
| TRO or the impacted certs, but it should have been able
| to move forward with the revocation of all other
| certificates. That IMO is the real issue/failure,
| alongside the concern/impact of TRO's on security
| processes in the future.
| tristor wrote:
| > By "revoke the cert in face of the legal consequences"
| do you mean "openly defy a valid and legal court order"?
| Because that would also violate the BRs.
|
| Yes, I think this would have been appropriate action. If
| the contractual language is extremely clear between the
| CA and the subscriber, there is no legal basis on which
| the customer can prevent revocation. The fact they found
| a court that doesn't understand technology is frankly
| irrelevant. This detail is exactly why Tim and other
| parties are requesting the exact language of the
| agreement between Digicert and the subscriber that filed
| the TRO. A customer acting in bad faith and abusing the
| legal system does not compel you to violate your own
| contract terms, your terms under the CAB/BR, or to take
| actions which are detrimental to the entire Internet.
| This is exactly the type of circumstance where you do
| what you are required to do, and then sort it out
| afterwards. Any appeals court would have easily
| overturned the TRO as it has no legal basis.
| SpicyLemonZest wrote:
| > A customer acting in bad faith and abusing the legal
| system does not compel you to violate your own contract
| terms, your terms under the CAB/BR
|
| Yes, it absolutely does. "I think the court will agree
| with my view of what the contract says once the case is
| heard in full" is not a valid reason to disregard a TRO.
|
| > or to take actions which are detrimental to the entire
| Internet
|
| That would be harder. But a delayed revocation stemming
| from a flawed validation process, when the CA is
| responsible for the flaw and knows that the result of the
| validation was in fact correct, simply does not cause any
| detrimental effects to the entire Internet.
| SpicyLemonZest wrote:
| You could just require publishing the revocation
| immediately with an effective date in the future.
|
| Of course, if that system had been in place, DigiCert
| would probably be facing hundreds of lawsuits from
| businesses disrupted through no fault of their own rather
| than inside baseball PKI drama.
| michaelt wrote:
| It's Saturday and the courts are closed. You park in a car
| park I own.
|
| I have put up a sign saying I can instantly crush any
| crossover vehicles I like, as I consider them ugly and
| lacking in character.
|
| As I load your car into my crusher, you dispute the
| legality of my sign. But the court is closed until Monday,
| and the sign says I can crush your car instantly, no
| waiting.
|
| Should I be allowed to crush your car today? Or should I
| have to wait until Monday, so the disputed legality can be
| resolved?
| nkrisc wrote:
| That's actually a question for you. You won't truly know
| if you're allowed to until a court decides. You can
| choose to proceed and crush it, and then deal with the
| consequences of doing so if it turns out you were wrong.
| Likely you'll end up owing damages to the owner of the
| car, perhaps even punitive damages on top.
|
| I think the same applies here.
| hmmm-i-wonder wrote:
| The TRO actually means you aren't allowed, that's the
| point. It's an ordered injunction that legally obliges
| you from not acting until the courts can review the
| facts.
|
| The court IS deciding, temporarily.
| nkrisc wrote:
| In this case, yes. That's pretty cut and dry for the time
| being. However regarding the analogy I was replying to, I
| was pointing out that it's less a situation of what
| you're allowed to do and more one of what you believe
| you're allowed to do, and weighing the consequences of
| being wrong against upholding your stated terms. In other
| words, something you should probably discuss with a
| lawyer.
| account42 wrote:
| There is no TRO in the car park analogy.
| hmmm-i-wonder wrote:
| But there is here, so its a flawed analogy.
| dragonwriter wrote:
| > How can a court inhibit revocation when every CA declares
| their right to do so when you purchase a certificate?
|
| A court can rule that a term of a contract is void because
| it contradicts public policy, and it certainly can issue a
| TRO pausing an action which would otherwise be allowed by a
| contract while resolving a dispute related to it.
| rlpb wrote:
| The order would normally be a matter of public record, and
| the ecosystem should follow these events carefully and ensure
| never to issue a certificate to such a company again as it
| undermines the entire system.
| asmor wrote:
| Isn't Sectigo Comodo? Extra rich coming from them.
| nickf wrote:
| No, Sectigo is the PKI business that was carved out of
| Comodo, back in 2017. Comodo still exists, doing whatever
| they do. They have been totally separate entities since then.
| jeroenhd wrote:
| Reads to me like Digicert is hiding behind the TRO to protect
| themselves from upset customers over following the procedures
| they should have followed. I don't think they want to update
| their legal work to prevent customers from legal action over
| revocation, legal action has been working in their favor so
| far.
|
| For a company whose entire business is verifying company names
| and handling the CA process, Digicert seems rather unwilling to
| actually stick to the process. They can't help having a TRO
| filed against them by customers with incompetent tech such as
| Alegeus Technologies LLC, but this isn't the first time they've
| failed to follow the appropriate processes.
|
| Going through the courts to stop negative discourse seems
| rather crass for a CA. I don't understand why a company that
| has been subject of suspicion and doubt lime Digicert would do
| such a thing unless it's a last-ditch effort to get the heat
| off their backs.
|
| I'm sure their clients love Digicert for not forcing them to
| replace their certificates at the appropriate times, but
| they're in for a surprise when this whole thing blows up and
| they suddenly need to find another cert vendor when Digicert
| gets delisted.
| michaelt wrote:
| I dunno, seems to me in the TRO case there wasn't anything
| wrong with the certificates that had been issued. There
| wasn't any suggestion the the certificates had been issued to
| the wrong party or anything like that.
|
| When performing DNS verification there's supposed to be an
| underscore in a record, which protects sites that let _their_
| users control arbitrary subdomains - you know, dyndns and
| things like that. Digicert mistakenly didn 't ask the
| customer to include an underscore. Could have been a problem,
| if Alegeus was a dyndns provider - but they aren't.
|
| Then Digicert spent a load of the 24 hour disclosure window
| checking things on their end, then e-mailed the customers
| after office hours so when they get in the next day they
| barely had any time to respond.
|
| The only "incompetent tech" here is the assholes who decided
| on the policy that certificates be urgently revoked, as if
| they'd been issued to the wrong person, when they _hadn 't_
| been issued to the wrong customer.
| Thorrez wrote:
| >Could have been a problem, if Alegeus was a dyndns
| provider - but they aren't.
|
| AFAIUI, Alegeus was just 1 customer of Digicert. But
| Digicert has tons of customers. Just because there wasn't a
| problem for Alegeus (due to Alegeus not being a dyndns
| provider), doesn't mean there weren't problems for other
| Digicert customers.
| wbl wrote:
| There was something wrong. They hadn't been issued in
| accordance with the baseline requirements.
| hmmm-i-wonder wrote:
| The middle ground here is DigiCert should have been able
| and continued with the revocation of all the other
| certificates not impacted by the TRO within the policies
| and contractual agreements it has.
|
| The policy and its designers aren't incompetent, the
| intention is to ensure security first when fact-finding can
| take time to establish a subset of actual abuse. This is
| common in IT/Security and is clearly communicated in the
| policy customers agree to as well as something agreed on by
| all CA's. Security > Uptime is the baseline, and while you
| may disagree with that and have good arguments for the
| contrary, the argument is much wider than us and has been
| extensively hashed out in the working groups and
| discussions around this and other similar issues and is
| implemented this way by consensus while weighing valid
| arguments for different approaches and pros and cons.
| hmmm-i-wonder wrote:
| This seems like a misunderstanding of the TRO.
|
| Alegeus, a client of DigiCert, filed a court motion to stop
| DigiCert from revoking their certificate. The courts granted
| the temporary restraining order.
|
| There is no "update their legal work to prevent customers
| from legal action" that can avoid a temporary restraining
| order that is ordered by the courts to provide time to
| establish facts. Its simply how the legal process works.
| "they can't help but" seems unfair as the issue was the
| client Alegeus was unable to replace their certificates in
| the revocation timeline.
|
| Further "Going through the courts", they didn't go to the
| courts a client of theirs did.
|
| Digicert is absolutely not blameless. Outside of the TRO,
| they have failed to meet revocation windows before and yes,
| are likely using the TRO in this instance as a shield for
| that.
|
| However the TRO itself is a concerning development that all
| CA's need to consider depending on their legal jurisdiction,
| as it could put companies in a bind of being legally ordered
| to not to complete something they are contractually and
| legally required to do within the required timeline and
| interrupt standard security practices in cases like this.
| lokar wrote:
| Would a judge not consider potential harm to each party if
| the TRO is granted or not?
|
| If the CA could credibly point out they would face severe
| consequences for failure to revoke, could that have an
| impact on?
| bluGill wrote:
| They should - but only if a good lawyer points that out
| to the courts. Remember the courts are not experts in the
| technical details in question (and should not be - there
| are far more details that could come before the court
| than any human has time to learn), the job of lawyers is
| to quickly give them enough education to figure it out.
|
| In "Common law" when something goes before a court in
| slightly different situations the second court will look
| at what the last one decided to see if it makes sense and
| if they agree. Then the third time looks at both the
| previous. After many many cases courts will have seen all
| the arguments and made decisions and so there is no need
| to educate the court anymore, they will just look at what
| was done before and decide the same thing.
|
| Not all courts follow the "common law" above though, and
| I'm not clear how the other systems are different. (I've
| only lived in common law areas)
| lokar wrote:
| Well, if the CA forum had a really clear automatic
| sanction, I would expect the lawyer to raise that
| concern.
| bluGill wrote:
| Other comments have since suggested this was an emergency
| order where the CA didn't have their lawyer present. The
| courts said "don't do anything for a week while the real
| paperwork happens". I don't know the cases in question
| well enough to figure out if that is correct, but it
| seems reasonable. But also a week is not very long.
| hyperman1 wrote:
| An answer for the civil law countries I know of: The laws
| tend to be more detailled to avoid the situation you
| describe. Also, there is still a lot of referencing old
| cases, but it is considered guiding, not binding.
| mistrial9 wrote:
| hmm Isn't there an applicable standard of review for
| legal cases?
| hmmm-i-wonder wrote:
| The TRO was filed for and granted without DigiCerts
| input. By the time they responded, both parties filed to
| vacate the TRO (3 days later). Judges I expect weigh, to
| the best of their abilities, the perceived harm in
| granting/not granting the TRO. Considering the technical
| literacy of our court system I expect that leaves much to
| be desired
| lelandbatey wrote:
| It was filed and granted without DigiCerts input, yes,
| but DigiCerts lawyer showed up within 24 hours but then
| did _not_ do anything. See the courtlistener link:
| https://www.courtlistener.com/docket/68995396/alegeus-
| techno...
|
| Timeline:
|
| Jul 30, 2024 - ORDER granting 2 Ex Parte Motion for TRO.
| IT IS HEREBY ORDERED that DigiCert is prohibited from
| revoking the security certificates for the Alegeus
| Websites for a period of seven (7) day
|
| Jul 31, 2024 - NOTICE of Appearance by Jess M. Krannich
| on behalf of DigiCert
|
| ...
|
| Aug 3, 2024 - DOCKET TEXT ORDER. 9 Joint Motion to Vacate
| 3 Order Granting Ex Parte Motion for TRO is GRANTED
|
| Looking at that timeline, I can see why other CA forum
| members are asking "are you really taking this
| seriously?" The whole point is that such a restraining
| order was _definitely_ a bad call by the judge and should
| absolutely have been contested immediately by any legal
| team that was interested in representing the security
| interests of the CA Browser Forum (which, ya know,
| members of the CA Browser Forum should do in such cases,
| hence why they 're in the CA Browser forum). The fact
| that DigiCerts legal team did show up for that TRO and
| then did _not_ act to try to defend their ability to
| secure their certs, is a bad thing. If you want to be a
| CA, you gotta be willing to defend your need to act in
| the name of security, and to defend that in a social,
| business, and legal context. The point of CAs in the
| world is not to make money, it is to provide security
| services. Responding with "hey, but my legal obligations
| in my country mean I can't always do that" is a valid
| explanation, but it also means that the rest of the CA
| Browser Forum should probably not trust you.
| SpicyLemonZest wrote:
| Any CA who believes that the courts in their country
| could not prevent them from revoking certificates is
| confused at best.
|
| It's true that DigiCert could have refused to cooperate
| with Alegeus and fought the court order instead of
| cooperating with them to rotate the certificates ASAP.
| But that would have taken a lot longer than 5 days, even
| if they eventually won. If the CA Browser Forum has such
| a strong security interest in swift revocation, it's hard
| to see how further delaying the revocation in order to
| provoke a legal battle promotes that interest.
| JoshTriplett wrote:
| > Would a judge not consider potential harm to each party
| if the TRO is granted or not?
|
| > If the CA could credibly point out they would face
| severe consequences for failure to revoke, could that
| have an impact on?
|
| That would be an excellent reason to make sure to impose
| strict consequences on CAs without regard to whatever
| legal orders they're subject to, so that the _next_ CA
| can point to those consequences when arguing against
| things like a TRO. "If you grant this TRO, the net
| effect will not be that you get what you want; the net
| effect will be to destroy our business and still not get
| what you want".
| e40 wrote:
| Here is the last reply from Callan:
|
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c73
|
| Seems extremely reasonable to me.
| naitgacem wrote:
| Can someone point to where i can read some context?
| nneonneo wrote:
| DigiCert's legal threat, while obviously biased towards
| DigiCert, gives some context:
| https://bug1950144.bmoattachments.org/attachment.cgi?id=9468...
|
| For further reading, consider these two incidents which
| resulted in delayed revocation from DigiCert and a bunch of
| comments about how DigiCert should not be allowing delayed
| revocation:
|
| - Incident report
| https://bugzilla.mozilla.org/show_bug.cgi?id=1894560, delayed
| revocation report
| https://bugzilla.mozilla.org/show_bug.cgi?id=1896053 - incident
| due to the issuance of some certificates with incorrectly-
| capitalized phrases in the certificate's Business Category
| field; baseline requirements require revocation within five
| days but DigiCert dragged that out much further
|
| - Incident report
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322, delayed
| revocation report
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910805, DigiCert
| information page https://www.digicert.com/support/certificate-
| revocation-inci... - incident due to incorrect CNAME-based
| domain validation (failure to check that the CNAME started with
| an underscore); baseline requirements require revocation within
| 24 hours but DigiCert was stopped by the TRO and revoked after
| five days.
|
| Essentially, DigiCert has been delaying the revocation process
| (twice now) and people are unhappy about that. DigiCert has
| apparently attempted to silence those unhappy people (Sectigo
| and their representative Tim Callan) with legal action.
| skylerwiernik wrote:
| I don't believe that that's a legal filing. DigiCert never
| filed anything with the courts. That's just a letter to
| Sectigo threatening to sue.
| nneonneo wrote:
| Whoops, sorry, should've said legal threat not filing -
| fixed.
| tbrownaw wrote:
| So what's happened in the ... two months and a bit since the
| dates mentioned for the letters that this is apparently in
| reference to?
| nickburns wrote:
| Continued back-and-forth on Bugzilla:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c63
| nneonneo wrote:
| DigiCert posted a message fifteen days ago saying that "We have
| not used a legal team as a shield against accountability."
| (https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c74).
| This is clearly contradicted by their legal threat against
| Sectigo. Hence, Sectigo decided to post the "Threat of legal
| action" bug to make the community aware of what DigiCert
| actually did.
|
| Maybe if DigiCert had _not_ decided to make that comment,
| Sectigo would have been willing to stay quiet...
| xmodem wrote:
| Looking at the original report (
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322 ) I can see
| a couple of questions that DigiCert appears to be avoiding:
|
| > The public record of Alegeus Technologies LLC v. DigiCert shows
| no attempt by DigiCert to contest the court's order prior to the
| end of its preferred period of nearly 120 hours, even though such
| a motion could have freed DigiCert to revoke the certificates
| days earlier.
|
| and
|
| > The other question in comment 28 was for the language
| establishing DigiCert's right to revoke Alegeus Technologies
| certificates. DigiCert has waffled on this point, first implying
| that this language was to be found on its website but later
| refusing to confirm that the language on the site applied to
| Alegeus Technologies at the time.
|
| SPECULATION: Digicert may have offered special terms to Alegeus,
| and possibly other customers. They may have chosen not to dispute
| the TRO in court because they did not have grounds to do so under
| those agreements. They may also have included confidentiality
| terms in those contracts that prevented them from speaking about
| it.
|
| OPINION: I am surprised that the forum allowed the issue to be
| closed without the above quoted questions being satisfied, though
| it is possible they are addressed elsewhere, I have not done a
| complete reading of all the linked issues.
|
| EDIT to add: DigiCert has a response in a different thread here:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910805#c43 that
| would appear to contradict my speculation. Specifically
|
| > Even though DigiCert's TOU and MSA prohibited Alegeus from
| taking the action it did, once it filed for a TRO and the court
| almost immediately granted it, DigiCert's hands were tied
| DarkmSparks wrote:
| From the bugzilla
|
| "The actual reason for the underscore is so that services which
| allow users to create DNS records at subdomains (e.g. dynamic DNS
| services) can block users from registering subdomains starting
| with an underscore and be safe from unwanted certificate
| issuance. It serves the same purpose that /.well-known does for
| Agreed-Upon Change To Website, and that
| admin/administrator/webmaster/hostmaster/postmaster do for
| Constructed Email to Domain Contact. By using DNS records without
| underscores, DigiCert has violated a security-critical assumption
| that these services have made.
|
| Therefore, this is truly a security-critical incident, "
|
| That is a pretty brutal f' up of epic proportions... not sure any
| digicert cert should be trusted after that.
| mtlynch wrote:
| Direct link to the comment:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c10
|
| The author of that comment is Andrew Ayer, who also does great
| writeups about CA incidents and processes on his blog:
|
| https://www.agwa.name/blog/index
| kevincox wrote:
| My bigger concern is that what percentage of people will
| realize when delegating subdomains that having a leading
| underscore is a security vulnerability?
| terom wrote:
| It's fascinating that we've built a system that has expended
| perhaps several million dollars of engineering, legal and admin
| etc time over the issue of a single letter not being capitalized
| [1], without any demonstrable impact beyond a failure to meet
| ambiguous specifications.
|
| I do hope that dealing with all of the underlying issues around
| revocation etc makes the time and effort spent useful, and the
| Web PKI doesn't just mire itself in squabbling that blocks
| progress on actually meaningful issues.
|
| [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1894560
| xmodem wrote:
| I think that the Van Halen Brown M&M anecdote is relevant here.
|
| > and the Web PKI doesn't just mire itself in squabbling that
| blocks progress on actually meaningful issues.
|
| In your view, are there any meaningful issues going un-
| addressed currently?
| ocdtrekkie wrote:
| Entrust got torpedoed basically for deploying an improvement to
| the requirements for its certificates slightly before the
| improvement got officially approved, and the browser people
| collectively lost their crud over the concept that Entrust
| didn't immediately revoke all of their... perfectly valid,
| secure certificates immediately.
|
| Fundamentally, there is no accountability in the web PKI
| stewards. You want to talk about utter waste and incredible
| damage to the Internet, you can see it right here, in the
| people determining who is allowed to issue you sets of magic
| numbers that browsers have agreed are trustworthy, despite
| everyone involved behaving like complete children.
|
| And of course, the browser operators all have their own root
| CAs, so basically have extremely motivated reasons to want to
| eliminate every commercial provider that isn't one of the
| monopoly companies.
|
| Meanwhile:
|
| - Compromised certificates are basically a non-issue from a
| threat model standpoint, every certificate error people hit are
| just... expired certificates people didn't rotate yet.
|
| - Expired certificates cause issues for the majority of
| businesses at some point or another, making the internet
| increasingly fragile and unreliable.
| drudgemetal wrote:
| Entrust didn't get nuked over a single incident, they were
| nuked because of a pattern of behavior that extended back
| several years. https://webpki.substack.com/p/entrust-
| considered-harmful-par...
| ajb wrote:
| The bug that prompted this was more serious:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c74
|
| Basically the missing '_' was supposed to allow DNS providers
| who allow programmatic DNS record creation to filter out
| unauthorised certificate creation. So the certificates created
| without it could have been unauthorized by the owner of the
| domain they claim to certify.
| DeepSeaTortoise wrote:
| Why do you all think DigiCert handled this badly?
|
| 1. Bugs happen. Critical ones, too. They didn't try to brush this
| under the carpet, but admitted to it, acted to resolve it and
| were transparent about it.
|
| 2. They worked quickly to make it happen. Would 24h been nice?
| Sure, but 24h is not much shorter than 120h. In general, 24h is
| plenty of time for some exploits and 120h doesn't open the window
| to many more. It would have been very different if it took them
| months or years to resolve it.
|
| 3. They genuinely engaged with the critics on bugzilla, even
| after Sectigo's CCO went completely off the rails with trying to
| strip customers off legal recourse and demanding to blacklist
| those who try to make use of it.
|
| 4. They could have taken legal actions against Sectigo's CCO
| directly but took the extra step to ask them to stop this
| nonsense. They didn't demand anything more and even outlined
| steps Sectigo needed to take to prevent any legal problems down
| the line, like affirming that their CCO did not make these
| statements on behalf of Sectigo, an affirmation that they would
| notify their employees to not make any actions that would violate
| the laws mentioned in their letter, affirm that their CCO would
| be instructed not to violate any of the laws outlined in their
| letter and lastly confirm that, upon consulting with their CCO,
| they were able to conclude that his statements were not meant to
| harm DigiCert.
|
| The only ick is the short timeframe they expect a reply within,
| but that's sadly usual corporate US law practice...
|
| Basically that letter is the result of asking an US law firm for
| help and telling them to be nice about it and helping their
| opponent through the process.
| SAI_Peregrinus wrote:
| DigiCert miss-issued thousands of certs.
|
| Certificate authorities are required by contract (the
| CA/Browser Forum Baseline Requirements is a contract they agree
| to when being granted trust as a CA) to revoke miss-issued
| certificates within 24 hours of the time they learn of the
| miss-issuance.
|
| One of their customers filed a court case & got a temporary
| restraining order (TRO) preventing DigiCert from revoking [?]70
| of those certs within 24 hours.
|
| DigiCert then proceeded to delay revocation beyond the 24 hour
| limit *for all the certs, not just the [?]70 that the TRO
| covered.
|
| That is a violation of the CA/Browser Forum Baseline
| Requirements. Failure to revoke [?]70 certs because of a court
| order could be accepted, but failure to revoke thousands of
| other certs (putting customers at risk) should not be.
| genewitch wrote:
| DigiCert said they could not extricate the TRO from "critical
| infrastructure"
|
| Whether you believe them, or not, there was a valid reason
| given. They also said they fixed this particular issue
| internally, so I guess we'll see.
|
| DigiCert had over two million certs issued per the
| "businessEntity" case sensitivity bug thread, which was 2023.
|
| They revoked ~84,000 out of an abundance of caution - my read
| was they revoked every cert issued from the (now defunct)
| system digicert called "OEM" - which had the bug where it
| wasn't prepending an underscore.
|
| I just read the entirety of the underscore and the case
| sensitivity threads.
| SAI_Peregrinus wrote:
| Mostly correct. The issue is that only about 70 certs out
| of those 84k were actually covered by the TRO. They could
| have revoked the rest, as they were obligated to do by the
| Baseline Requirements. They didn't, they delayed.
| aaronmdjones wrote:
| Yes, bugs happen. However, DigiCert could have handled these
| incidents a lot better.
|
| For example, in the first issue (delayed revocation after
| incorrect capitalisation), DigiCert chose to not revoke several
| thousand certificates within the required 5-day window for
| reasons ranging from "is in use in embedded devices and can't
| be replaced in a timely fashion" (this is a failure of the
| customer's automation and planning, not a failure of DigiCert,
| and is not a reason for DigiCert to not revoke) to "is being
| pinned in a mobile app and the app needs to be rebuilt and
| pushed to app stores" (this is again a failure of the customer;
| the customer should have foreseen the event that their
| certificate would be revoked, and, if they really wanted to
| pin, should have also had a hot standby certificate from
| another CA pinned, for example). Further, DigiCert's own
| certificate policy at time included the following text (in
| section 1.4.2): DigiCert strongly discourages
| key pinning and shall not consider it a sufficient reason to
| delay revocation.
|
| DigiCert chose to ignore their own policy, and allowed this
| decision to delay revocation. That was entirely their own
| choice, to appease their own customers at the expense of
| complying with both their own policy and the Baseline
| Requirements.
|
| The other more recent incident (the TRO) would have restrained
| DigiCert from revoking approximately 70 certificates. DigiCert
| chose to extend this delay to tens of thousands of
| certificates, issued to other customers who sought no
| restraining orders. DigiCert chose to ignore their own policy,
| and allowed this decision to delay revocation. That was
| entirely their own choice, to appease their own customers at
| the expense of complying with both their own policy and the
| Baseline Requirements.
|
| Are you starting to see a pattern?
| darkarmani wrote:
| > They could have taken legal actions against Sectigo's CCO
| directly
|
| You are only suggesting they could have handled it worse. Why
| would they take legal action against the CCO for statements on
| a bug report other than to squash transparency?
| coldpie wrote:
| > Sectigo's CCO went completely off the rails with trying to
| strip customers off legal recourse and demanding to blacklist
| those who try to make use of it
|
| Can you provide a link to this? I looked Callan's comments on
| the cited bug[1] and none of it struck me as being even a
| little off the rails.
|
| [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1910322
| DeepPPP wrote:
| CNAME bug must have had something so bad DigiCert didn't want
| people poking around. Perhaps it's worth taking a closer look
| at the bug thread. They were also pressing really hard to
| close it. Something smells.
| DeepSeaTortoise wrote:
| From here on:
|
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910805#c30
| mightysashiman wrote:
| Streisand effect 101?
| kuon wrote:
| What is the actual impact on infrastructure? Is it just some old
| certificates being valid past expiration?
| drpossum wrote:
| I don't think DigiCert has the power to change literally every
| internal validating mechanism to check if a certificate is
| expired or not. But you seem to know more, so I'd ask you
| expound on that.
| aaronmdjones wrote:
| Certificate authorities have enormous trust placed upon them by
| every Internet user (whether they know it or not). Commensurate
| with this trust, they have enormous responsibilities. As the name
| implies, the Baseline Requirements are the _minimum_ standard
| they should achieve. If they can 't even do this (being unable or
| unwilling to revoke issued certificates within the required time-
| frame), then they do not deserve this trust, and it should be
| removed.
|
| I understand that the TRO prevented them from revoking
| approximately 70 certificates, and there really is nothing they
| could have done differently in that case. Their other revocation
| failings are inexcusable.
| csense wrote:
| This is a really stupid situation all around.
|
| To issue a cert for a client (say example.com), the CA sends a
| random number challenge to the client (say 12345678). The client
| then responds by making a DNS record for _12345678.example.com
| but due to a bug, DigiCert accepted 12345678.example.com instead.
|
| Apparently there's a rule that says any such certs must be
| considered mis-issued and must be revoked by the issuing CA
| within 24 hours.
|
| The CA talked to some clients who said it would be too much
| trouble to replace their cert on such short notice. One of those
| clients filed a lawsuit with the court system, which responded by
| issuing a restraining order telling the CA not to revoke the
| cert.
|
| First of all, this is a dumb reason to revoke certs. It's
| difficult to imagine a situation where any of the mis-issued
| certificates correspond to an actual compromise of somebody's
| website by a bad actor. But we can perhaps give the benefit of
| the doubt: I think the intent here is a fail-safe procedure is
| implemented so revocation is triggered by a broad class of
| circumstances, which is constructed so that it's easy to check.
|
| Second, this is a dumb thing for a website to file a lawsuit
| about. If you have to replace the cert on your website because
| your CA made a booboo, the engineer-hours needed to change out
| your new cert are far less than the lawyer-hours needed to try to
| work through the court system.
|
| Third, it's dumb for people to blame DigiCert for the TRO. The
| TRO was filed by a service provider called Alegeus. If DigiCert
| doesn't follow the TRO, they'll be in contempt of court.
| Theoretically, a DigiCert employee who presses the "Revoke"
| button at this point could be sent to jail for it!
|
| Fourth, a certificate revocation is basically a signed message
| that says "News Flash: I think this certificate might not
| correspond to this website." A court preventing the publication
| of such a message is equivalent to a court preventing a newspaper
| from publishing an article. This is called "prior restraint" and
| it has a high bar in the US. Prior restraint analysis was not
| performed by the court before the TRO was issued. I think the
| court violated DigiCert's due process rights (by not analyzing
| whether prior restraint applies) and DigiCert's free speech
| rights (by not denying the TRO as it would be prior restraint).
| (That browsers have a hair-trigger response to certain kinds of
| news flashes and will take drastic action isn't the fault of the
| publisher of the news.)
|
| Fifth, it's obvious there ought to be some technical means to
| prevent this kind of situation in the future: What is the
| protocol to handle a case of a CA refusing (or court-ordered not
| to) revoke certs known to be compromised? I think you need to
| have a system that allows a quorum of CA's _not including the
| issuing CA_ to speedily revoke certificates, or an entire CA. (A
| public real-time quorum of a dynamic set of validators passing
| messages they want to reach consensus on: Perhaps this is a good
| use case for a blockchain?) Of course you also have to deal with
| the problem of, what if the next TRO targets the entire validator
| set? You 'd need to be sure those participants are
| jurisdictionally diverse (or anonymous) so it would be difficult
| for a single entity to compromise the entire system at once.
| chuckadams wrote:
| Thank you for explaining what all of this was about. Everyone
| else keeps talking in abstract and apocalyptic terms.
|
| An underscore. WTF.
| lelandbatey wrote:
| 1. It's not a dumb reason to revoke certs, it's a VERY good
| reason to revoke certs. Lot's of hosting providers allow you to
| register subdomains with them e.g. mywebsite.squarespace.com;
| but Squarespace won't allow you to register a domain that
| starts with an underscore such as
| _123-mywebsite.squarespace.com because starting with an
| underscore is used for these DNS-based SSL checks. The concern
| was that DigiCerts system allowed you to receive an SSL cert
| for squarespace.com ( _the parent domain_ ) by registering
| 12345678.squarespace.com ( _no underscore_ so as far as
| Squarespace knows, a normal subdomain). This is not a niche use
| of this; it 's literally why the underscore prefix was there:
| https://bugzilla.mozilla.org/show_bug.cgi?id=1910322#c10
|
| 2. It is a pretty dumb thing for a website to file a lawsuit
| about, that is true.
|
| 3. DigiCert deserves a lot of blame for the TRO. DigiCert
| received one TRO for one client who had 72 certs from
| DigiCert[1]. What they should have done is continue with
| revoking the other 80,000+ certificates in the 24 hour time
| window and then inform the CABF that they have 72 (or however
| many) holdout certs caused by legally binding temporary
| restraining order. They should have been proactive in complying
| and in their communication. DigiCert did not do that though.
| They delayed all 80,000+ certificates for 5 days instead.
| That's _not good enough_.
|
| [1] - https://www.courtlistener.com/docket/68995396/2/alegeus-
| tech...
| registeredcorn wrote:
| Fascinating. I think there was a fair amount of snark on both
| sides, but I do think some good points were raised by both, as
| well.
|
| 1) To DigiCert's point: If certs need an emergency revocation
| _but_ it will impact a service which say: provides life saving
| services, or keeps the electricity on for the majority of a
| country, would it not be wise to file them as a one-off
| "exceptional circumstance". I think that common sense should
| prevail and everyone can agree that, "Yes, computer security is
| absolutely essential. Essential services are _also_ essential. "
| I wish that that was the direction the debate had gone in.
|
| For instance, What is considered an 'exceptional circumstance'?
| What kind of services are covered, and what are not?
|
| Personally, I would think that things like: health, heat, water,
| electricity, and physical security (prison and law enforcement)
| are all potentially essential areas. They are industries that
| ought be able to _request_ an emergency, 48-hour exception if
| they know they can 't meet it within 24 hours and their services
| will go down as a result. I feel like two days should be enough
| time for just about any organization to work through a
| certificate issue, unless it's a long holiday, or something very,
| very niche.
|
| I think that, to a degree, Tim Callan (Setigo CEO) was being
| unreasonable in expecting DigiCert to not offer any kind of
| possibility for exceptions. Some services should not go down,
| just because it goes against the principals of computer security.
| It hate saying that, but it's true. Keeping the ICU running
| matters more than whether the hospital is following best security
| practices during an emergency.
|
| Could it cause more problems by ignoring best practices?
| Possibly! Will enforcing best practices possibly kill someone? If
| the answer is anything other than a firm "No", then it is
| secondary to protecting that service.
|
| 2) To Sectigo's point: We should not allow any CA to hide behind
| Policies or poorly written MSAs. If things went the way they did
| because they were _allowed_ to go that way, then that means you
| should learn from those things in the post mortem! Take steps to
| shore it up! Try and prevent other companies from following suit,
| otherwise more _will_ take action whenever it meets their own
| best interest. It is disappointing that this part seems to fell
| into snarky retorts too, because there were some legitimate means
| to discuss this.
|
| For instance: Instead of barring from someone from being
| _allowed_ to file a TRO, simply have an agreement in place that
| _before_ any legal action like a TRO is filed, the customer will
| meet with the CA and a emergency mediator. Just take 30 minutes
| to one hour to see if you can work things out _before_ the
| customer submits a TRO!
|
| It seems logical, right? If a customer has the cycles to file for
| a TRO, they should have the time to spare talking to the company
| they are filing a TRO against. Explain a clear reasons to a
| mediator _why_ the TRO is needed, and _why_ they can 't get it
| done in time. Assuming that the customer can explain all of that
| in clear terms, it would then be obvious for DigiCert to
| _acknowledge_ that level of criticality and "exceptional need",
| and offer their customer an emergency, temporary exemption.
|
| Neither side wants a TRO! It makes DigiCert look weak during an
| emergency, and it makes Alegeus (the company that filed the TRO)
| look incompetent, desperate, and underhanded.
|
| The crux of what Tim Callan (Sectigo) was getting at, is that
| there needs to be a correction to DigiCert's policies. It's
| blaringly obvious. DigiCert were, in a way, "legally attacked" in
| a manner that should be prevented in the future, as best they can
| prevent it.
|
| DigiCert lackadaisically shrugging their shoulders and saying
| "B-But...that goes against Mozilla policy!" is just deflection
| and meaningless. DigiCert can go to the trouble of sending legal
| council after Sectigo for comments on Bugzilla, but they _can 't_
| use legal council to protect DigiCert from surprise TRO's?
| Really? Bugzilla feedback... _that_ is the legal issue? Not
| DigiCert being sucker punched by their own customers?
|
| The whole thing is just so aggravating. Both sides need to get
| over themselves and try to work together. They don't need to like
| each other, but they should do what is best for the industry.
| Each side sending out daddy lawyer to fight for them completely
| misses the point, and kills the chance for constructive feedback.
| SpicyLemonZest wrote:
| > It seems logical, right? If a customer has the cycles to file
| for a TRO, they should have the time to spare talking to the
| company they are filing a TRO against. Explain a clear reasons
| to a mediator why the TRO is needed, and why they can't get it
| done in time. Assuming that the customer can explain all of
| that in clear terms, it would then be obvious for DigiCert to
| acknowledge that level of criticality and "exceptional need",
| and offer their customer an emergency, temporary exemption.
|
| Have you read the court filings? Alegeus did indeed have this
| discussion, explaining that they have a detailed manual process
| for dozens of distinct websites, and could not complete it
| within the notice period even though they began working on it
| immediately. They escalated it all the way up to the DigiCert
| CEO, and filed for a TRO only after DigiCert told them -
| exactly as the Bugzilla discussion participants wanted - that
| no exemption would be offered. That's the key point they seem
| to be missing: taking a hard line on the revocation timeline
| does not _deter_ legal action but generates more of it.
| ziddoap wrote:
| Bug has been updated with a DigiCert response.
|
| Obviously draw your own conclusions, but I actually laughed out
| loud at the following statement from DigiCert:
|
| " _In reality, our letter to you was consistent with our desire
| to promote open and honest dialogue._ "
___________________________________________________________________
(page generated 2025-02-25 23:01 UTC)