https://www.jefftk.com/p/history-of-https-usage

  * Jeff Kaufman
  * Posts
  * RSS
  * <<RSS
  * Contact

---------------------------------------------------------------------

History of HTTPS Usage March 8th, 2018
                                  tech

"What should I use HTTPS for?" is a question whose answer has changed
over the years. Here's an attempt to piece the history together.

The web started off as just HTTP. This allowed for an enormous amount
of things, but online shopping wasn't one of them. The problem was,
sending credit card numbers over HTTP opened them up to theft: anyone
between you and the server could keep a copy of your card
information. Netscape saw this as an opportunity, if they could make
a safe way to do this, using their browser and server. This would be
good for them and good for the success of the web as a whole.

The decided to build SSL, a protocol for creating a secure channel
between a client and server, which they could then run HTTP over.
They called the combination HTTPS.

It was initially viewed as just a thing for credit cards. For
example:

    Netscape charges $1,495 for an entry-level server; adding the
    ability to handle secure credit card transmissions bumps the
    price up to $5,000
      --The rise of Netscape, July 1995

Later that year, when there was a vulnerability found in the
protocol, reports only discussed the impact on credit cards:

    The problem in the Netscape Navigator was discovered by two
    first-year computer science graduate students at the University
    of California at Berkeley. They said a criminal could break into
    the software's coding system in less than a minute and gain
    access to credit card information.
      --Netscape: Internet security flaw can be fixed, September 1995

And:

    Several weeks ago Mr. Goldberg, along with another student, David
    Wagner, discovered a troubling flaw in the security of a Netscape
    Communications Corporation software program designed to let
    people safely make credit card purchases via the World Wide Web.
      --The New Watchdogs of Digital Commerce, October 1995

History from the late 1990s and early 2000s is hard to dig up, with
so many dead sites and broken urls, but one place I looked to see
things changing was old software manuals. Unfortunately, vendors seem
to have been very wary about giving specific advice about what sort
of pages needed HTTPS. Instead, we get things like mailing list
administration software saying:

    If security is a concern, consider securing access to the
    LISTSERV Maestro servers with encrypted communication, so that
    intruders cannot listen in on the communication between browser
    and server, and cannot gain knowledge about the data exchanged or
    spy out passwords.
      --Securing Access With SSL, May 2002

By 2004 HTTPS was being described as "secur[ing] web sites for online
banking, stock trading and retailing, and users were generally being
taught that they should expect to see https:// for their bank or when
buying things, but not most other places. For example, here's some
advice from 2005 about forums that have logins but don't support
HTTPS:

    Most message boards and other sites that have membership options
    do not handle your login information through a secure layer,
    which means it is unencrypted, and anyone who manages to
    intercept data packets can read it.

    Does this mean you should never use sites like this? No. What it
    does mean is, you should have a password for unsecure sites which
    is different from passwords you use for banking and credit card
    transactions, or anything else which requires security.
      --Should I provide passwords at unsecure sites?, May 2005

It seems like around this time it became common practice to use HTTPS
for logins even if the rest of the site was HTTP-only. For example,
in 2006 the Wikipedia page for HTTPS was was edited to add that in
addition to "payment transactions" HTTPS was also commonly used for
"corporate logons".

Then we had a step backwards: sites realized they could put login
forms on their home pages without serving the home page as HTTPS, by
setting the form to POST to an HTTPS endpoint:

    After years of training customers to trust only SSL-enabled
    sites, banks are shifting their online banking logins to the
    unencrypted home pages of their websites. Although the data is
    encrypted once the user hits the "Sign In" button, the practice
    runs counter to years of customer conditioning, as well as the
    goals of the browser makers. Three of the five largest U.S. banks
    now display login forms on non-SSL home pages, including Bank of
    America, Wachovia and Chase, as well as financial services giant
    American Express.
    ...
    Mindful of this, many of the banks using homepage logins include
    a link to security information. "You may notice when you are on
    our home page that some familiar indicators do not appear in your
    browser to confirm the entire page is secure," Bank of America
    notes in its security note, accessed by clicking an icon on the
    login form. "Those indicators include the small 'lock' icon in
    the bottom right corner of the browser frame and the 's' in the
    Web address bar (for example, 'https'). To provide the fastest
    access to our home page for all of our millions of customers and
    other visitors, we have made signing in to Online Banking secure
    without making the entire page secure. Please be assured that
    your ID and passcode are secure and that only Bank of America has
    access to them."
      --Banks Shifting Logins to Non-SSL Pages

As Microsoft pointed out at the time, this isn't secure against
active attacks since the login page could be modified in transit.

This was still common in 2009:

    Some major websites--such as Facebook, Vox and others (but not
    Metafilter!)--have users enter username & password on an http
    page rather than an https page. Is this as secure as logging in
    via https:// and if so, why?
      --Are login boxes on HTTP pages secure?, December 2009

Around this time Gmail introduced an option to always use HTTPS for
your account:

    We use https to protect your password every time you log into
    Gmail, but we don't use https once you're in your mail unless you
    ask for it (by visiting https://mail.google.com rather than http:
    //mail.google.com). Why not? Because the downside is that https
    can make your mail slower. Your computer has to do extra work to
    decrypt all that data, and encrypted data doesn't travel across
    the internet as efficiently as unencrypted data. That's why we
    leave the choice up to you.
    ...
    We've added an option to Settings to always use https. If you
    don't regularly log in via unencrypted wireless connections at
    coffee shops or airports or college dorms, then you might not
    need this additional layer of security. But if you want to always
    use https, then this setting makes it super easy.
      --Making Security Easier, July 2008

[Disclosure: I work for Google, though this post isn't a work
project.]

And they made HTTPS the default a year and a half later:

    If you trust the security of your network and don't want default
    https turned on for performance reasons, you can turn it off at
    any time by choosing "Don't always use https" from the Settings
    menu. Gmail will still always encrypt the login page to protect
    your password. Google Apps users whose admins have not already
    defaulted their entire domains to https will have the same
    option.
      --Default https access for Gmail, January 2010

In 2010 they also added HTTPS support for search:

    Years ago Google added SSL encryption to products ranging from
    Gmail to Google Docs and others, and we continue to enable
    encryption on more services. Like banking and e-commerce sites,
    Google's encryption extends beyond login passwords to the entire
    service. This session-wide encryption is a significant privacy
    advantage over systems that only encrypt login pages and credit
    card information.
    ...
    when you search using SSL, you won't see links to offerings like
    Image Search and Maps that, for the most part, don't support SSL
    at this time. Also, since SSL connections require additional time
    to set up the encryption between your browser and the remote web
    server, your experience with search over SSL might be slightly
    slower than your regular Google search experience.
      --Search more securely with encrypted Google web search, May
    2010

Deployments where the site supported HTTPS but wouldn't serve it to
you unless you explicitly chose to go there had been around since at
least the 2004 launch of Gmail, but this was a big one and prompted
the creation of the HTTPS Everywhere browser extension:

    Today EFF and the Tor Project are launching a public beta of a
    new Firefox extension called HTTPS Everywhere. This Firefox
    extension was inspired by the launch of Google's encrypted search
    option. We wanted a way to ensure that every search our browsers
    sent was encrypted. At the same time, we were also able to
    encrypt most or all of the browser's communications with some
    other sites: [list of sites]
      --Encrypt the Web with the HTTPS Everywhere Firefox Extension,
    June 2010

This extension kept a list of sites that provided HTTPS versions,
along with regexps to go from an HTTP url to an HTTPS one. This was
not always as simple as adding an 's' to the protocol, since many
sites used to do things like having http://en.wikipedia.org for HTTP
but https://secure.wikimedia.org/wikipedia/en for HTTPS Their FAQ
describes their thinking:

    Q. What if the site doesn't support HTTPS, or only supports it
    for some activities, like entering credit card information?

    A. You could try to contact the site and point out that using
    HTTPS for all site features is an increasingly common practice
    nowadays and protects users (and sites) against a variety of
    Internet attacks. For instance, it defends against the ability of
    other people on a wifi network to spy on your use of the site or
    even take over your account. You can also point out that credit
    card numbers aren't the only information you consider private or
    sensitive.

    Sites like Google, Twitter, and Facebook now support HTTPS for
    non-financial information--for general privacy and security
    reasons.
      -- HTTPS Everywhere FAQ, June 2010

While some users installed the extension, most, as with all things
you have to go out of your way to enable, did not. For example, I
don't think I did.


Over the course of the decade people had been shifting from mostly
wired internet connections to mostly wireless ones. It had become
common for people to connect to WiFi in public places like coffee
shops, and this changed the effective security level of HTTP. Instead
of an attacker needing to subvert your ISP or an intermediate
network, they just needed to listen to what was being sent in the
open over WiFi. Using HTTPS only for login but then sending
authentication cookies in the clear on each request had never been
all that secure, but now it was easily exploitable. Still, users and
companies mostly didn't worry about it.

Then in October 2010 Firesheep came out. This was an active exploit,
packaged as an easy-to-use Firefox extension. Anyone could go down to
the local coffee shop, open up Firesheep, sniff people's cookies, and
impersonate people on Facebook, Twitter, etc. The author of Firesheep
wrote:

    The only effective fix for [session hijacking] is full end-to-end
    encryption, known on the web as HTTPS or SSL. Facebook is
    constantly rolling out new "privacy" features in an endless
    attempt to quell the screams of unhappy users, but what's the
    point when someone can just take over an account entirely?
    Twitter forced all third party developers to use OAuth then
    immediately released (and promoted) a new version of their
    insecure website. When it comes to user privacy, SSL is the
    elephant in the room.
      --Firesheep, October 2010

Companies hurried to at least offer an HTTPS option, and Facebook
announced an opt-in setting in January 2011, along with a goal of
having HTTPS as the default at some point:

    Starting today we'll provide you with the ability to experience
    Facebook entirely over HTTPS. You should consider enabling this
    option if you frequently use Facebook from public Internet access
    points found at coffee shops, airports, libraries or schools.
    ...
    There are a few things you should keep in mind before deciding to
    enable HTTPS. Encrypted pages take longer to load, so you may
    notice that Facebook is slower using HTTPS. In addition, some
    Facebook features, including many third-party applications, are
    not currently supported in HTTPS. We'll be working hard to
    resolve these remaining issues. We are rolling this out slowly
    over the next few weeks, but you will be able to turn this
    feature on in your Account Settings soon. We hope to offer HTTPS
    as a default whenever you are using Facebook sometime in the
    future.
      --A Continued Commitment to Security, January 2011

Three months later, Twitter added support:

    we are adding a user setting that lets you always use HTTPS when
    accessing Twitter.com.
    ...
    This will improve the security of your account and better protect
    your information if you're using Twitter over an unsecured
    Internet connection, like a public WiFi network, where someone
    may be able to eavesdrop on your site activity. In the future, we
    hope to make HTTPS the default setting.
      --Making Twitter more secure: HTTPS, March 2011

Google was already using HTTPS by default for Gmail, and offered an
option to use search over HTTPS, but search was still unencrypted by
default. In October they rolled out HTTPS by default for logged in
users:

    As search becomes an increasingly customized experience, we
    recognize the growing importance of protecting the personalized
    search results we deliver. As a result, we're enhancing our
    default search experience for signed-in users. Over the next few
    weeks, many of you will find yourselves redirected to https://
    www.google.com (note the extra "s") when you're signed in to your
    Google Account. This change encrypts your search queries and
    Google's results page. This is especially important when you're
    using an unsecured Internet connection, such as a WiFi hotspot in
    an Internet cafe.
      --Making search more secure, October 2011

Other providers moved slower. For example, Yahoo Mail didn't even
offer the option to always use HTTPS until late 2012, when they
apparently introduced it without an announcement. Small companies
were generally even slower to add support, unless they were
explicitly privacy or security focused.


In June 2013 Edward Snowden revealed how much spying the US had been
doing, and how pretty much all HTTP traffic was being monitored. It
looks to me like these disclosures really changed how technical
people were thinking about HTTPS. Before the idea was that you should
use HTTPS just for sensitive things (purchases, financial stuff,
logged in sessions) but most things could stay in the clear (news
sites, blogs, reference), while after the disclosures there was much
more support for the idea that eventually everything should be over
HTTPS.

In the short term, companies that hadn't been using HTTPS
consistently for sensitive things scrambled to roll it out:

    We now use https by default for all Facebook users. This feature,
    which we first introduced as an option two years ago, means that
    your browser is told to communicate with Facebook using a secure
    connection, as indicated by the "https" rather than "http"
      --Secure Browsing By Default, July 2013

Wikipedia wrote:

    Our current architecture cannot handle HTTPS by default, but
    we've been incrementally making changes to make it possible.
    Since we appear to be specifically targeted by XKeyscore, we'll
    be speeding up these efforts.
      --The future of HTTPS on Wikimedia projects, August 2013

And then followed up with HTTPS by default for logged in users later
that month.

Yahoo was again on the slower end, finally deploying it at the
beginning of 2014:

    As we promised back in October, we are now automatically
    encrypting all connections between our users and Yahoo Mail.
    Anytime you use Yahoo Mail - whether it's on the web, mobile web,
    mobile apps, or via IMAP, POP or SMTP- it is 100% encrypted by
    default and protected with 2,048 bit certificates.
      --HTTPS Now Default in Yahoo Mail, January 2014

By this point most of the barriers to HTTPS adoption had been
resolved. You no longer needed an IP for each host [1], and the extra
computation was too low to matter, but certificates were still
annoying. For example, this was the main reason I didn't offer HTTPS
on any of my (very much "long tail") sites. Let's Encrypt was founded
to fix this:

    Then why don't we use TLS (the successor to SSL) everywhere?
    Every browser in every device supports it. Every server in every
    data center supports it. Why don't we just flip the switch?

    The challenge is server certificates. The anchor for any
    TLS-protected communication is a public-key certificate which
    demonstrates that the server you're actually talking to is the
    server you intended to talk to. For many server operators,
    getting even a basic server certificate is just too much of a
    hassle. The application process can be confusing. It usually
    costs money. It's tricky to install correctly. It's a pain to
    update.

    Let's Encrypt is a new free certificate authority, built on a
    foundation of cooperation and openness, that lets everyone be up
    and running with basic server certificates for their domains
    through a simple one-click process.
      --Let's Encrypt: Delivering SSL/TLS Everywhere, November 2014

They announced the project in November 2014, started public beta a
year later, and opened fully in April 2016. They're now covering 25M
domains, including this one.

At this point browsers could start pushing pretty heavily towards
HTTPS. As they introduced new features, especially ones with any
security impact, they made them only available to HTTPS pages. (
service workers, getUserMedia, Prefer Secure Origins For Powerful New
Features) In September 2016 Chrome announced plans to eventually mark
all HTTP sites as insecure:

    Historically, Chrome has not explicitly labelled HTTP connections
    as non-secure. Beginning in January 2017 (Chrome 56), we'll mark
    HTTP pages that collect passwords or credit cards as non-secure,
    as part of a long-term plan to mark all HTTP sites as non-secure.
      --Moving towards a more secure web, September 2016

And this is still on track:

    For the past several years, we've moved toward a more secure web
    by strongly advocating that sites adopt HTTPS encryption. And
    within the last year, we've also helped users understand that
    HTTP sites are not secure by gradually marking a larger subset of
    HTTP pages as "not secure". Beginning in July 2018 with the
    release of Chrome 68, Chrome will mark all HTTP sites as "not
    secure".
      --A secure web is here to stay, February 2018

You can see the overall rise of HTTPS from Google's HTTPS
Transparency Report:

[chrome-https-adoption-graph]

You can also see it from Firefox data, via Let's Encrypt stats:

[firefox-https-adoption-graph]

We're not yet to where there is only https://, but it looks like
we'll be there within a few years.


[1] In 2003 RFC 3456: Transport Layer Security (TLS) Extensions came
out, which defined Server Name Indication (SNI):

    TLS does not provide a mechanism for a client to tell a server
    the name of the server it is contacting. It may be desirable for
    clients to provide this information to facilitate secure
    connections to servers that host multiple 'virtual' servers at a
    single underlying network address.
      --RFC 3456, June 2003.

For example, I host jefftk.com, trycontra.com, lilywise.com, and
several other sites all on the same server with a single IP address.
When a request comes in, it specifies which of these it wants with
the Host: header:

    GET /index HTTP/1.1 Host: www.jefftk.com ...

Before SNI, if I had been trying to host all of these on HTTPS at one
IP, the conversation would have started with a request for a
certificate. But how would I know which certificate to provide? If
they're trying to visit jefftk.com they want Cert A, while for
trycontra.com they want Cert B, etc. At the time you could work
around this by using different IP addresses for the different sites,
and then you know a request on IP A should get Cert A, while IP B
should get Cert B. This was expensive and awkward, however, and
limited the deployment of HTTPS.

(There was another way to do it, subjectAltName to make a single
certificate that was good for multiple domains. This would have been
fine for my sites, and actually is still how I handle certs, but it
doesn't work well for hosting providers or anyone else with a lot of
domains that change often.)

With SNI, the browser includes the domain name of the site it's
trying to visit when it first starts setting up an SSL connection.
This lets the server pick the right certificate to serve, without
relying on the multiple IP addresses hack.

On the other hand, as long as any browsers you want to support don't
handle SNI yet you can't switch over to it. This was a very long
transition. Since SNI was handled by the SSL library, support for SNI
was usually dependent on the operating system version. Windows XP and
Android Gingerbread were the last two widely used OSes to not support
SNI, and they stuck around for a long time. For example, in July 2013
Gingerbread and earlier were still at 41%, in July 2015 they were at
6%, and they didn't drop below 1% until June 2017. (See pretty charts
.) if you specifically visited https://gmail.com.

 
-
Thoughts on Existential Risk Touring As a Contra Band
-

Comment via: google plus, facebook

Recent posts on blogs I like:

Bad Therapy Review Index

Abigail Shrier's bad book

via Thing of Things August 7, 2024

Relaxing my standards on housekeeping

Getting to "good enough" The post Relaxing my standards on
housekeeping appeared first on Otherwise.

via Otherwise July 31, 2024

Categories of leadership on technical teams

overall direction * people management * project management
* technical leadership * example divisions of labor

via benkuhn.net July 21, 2024

more     (via openring)

More Posts

  * Growing Independence

  * Bets, Bonds, and Kindergarteners

  * Lifelong investments

  * How to parent more predictably

  * Equal Parenting Advice for Dads

open
---------------------------------------------------------------------

  * Jeff Kaufman
  * Posts
  * RSS
  * <<RSS
  * Contact