[HN Gopher] See this page fetch itself, byte by byte, over TLS
___________________________________________________________________
See this page fetch itself, byte by byte, over TLS
Author : gmac
Score : 1118 points
Date : 2023-05-10 06:51 UTC (16 hours ago)
(HTM) web link (subtls.pages.dev)
(TXT) w3m dump (subtls.pages.dev)
| can16358p wrote:
| While we all (HN audience) know roughly what kind of things are
| going on, seeing it in all the delicate details is fascinating.
|
| Just thinking that all these bytes and everything that's
| happening have 6 more layers below all the way to physical
| electrical/electromagnetic flow, all routed potentially through
| many different ISP backbones and possibly international routes,
| all happening in roughly less than 1/10 of a second is even more
| fascinating.
| [deleted]
| xenadu02 wrote:
| It reminds me of being a teenager in the late 90s and the early
| days of the internet. I discovered entirely on my own that I
| could telnet to port 80, type "GET / HTTP/1.1\n\n" and the
| server would send me the headers + page content. Shortly after
| I discovered the same worked for SMTP.
|
| I was very far from the first person to have this revelation
| but it was definitely an eye-opening "there is no magic, it's
| all just software" moment for me. It fundamentally changed the
| way I think about computers at every level and inspired me to
| look "under the hood" at everything... how CPUs implement
| superscalar OOO execution. How atomic operations work at the
| CPU level. What a syscall actually is. How subroutines are
| called (and calling conventions). How dynamic linkers work.
|
| You don't have to be an expert at all these things but it is
| like a superpower to understand the basics of every layer.
| Netcob wrote:
| Same! I didn't even know anything about telnet or
| programming. I was using Klik&Play or MultiMedia Fusion to
| make games, and some component supported TCP, so I opened a
| port using that and just for fun connected to it with a
| browser. Then I saw a request, so I used that the other way
| around on a real web server and it worked. Same thing with
| SMTP.
| swyx wrote:
| > What a syscall actually is
|
| ive only regarded it as a literal system call, the lowest
| possible level, "language agnostic" api that does a thing in
| the OS. do you have some deeper insight?
| harikb wrote:
| I was lazier and just did "GET / HTTP/1.0\n" and saved one
| character :P
|
| Edit: I am probably wrong about "1.0", might have been that I
| just did "GET /" and saved 8+ characters. I was just trying
| to make a funny remark about "Single line request" vs "Multi-
| line request"
| varenc wrote:
| > It fundamentally changed the way I think about computers at
| every level
|
| I had the _exact_ same revelation also in the late 90s!
| (~1998 for me). I was already telnetting into servers a bunch
| and was getting into running an Apache server. I remember the
| moment I typed "GET / HTTP/1.1" into port 80 so clearly
| because it suddenly turned the "web" and "HTTP" into
| something comprehensible with composable parts I could
| understand at a deeper level.
|
| In our current world of SSH and HTTPS, it seems less likely
| the new generation will have the same experience. But we also
| have browser developer tools nowadays which make it so much
| easier and motivating for someone to start to learn the about
| the web and JavaScript. In the 1990s I had to use proxomitron
| to painstakingly modify a webpage's JavaScript or HTML, but
| these days it's dead simple.
| connordoner wrote:
| Incredible, really, isn't it?
| bheadmaster wrote:
| Assuming artificial superintelligence is possible, imagine how
| would its "experience" be in contrast to ours - it would be
| orders of magnitude faster in both thinking and execution. We
| would be like some weird plants to them, that take a year just
| to move from one location to another.
|
| Truly fascinating.
| nomel wrote:
| The craziest thing is, if you're using Ethernet, there's a very
| high chance you have an actual physical data connection with
| the other computer on the other side of the world. A giant
| physical web blanketing the Earth.
| vikingerik wrote:
| Well, that's not new with computers or the internet. We had
| that ever since the analog telephone system. Even telegraphs
| before that for a smaller set of points and routes.
| nomel wrote:
| I imagine we're somewhere near peak density though, or
| maybe even past it. Wireless will slowly remove your
| fingers from laying on that physical web.
| LK5ZJwMwgBbHuVI wrote:
| Why isn't wireless physical? Perhaps it's not massive :)
| foobiekr wrote:
| Really no. The RF world has nothing on fiber and copper
| for density.
|
| There is almost unlimited desire for bandwidth in the
| access layer and over subscription ratios are still very
| very high.
|
| The transition to 800G ports and then post-800G is not
| even warmed up yet. You can run the entire country of
| Australia on a backbone of two dozen routers today
| because of the lack of endpoint access bandwidth.
|
| We are at a place where bandwidth is once again plentiful
| in the core ... not so much on the edge. Sort of like
| 2003 when oc768 created that situation.
| bigdict wrote:
| In what sense? What's a physical data connection? There's a
| shit ton of routers and switches in between, it's not like
| there's an electrical connection.
| nomel wrote:
| You could run your finger across from my keyboard to yours,
| without a single break in the continuous physical
| connection between them. Connector clips included, of
| course. ;)
| bigdict wrote:
| I don't know, I alternate between thinking that's
| remarkable and not so much. I could run that connection
| via the power grid too, or the water supply (if we were
| in the same city).
|
| Edit: I concede it's actually pretty remarkable. It is
| like the nervous system of the planet. Sure there's not a
| purely electrical continuity, but neuron synapses don't
| have an electrical nature either.
| silviot wrote:
| Are you sure? As far as I'm aware a single city might
| have its electricity network split in a few macro areas,
| and not really connected to each other. The Internet, on
| the other hand, needs all nodes to be connected somehow
| (and I bet the vast majority of connections are
| physical).
| bigdict wrote:
| https://en.wikipedia.org/wiki/Synchronous_grid_of_Contine
| nta...
|
| https://en.wikipedia.org/wiki/North_American_power_transm
| iss...
| mikea1 wrote:
| I understand that you are being poetic, but just in case
| someone reads this as fact: you are describing a
| dedicated circuit - which is what telephones used. The
| internet works on packet switching, so there are numerous
| little breaks between the signal and receiver as your
| data is routed along a "connection".
| Humdeee wrote:
| Yes, and if the comment implied a purely electrical
| connection, it is likely not the case either, as there is
| electrical to optical and vice versa transitions
| throughout.
| nomel wrote:
| No, I'm talking about the physical layer of the OSI
| model, and including the mechanical connections between
| those physical interfaces. You're talking about the link
| layer.
|
| Unless your backbone/computer has a wireless hop, a
| literal, uninterrupted, physical chain of physical
| electronic devices, physically connected to one another
| with wires/cables, goes from my keyboard to yours. This
| is literal, not poetic. I'm not saying a galvanic
| connection. I'm saying a physical connection where, if
| nothing was bolted down, and high tension cables were
| used, I could pull and you would feel it.
| TechBro8615 wrote:
| Somebody might be running wireless in there at some
| point. But I agree it's a neat thought.
| can16358p wrote:
| Never though of it this way. Right!
| anamexis wrote:
| Similarly, every time you step onto a road, you're stepping
| on a contiguous strip of pavement that spans the entire
| continent
| Gare wrote:
| Even more remarkable with rail.
| kzrdude wrote:
| I often daydream about this one and wonder if there is
| continuous asphalt from me (Sweden) down to South East
| Asia or if there are any breaks.
|
| When I search for answers right now, AH1 looks like a
| candidate, but no confirmation
| https://en.wikipedia.org/wiki/AH1
| lelandfe wrote:
| My lifetime of road trips is just a really inefficient
| flood fill
| emielm wrote:
| This brings back memories of when I created a webbased version of
| spotify[1], using their (custom) encrypted protocol over TCP[2].
|
| Notably: back then I used flash to speak TCP, with a memory-leaky
| and bug-ridden mechanism to transfer byte buffers between JS and
| Flash.
|
| [1] https://news.ycombinator.com/item?id=2556118 [2]
| https://github.com/EmielM/spotifyontheweb-crypt
| swyx wrote:
| how did you reverse engineer a custom encrypted protocol? i
| wouldnt even know where to begin.
| bollwarm wrote:
| Maybe there should be a diagram to show the processes in the
| right
| globular-toast wrote:
| This reminds me of when I first discovered Wireshark (called
| Ethereal back then). It's always a good day when I have to break
| it out and still fun. I would try to reverse engineer the
| protocols. It's nice to see the annotations here as a lot of it
| doesn't make much sense without it.
| simonjgreen wrote:
| Not trying to diminish it's importance, however isn't it slightly
| sad how much total payload is devoted to encryption?
|
| Also, and I know this is slightly pedantic, but this is only
| speaking to layer 7. If you really wanted to watch everything
| byte by byte you'd need everything upwards of layer 2 where we go
| from bits on wire to bytes and frames. If you've not used
| Wireshark before run it yourself and load some pages.
| tomtomtom777 wrote:
| I don't think much is wasted on encryption. Most is wasted on
| _backwards compatibility_ in the form of negotiating protocols,
| algorithm, extenstions, etc.
|
| If there would forever be only one encryption algorithm in use
| we we wouldn't need most of the stuff.
| MichaelZuo wrote:
| Yeah, I'm pretty sure this is not the byte stream as seen by
| the ethernet port or Wifi/Cellular antenna.
|
| So it's quite a misleading title if the reader takes 'byte by
| byte' literally.
| hashhar wrote:
| Contrasting this with what happens over plain HTTP would be
| fascinating as well.
| gmac wrote:
| Well, the equivalent for plain HTTP is very short. Client sends
| the GET request, shown near the end, and server sends back the
| payload, same as you see decrypted.
| menus wrote:
| This was cool. Are there other tools/websites like this?
| demarq wrote:
| This is a really beautiful way to see something we take for
| granted all the time. The fact that computers can do all this in
| the blink of an eye (sans network speed) is just mind blowing.
| IanCal wrote:
| One kinda related thing I find constantly amazing is that your
| processor running at x GHz will have had several more ticks in
| the time it takes the photons to go from your screen to your
| eyes.
|
| (Light travels at about 30cm/1ft per nanosecond)
| runlaszlorun wrote:
| Was going to say the same. My mind was fairly blown when I
| realized that light only travels approx 4 inches in a clock
| cycle.
|
| Then again, every time my computer pauses for multiple
| seconds, me knowing how many billions of clocks cycles I'm
| waiting for doesn't help my already challenged patience...
| haha.
| xpil wrote:
| Not to mention eons required for our optical organics to
| start registering the photons!
| IanCal wrote:
| The race is about to begin. I wait for the screen to flash
| green, finger hovering over the enter key. It flashes and I
| press the button immediately, and the race is over.
|
| In this time, my year old consumer graphics card has
| performed more floating point calculations than my entire
| town could in their entire lives.
|
| Occasionally I work this stuff out and it's frankly hard or
| impossible to truly appreciate just how fast these things
| are.
|
| (36 TFlops, 250ms reaction time = 9e12 calculations. One
| calculation every four seconds from birth to age 100 for
| 10,000 people is about 8e12 in total)
| Arrath wrote:
| > The race is about to begin. I wait for the screen to
| flash green, finger hovering over the enter key. It
| flashes and I press the button immediately, and the race
| is over.
|
| And you missed your window to get into BIOS, you need to
| reboot and try again. /s
| ByThyGrace wrote:
| I'm fairly sure this is more than one somebody's
| recurring nightmare.
| SamBankmanFried wrote:
| And once you understand how _fast_ modern computers are,
| you realize how absurd it is when software is still slow.
| How is it that Microsoft Teams takes 15 seconds to load?
| How many billions of calculations does it take to put a
| list of names on the screen?
|
| Never forget what they took from you.
| tialaramex wrote:
| Although it can be laziness/ incompetence wasting cycles,
| sometimes the problem is that only the CPU got so much
| faster, and only in certain ways.
|
| If Teams talks to a remote server, doesn't matter how
| fast your CPU is, it takes time for the question to get
| to the server, time to get back, that's some time
| regardless. Now yes, maybe they write code which goes
|
| Hi! / Yes? / I'm a Teams client / That's nice / And you?
| / I'm the Teams server, what username? / SamBankmanFried
| / OK, and password? / Hunter2 / OK, you're logged in
| SamBankmanFried / OK do I have new #catpics ? / Nope /
| How about #emergency ? / Nope / How about #general ? /
| Yes, six messages ...
|
| And that's a lot of round trips whereas they could have
| done:
|
| Hi, I'm a Teams client, hopefully you're a Teams server /
| Yes, I'm the Teams server, what username & password ? /
| SamBankmanFried password Hunter2, also summarise my new
| stuff / Hello SamBankmanFried, you have six #general
| messages ...
|
| Which is far fewer round trips.
| goolz wrote:
| In game development we can abuse this fact with things like
| client-side prediction in authoritative online environs. The
| server plays along with the client in a no-graphics mode and
| plays the inputs from the client, validates said moves on the
| server, and if all goes well sends back the response allowing
| for client action all before the blink of an eye. In the case
| of an invalid move, the server yanks the client back to the
| state preceeding. Pair that with some lag compensation and
| you've got yourself a stew.
| csomar wrote:
| Do you have more detailed links about that?
| eXpl0it3r wrote:
| There's a great article by Valve: https://developer.valve
| software.com/wiki/Source_Multiplayer_...
| swyx wrote:
| what kind of latency budget do you have to do all that?
| even at like 15fps i cant imagine that you can provide a
| non janky experience when this "server yanks client back to
| valid state" move is done. i mean obviously you guys have
| figured it out, just the math doesnt quite make sense to
| someone like me not in gaming
| p-e-w wrote:
| My understanding is that this puts a fundamental limit on how
| fast CPU clocks can be. If the CPU die has a diameter of 3
| centimeters, and the clock speed is a hypothetical 20 GHz,
| light (and electrical signals) can traverse at most half the
| size of the die in the course of one clock cycle. It's hard
| to imagine how such a CPU could work, since components at
| opposite sides of the die cannot communicate within a cycle.
| HALtheWise wrote:
| Definitely nitpicking here, but for a modern pipelined CPU
| there's already lots of components of the die that aren't
| designed to be able to exchange data within a single clock.
| I don't see what would be fundamentally different about
| laying out a CPU die such that operations that need to
| happen in the same clock are physically nearby, and it
| seems like it would (at worst) have similar properties to
| multi-socket and pipelined CPUs today where cache coherency
| operations and branch mispredictions have multi-cycle
| penalties associated with them. I'm pretty sure that multi-
| socket CPUs can't talk in less than one clock cycle anyway.
| laci37 wrote:
| I only had an intro course in CPU architecture, but AFAIK
| pipelining depends on parts of the pipeline running on
| the same clock. I'd say CPU manufacturers already
| sidestepped this limitation by using multicore dies.
| foobiekr wrote:
| No, it doesn't. Putting aside that there is an entire
| field of clockless designs, asynchronous logic, free-
| running clocks, etc.
|
| You should think of this as "when the cpu is processing
| an instruction, what does it need to track to dispatch
| and retire it?" Designing to a click tick is one version
| and kind of where things were in the 1970s. It's an
| educational model.
| ByThyGrace wrote:
| And then you factor in the occurrence of different systems
| running at different clock speeds having to interchange across
| several interfaces in a way that the entire communication
| somehow stays afloat and even produces feedback.
| kulahan wrote:
| lol, this page did not work at all for me, and the only addon I
| have on firefox is ublock origin, which I disabled to no avail.
| Did anyone else find a workaround?
| sjnonweb wrote:
| working for me with ublock origin enabled
| codemanjoe wrote:
| Nice, super helpful for cs students.
| mvandermeulen wrote:
| Does anyone know what the NEL and Report-To headers are
| about/purpose?
| gmac wrote:
| https://developers.cloudflare.com/support/network/understand...
| er4hn wrote:
| Would this make it a TLS quine?
| xpe wrote:
| 84 76 68 82
|
| (ASCII for "TLDR")
| webprofusion wrote:
| This is great, would be useful to be able to point it at any
| domain.
| webprofusion wrote:
| For those looking for in-depth info on TLS, see also
| https://www.feistyduck.com/books/bulletproof-tls-and-pki/
| jxf wrote:
| This was beautiful. Two things stood out to me:
|
| * There's more stuff than I expected which exists only for
| backwards compatibility. That's a lot of bytes when you add it
| all up over the whole Internet.
|
| * There's a lot of "expect N more bytes for a data structure of
| type T1" messages followed immediately by "expect N-k more bytes
| for a data structure of type T1::T2". I assume this is because
| there's other stuff that could go in that spot, but it still
| looks strange.
|
| I'm sure all of this is necessary and important; I just found it
| really fascinating to peek under the covers. It's nice to be able
| to have it all (mostly) just work thanks to the tireless efforts
| of many engineers and protocol developers.
| bitexploder wrote:
| TLV encoding. It can be a pain to implement correctly.
| gmac wrote:
| Since I didn't know the acronym: https://en.wikipedia.org/wik
| i/Type%E2%80%93length%E2%80%93va...
|
| In fact, a key part of the subtls library is a simple class
| to make this relatively painless (it's currently just called
| `Bytes`, but perhaps something like `TLVCoder` would be a
| more informative).
|
| It reserves space for the length field, and provides a
| callback to write it automatically once you're done writing
| the value:
| https://github.com/jawj/subtls/tree/main#navigating-the-code.
| It also enables the indented and annotated output options.
| codeflo wrote:
| That kind of stuff is necessary for binary protocols to evolve
| in a compatible way. When a TLS 1.4 is defined, we need a
| window where clients and servers can still negotiate 1.3 until
| both have been upgraded. And 1.3 had to find ways to be
| compatible with 1.2, and so forth. Decades of that kind of
| evolution are guaranteed to leave some marks in the protocol.
|
| But let's keep a sense of proportion. I find it hard to worry
| about "wasting" maybe 0.01% of the total internet bandwidth for
| a few extra bytes here or there, when that's necessary to keep
| the internet working at all, when on the other hand, for no
| end-user benefit, we don't hesitate to waste maybe 15% (after
| compression) by insisting on using text-based formats for the
| payload in all our web standards.
|
| In fact, I'd love to see a back-of-the-envelope calculation of
| how many tons of CO2 would have been saved in total if HTML was
| a well-engineered binary format. (Including bandwidth, storage,
| parsing on the client etc.) The number must be insane.
| chx wrote:
| Don't forget to add a few billion dollars saved because of
| how easy it is to debug HTML. How easy to is for people to
| start. You press ctrl+u and see the source. There were not
| many platforms where you could just view the underlying
| instructions... when I grew up, some 8 bit machines had
| cartridges to hack but let's face it 6502 or Z80 assembly is
| nowhere near as friendly as HTML.
| stusmall wrote:
| ctrl+u could just make the displayed representation as easy
| to read. There are many binary protocols/formats that are a
| breeze to work with if you have the right tooling. You
| never see the 0xa 0x8 0xf. Your tools will either show you
| what the bytes represent or that there was a parsing error.
| Those parsing errors would be rare in the average case,
| just like debugging strange unicode issues in HTML today.
| foobiekr wrote:
| Most binary formats are easier, not harder, to parse.
|
| It's sad. An entire generation or two that doesn't know
| the basics.
| philsnow wrote:
| Exactly; when you click the padlock icon, the browser
| shows you the parsed representation of the x.509
| certificate chain, not the ASN.1 bytes.
| dylan604 wrote:
| >In fact, I'd love to see a back-of-the-envelope calculation
| of how many tons of CO2 would have been saved in total if
|
| *we didn't include massive JS libraries that are not truly
| necessary
|
| *we didn't track user's every move and report that data back
| maccard wrote:
| The thing about html is that it's a verbose text format on
| the surface, but it compresses incredibly easily, and support
| for gzip is widespread. If you're concerned about size,
| brotli is better again.
| codeflo wrote:
| > The thing about html is that it's a verbose text format
| on the surface, but it compresses incredibly easily, and
| support for gzip is widespread. If you're concerned about
| size, brotli is better again.
|
| That's why I said it wastes 15%, not 100%. Whenever text-
| based formats and binary formats are compared, the results
| after compression (of both) are usually in that range. You
| can debate those precise numbers, they might be lower for
| the brotli/HTML combination. That won't really change the
| point I was making in context: That a few extra bytes for
| backwards compatibility in the TLS handshake pale in
| comparison to the amount of waste we accept for encoding
| the payload.
| foobiekr wrote:
| The compression doesn't help memory usage or cache
| locality on the receiving end. Http is terrible and JSON
| and other web technologies are terrible. The whole
| concept of how to write efficient code and the resulting
| performance left on the table by terrible coders is
| enormous.
|
| A terrible waste.
| saiya-jin wrote:
| I dare to say that human readability that accounts for
| that extra size allowed many people to learn basics about
| how internet works, namely HTML and its compatriots.
| Mankind would be poorer for this enriching experience,
| even if it resides mostly in the past, and for me
| personally 15% (or 100% extra) could easily justify that.
|
| Plus add easiness of debugging this (before you bring
| some javascript monstrosity that hacks around it and will
| stop working probably in weeks after paid support of it
| ends)
| toomim wrote:
| A better example is HTTP, which is text, but read by
| humans much less frequently.
| tialaramex wrote:
| Only HTTP/0.9 HTTP/1.0 and HTTP/1.1 are probably text in
| the sense you mean
|
| HTTP/2 and HTTP/3 are binary formats, they are
| semantically very similar to the older formats in some
| sense, but you would benefit from more tools to examine
| them properly because human readability was not the
| priority.
| philsnow wrote:
| You're saying that compressed HTML, because it is text,
| wastes 15% (could be however much, the number doesn't
| really matter) over some binary format that would express
| the same content?
|
| What binary format would/could that be? I'm just not
| seeing how (if it expresses the same content) it could be
| smaller than compressed HTML.
| foobazgt wrote:
| They're comparing compressed text vs compressed binary,
| apples to apples. While text compresses amazingly well,
| binaries aren't 100% entropic themselves. They usually
| also benefit from compression. For example, the defacto
| "executable" format for the Java runtime is a compressed
| archive (jar file).
| Szpadel wrote:
| the thing is that you need additional CPU power to compress
| and decompress that data, maybe that is not much by today
| standards, but when accumulated it could be a significant
| number
| szundi wrote:
| The alternatives are not free either and would accumulate
| different costs probably as well
| tialaramex wrote:
| > That kind of stuff is necessary for binary protocols to
| evolve in a compatible way. When a TLS 1.4 is defined, we
| need a window where clients and servers can still negotiate
| 1.3 until both have been upgraded. And 1.3 had to find ways
| to be compatible with 1.2, and so forth. Decades of that kind
| of evolution are guaranteed to leave some marks in the
| protocol.
|
| It's necessary because people are incompetent, and because
| overall the market rewards them for incompetence. _From the
| outset_ TLS provided a trivial version negotiation mechanism
| but it was easier to ignore it and write incompatible garbage
| especially for so-called "Middle boxes" often sold as a
| drop-in security "solution" for businesses.
|
| So when it came time to ship TLS 1.1, it was soon discovered
| that in practice you can't just say "Hi, I speak TLS 1.1"
| which would be a couple of bytes - that won't work, you need
| to find some other way to quietly signal to competent people
| that you know the newer protocol. So they did, slightly
| weakening the security in the process, and this continued
| into TLS 1.2 where browsers began doing "Fallback" which was
| a risky but sadly necessary process where you give up
| attempting the new protocol altogether sometimes, thus
| opening yourself up to supposedly obsolete attacks.
|
| By TLS 1.3 things had become so bad that TLS 1.3 essentially
| begins, as you can see if you inspect the data shown on that
| page, by pretending we're speaking TLS 1.2 and then saying we
| want to negotiate an optional "extension" to TLS 1.2 which is
| where we confess we actually speak TLS 1.3.
|
| Every single packet of TLS 1.3 encrypted data is _also_
| wrapped in a TLS 1.2 layer saying "Don't mind me, I'm just
| application data". Why? Because we can't confess we're not
| speaking TLS 1.2, ever, and if we said we were doing TLS 1.2
| crypto system stuff the same incompetent garbage software
| would try to get involved because it "understands" (badly)
| how to speak TLS 1.2, so we just pretend it missed the
| negotiation phase, this is just application data, nothing to
| see.
|
| _And it works_. That 's crucial. It's why we did all this,
| and yet it reveals that because the products people bought
| were developed incompetently they wouldn't even have detected
| serious attacks anyway, let alone prevented them. Need to
| sneak 40GB of stolen financial data over a network
| "protected" by this Genuine Marketing Leading Brand Next
| Generation Firewall? Don't worry, just label it "application
| data" with no explanation and it'll be completely ignored.
|
| If you've ever watched a Lock Picking Lawyer video on
| Youtube, it was like one of the ones where it's several
| minutes so you expect it'll be hard to pick, but then you
| find out he's actually so disgusted by the lacklustre
| security of this $150 "Pick Proof High Security Lock" that
| although he rakes it open in 2 seconds with a cheap tool, and
| then shims it open with a discarded Redbull can, and then
| knocks it open with a hammer, and then uses a purpose built
| bypass tool to open it instantly in a single flowing motion,
| he _also_ takes time to disassemble it and show you that the
| manufacturer fucked up, wasting material solving a non-
| existent problem and in the process making the lock much
| worse, which is why the video was so long.
|
| Learning from their experience with these "Security" products
| for TLS 1.3, the QUIC people designed QUIC specifically with
| the intent that you can't even tell what version it is unless
| you're the client or the server, and then they shipped a new
| QUIC version to check that works even though they don't
| really need one yet, so that they don't have to do this whole
| dance again every few years.
| _hl_ wrote:
| While I mostly agree, I wouldn't attribute buying
| middleboxes (or building shitty middleboxes) to
| incompetence. This is about doing the minimum viable effort
| for compliance, not about actual security, and everyone who
| bought or sold these boxes knew that.
| MichaelZuo wrote:
| From what I understand of the parent's comment, there
| wasn't even the minimum viable effort for compliance,
| because the version checking system doesn't comply with
| the TLS standard.
| somehnguy wrote:
| In my opinion we should consider doing things just for
| compliance without any real benefit as incompetence. I
| know this isn't the reality we live in at the moment
| however. Way too many things are just box-checking and I
| find that frustrating.
| ChoHag wrote:
| If HTTP or HTML were binary formats 1000s of embryonic
| engineers would not have been able to learn them by hacking
| codes into telnet or notepad and the web would now be even
| more concentrated in the hands of the few players who are
| busily engaged in fucking it to death.
| MrGilbert wrote:
| > In fact, I'd love to see a back-of-the-envelope calculation
| of how many tons of CO2 would have been saved in total if
| HTML was a well-engineered binary format.
|
| I wonder if it is feasible to create something like that.
| Because, a binary format requires specialized tooling, which
| needs to be created and maintained.
|
| But maybe, with further adoption of WebAssembly, HTML will be
| less important in 10 - 15 years. :)
| codeflo wrote:
| You wonder if its feasible to create what, a binary format?
| wizzwizz4 wrote:
| > _In fact, I 'd love to see a back-of-the-envelope
| calculation of how many tons of CO2 would have been saved in
| total if HTML was a well-engineered binary format._
|
| Far, far less than would be saved if HTML was still expected
| to be human-readable. Things like React (server-side _and_
| client-side: think of all those poor <div>s!) waste far more
| resources than decompressing and parsing HTML.
| diginova wrote:
| As a Computer Science undergrad student we just finished our
| course on Computer Networks in Depth along with Cryptography and
| it feels really good to be able to at least know what is going on
| in every output step of the page. Amazing demonstration for
| beginners! Thank you for sharing
|
| btw I was thinking if we had a visual interface with the request
| details (as in local switch, router, which AS, DNS or host is the
| request at) then it might be even more intuitive and easy to
| learn/understand. I don't think the exact private position of the
| packet can be traced that easily on a web request, but if it can
| then it probably might already be out there somewhere.
| tonymet wrote:
| awesome work. It's a bit of intersection of instructional
| material, technical material and art.
|
| TLS debugging is infamous and this goes a long way to improve
| understanding. great work here
| amelius wrote:
| Since they add annotations to what is actually loaded, shouldn't
| the loading continue forever?
| laszlokorte wrote:
| If the annotations were added by the server, yes. Not if they
| are added client side.
| H8crilA wrote:
| This site is fetched twice, once by your browser, and another
| time by the JS code after you press the button. For example the
| JS loader specifies that only one cipher suite is to be used
| (TLS_AES_128_GCM_SHA256 ), while your browser will definitely
| support more than one. Any additional traffic is again fetched
| by your browser.
| JadeNB wrote:
| Revealing my non-nerd credentials: when I click "Get this page,
| byte by byte" I just see:
|
| > We begin the TLS handshake by sending a client hello message
| (source):
|
| and nothing else seems to happen, no matter how long I wait. I
| thought I might have to click the button again, but it just seems
| to display the same message, no matter how often I click it. What
| else should I be doing?
| gmac wrote:
| Huh, that's odd. Are you able to open your browser dev tools
| and check for JS errors in the console?
| JadeNB wrote:
| I tried in Firefox, Safari, and Chrome. Here it is from
| Chrome:
|
| > index.js:2231 WebSocket connection to
| 'wss://ws.manipulexity.com/v1?address=subtls.pages.dev:443'
| failed:
|
| > ws @ index.js:2231
|
| > index.js:2235 ws error: Event [I didn't expand this, but
| let me know if you need it]
|
| > connection closed index.js:2238
| gmac wrote:
| Hmm. Could you be behind some kind of interfering proxy
| that doesn't like WebSockets? Does this demo page work for
| you? https://libwebsockets.org/testserver/
| JadeNB wrote:
| > Hmm. Could you be behind some kind of interfering proxy
| that doesn't like WebSockets?
|
| I tried again from home, and it worked fine, so that must
| be it.
| gmac wrote:
| Great!
| JadeNB wrote:
| > Hmm. Could you be behind some kind of interfering proxy
| that doesn't like WebSockets?
|
| I'm at work, so it's possible, but I'm not sure.
|
| > Does this demo page work for you?
| https://libwebsockets.org/testserver/
|
| It shows me a rapidly incrementing number, that resets
| when I click "reset counter". Nothing obviously happens
| when I click "send junk". Is that what should happen?
| globalise83 wrote:
| Very good! This would be very helpful for students on every
| "intro to networks" course on computer science courses as a way
| to visualise something that can seem very abstract.
| electrosphere wrote:
| The Cisco Packet Tracer software is also good for this, as is
| Wireshark.
| captn3m0 wrote:
| The subtls hack is an amazing idea. I've been wanting to build a
| TLS Playground for a while(play around with CAs, Cert Issuance,
| Client Certs etc) and this would be great for it.
| gmac wrote:
| Subtls author and OP here. That sounds great -- feel free to
| ping me if useful! (Assemble these into an email address:
| george mackerron com).
| webprofusion wrote:
| Would be good to be able to point at any domain, a common
| issue with debugging cert stuff is working out which cert is
| being presented, TLS level, cipher suites etc are being
| negotiated.
| gmac wrote:
| That's true, but I was wary of running what would
| essentially be an open TCP relay. Also, I don't currently
| bundle a comprehensive set of root certs in subtls, nor
| support all ciphers or signing methods.
| waboremo wrote:
| Absurdly fascinating. I should probably stop reloading the site
| now!
|
| Also is there a reason why on HN this domain just shows itself as
| "pages.dev"? That's the root of Cloudflare Pages, so probably
| best to also show the subdomain for it like HN does for github.io
| dang wrote:
| We have to specifically mark a domain for sharding like that.
| I've done so for pages.dev now. Thanks!
| 8organicbits wrote:
| I wonder if its an out of date public suffix list? Thats how
| I'd decide when to show a subdomain. pages.dev was added a few
| years back:
|
| https://github.com/publicsuffix/list/pull/1093
| status200 wrote:
| As a network guy I am blissed out by this, thank you for sharing.
| Often when I study the concepts I repeatedly have that same shock
| of how much is going on under the hood across large distances at
| tremendous speed.
| runlaszlorun wrote:
| That's pretty neat. I'm all for things that make these systems
| more visual.
|
| I do chuckle that, for all of the advancements we've made with
| the web and 3D graphics, we're still mostly worried about
| stringing together little squiggly characters and writing test
| cases or print statements to see what's actually going on.
| BrandoElFollito wrote:
| Very nice, thank you.
|
| There is an older document (2009) which I liked very much and
| which taught me the details of a HTTPS connection (in a quite
| leisury way): http://www.moserware.com/2009/06/first-few-
| milliseconds-of-h...
| H8crilA wrote:
| I couldn't quite follow the symmetric key derivation.
|
| First there's something called "handshake key computations",
| which generates a full set of encryption and mac keys. It seems
| to be entirely based on secp256r1 (NIST P-256) key share. I
| understand that this part is not protected against man-in-the-
| middle, right?
|
| Then there's "application key computations", what are those
| values computed from? This process creates another complete set
| of encryption and authentication keys, and I assume this set is
| protected against man-in-the-middle.
| gmac wrote:
| You might find this useful (it was a helpful source when
| implementing): https://tls13.xargs.org/. Both sets of keys
| should be MITM-proof.
| H8crilA wrote:
| Thanks for the link.
|
| But I believe that the first set of keys/IVs is not protected
| whatsoever. Anyone can run the "Server Key Exchange
| Generation", as it doesn't require any secrets such as
| certificate private keys, only random inputs.
|
| EDIT: Specifically, at least up to this point anyone can
| impersonate the server.
| tialaramex wrote:
| Yes, maybe it's easier to understand what's going on once
| you understand the strategy in terms of a conversation.
| Suppose I'm Alice and I want to securely talk to Bob with
| TLS 1.3
|
| The key exchange (KEX) allows me to agree a secret S with
| somebody I'm talking to over the network. I don't know if
| they're Bob, but whoever they are, eavesdroppers can't
| discover this secret.
|
| Now, we (Alice, and the maybe Bob) can communicate securely
| with our secret S, nobody can MITM this conversation, they
| don't know S.
|
| Bob sends public documents (a certificate) which say Bob
| knows some Private Key such that I can verify it with a
| published Public Key mentioned in the certificate. That's
| no big deal, they're public documents, anybody could have
| those but then...
|
| Bob also sends a _transcript_. The transcript is of our KEX
| earlier, and it 's signed using Bob's private key, which
| means I, Alice, can verify this is a signed transcript of
| our conversation. I was in the conversation, so I can
| confirm it's identical.
|
| Having verified this signed transcript, I know that Bob was
| in the conversation I had earlier, with the unknown maybe-
| Bob, and it stands to reason that's because this is Bob.
|
| Notice that if Mallory, a hypothetical MITM, tried to
| attack us, this won't work. The conversation between me
| (Alice) and Mallory must involve parameters that Mallory
| chose, if Mallory just forwards Bob's parameters to Alice
| and vice versa, Mallory doesn't learn S and so is cut out
| of the picture. But, if Mallory changes the parameters
| Alice and Bob see, then the transcript that Bob signed
| won't match the KEX that Alice experienced, so the
| connection is aborted.
| H8crilA wrote:
| This makes a lot of sense.
|
| Slightly stupid question but maybe you know the exact
| reason: could we apply symmetric encryption only after
| Alice has determined that maybe-Bob is the real Bob? That
| is only after Bob sends the certs and the signed
| transcript of key exchange? I'm calling this a stupid
| question because I see no real advantage, but I wonder if
| there are clear disadvantages.
| tialaramex wrote:
| If the message from Bob, proving his identity, was sent
| in the clear, we're needlessly telling people that Alice
| is talking to Bob.
|
| In fact earlier TLS versions did send the _certificates_
| in the clear which is a lesser version of this problem,
| TLS 1.3 no longer does that, everything after "Hello" is
| encrypted.
|
| IETF Best Common Practice 188 (RFC 7258) "Pervasive
| Monitoring is an Attack" says that for Internet Protocols
| we should assume that surveillance is inherently an
| atttack on the Internet and pro-actively defeat it by
| designing protocols not to leak information
| unnecessarily.
|
| Now, in this specific case, the Hello message from Alice
| likely has SNI (Server Name Indication) so probably Alice
| is saying "Hi Bob" at the start in the clear, but (a) Not
| necessarily, in some cases this won't happen, so no need
| to tell us it's Bob later (b) There is ongoing work to
| fix that, Encrypted Client Hello, which will leverage DNS
| to give Alice keys she can use to make it harder to know,
| so there's no reason to make it _worse_.
| joelrwilliams1 wrote:
| The secret life of packets.
| vrglvrglvrgl wrote:
| [dead]
| TrackerFF wrote:
| I remember back in the day some software companies typically used
| to ask a question like _" What happens (under to hood) when you
| type in a URL in your web browser, and hit enter?"_ and you'd
| have to jog them through every step you could remember. This
| takes it to the next level.
| jpopesculian wrote:
| Very cool! Reminds me of the illustrated TLS connection which was
| a helpful resource for me when I had to implement a variation of
| it.
|
| https://tls12.xargs.org/
| lucidyan wrote:
| > https://tls12.xargs.org/
|
| in the first paragraph on the page there is this link:
|
| > It's inspired by [The Illustrated TLS 1.3
| Connection](https://tls13.xargs.org/) and Julia Evans' [toy TLS
| 1.3](https://jvns.ca/blog/2022/03/23/a-toy-version-of-tls/).
| FujiApple wrote:
| Great resource, thanks for sharing.
| jeroenhd wrote:
| I'm annoyed TLS 1.2 compatibility mode is still required. Is
| there a way to turn it off, accepting that people with
| incompetent network administrators can't access your website, or
| have shitty middleboxes forever ruined TLS?
| defluct wrote:
| You can definitely just require TLS 1.3 with no backwards
| compatibility.
| gammalost wrote:
| The problem is that you do not know where the middleboxes are.
| Removing support might therefore block users in weird and
| silent ways. The middlebox might be out of the users control
| and then nothing can be done to solve it. Blocking TLS 1.2 from
| a small blog is no big deal, but not something FAANG can risk
|
| One way of solving this is by circumventing the middleboxes.
| This is more or less the way QUIC works. QUIC uses UDP which
| avoid most middleboxes and their failed attempts at being
| helpful. So QUIC can without issue upgrade to a new version of
| TLS without middleboxes ruining it, while TLS over TCP is stuck
| (at least for the moment). This brings with it other issues,
| but hey one problem solved at least.
| jeroenhd wrote:
| If TLS 1.3 will fail, I doubt QUIC will make it through. I'm
| no fortune 100 sysadmin but if I were that worried about
| network traffic, I wouldn't allow some unknown UDP protocol
| to go out unnoticed.
|
| I don't know where these middleboxes are but this doesn't
| seem that important until people report having issues. It's
| been five years since TLS 1.3 was introduced and even longer
| since the survey that led to the compatibility mode was
| conducted, many companies with bad middleboxes will probably
| have had to upgrade them by now.
| gammalost wrote:
| It's not a problem of being paranoid about TLS. It's a
| problem with well intended middleboxes which worked at the
| time, but now do harm instead. It's easy to say that people
| should of updated by know. But that's hard to force when
| nobody fully knows what will break ( ipv4 to ipv6 for
| example).
|
| QUIC works better becuase UDP haven't gotten the same
| treatment as TCP and TLS. It brings with it other problems
| . One problem for example being NAT's sometimes poor
| handling of long lasting UDP connections. But QUIC has
| functionality to handle that.
|
| > I wouldn't allow some unknown UDP protocol to go out
| unnoticed
|
| A bit of a compromise has been made in this regard by using
| something called a spin bit.
|
| https://greenbytes.de/tech/webdav/draft-ietf-quic-spin-
| exp-l...
___________________________________________________________________
(page generated 2023-05-10 23:01 UTC)